Blog

I’ve recently completed and passed the C10 LLM Foundations Certified PM certificate by The Product Compass.

This is the foundational module of The Product Compass’s AI PM Learning Program, designed to give product managers a solid understanding of large language models (LLMs) – both technically and strategically.

What the Certificate Covers

The C10 module focuses on LLM fundamentals – how these models work, where they fit in the AI landscape, and how they can be applied in product management.

The exam consists of 20 multiple-choice questions, and you need about 80% to pass.
After this foundation, the next level is C11 – LLM Practitioner, where the focus shifts to hands-on work such as building LLM-based products, implementing RAG systems, and fine-tuning models.

This credential verifies that you understand the core mechanics, potential applications, and management considerations of LLM-driven products.

Knowledge Areas Covered

Here are the main domains the certificate focuses on:

1. Fundamentals of LLMs and AI Technologies
Neural networks, transformers, self-attention, tokenization, and reinforcement learning. Understanding how LLMs fit into modern AI stacks.

2. Prompt Engineering
How to design and structure prompts, create templates, and optimize model outputs while managing limitations.

3. APIs and Integration Patterns
Connecting LLMs via APIs, handling completions, streaming, and tool calls. Emphasis on product implications and system integration.

4. Context Engineering, RAG, and Memory
Building Retrieval-Augmented Generation (RAG) workflows with vector stores, embeddings, and data chunking. Creating memory architectures for better model context.

5. Fine-Tuning and Model Adaptation
Covers SFT, RLHF, and other fine-tuning methods like DPO and ORPO. Focuses on when to use fine-tuned vs. general-purpose models.

6. Evaluation and Observability
How to monitor and measure LLM performance – from hallucination rate and bias to safety and efficiency.

Why It’s Worth Taking

What I liked about this certificate is that it moves beyond the hype and focuses on what actually matters when building or managing AI products – understanding not just how LLMs work, but how to integrate them responsibly and effectively.

It’s a great entry point before diving into the more practical C11 Practitioner level, which includes real-world applications and hands-on projects.

Here is a Summary of a new Guide coming from the Project Management Institute that is helping us as AI Technical Project Managers lead the transformation of organizations using AI.

Overview

This guide offers a strategic roadmap for integrating artificial intelligence (AI) into organizational operations, with a focus on the roles of the Project Management Office (PMO), project professionals, and Transformation Management Office (TMO). It provides practical strategies to drive AI adoption, manage change, and achieve measurable business value.

The Benefits of AI Adoption

• Enhanced decision-making – AI driven analytics can provide predictive insights that improve strategic decisions across organizations
• Increased efficiency and productivity – AI automates repetitive tasks such as data processing, reporting and project scheduling
• Enhanced customer experience – AI technologies like chatbots and personalized marketing enable companies to provide timely and relevant services
• Scalability – AI systems handle large-scale tasks more efficiently than manual processes. As companies grow, AI enables them to scale operations by automating tasks and providing analytics that facilitate expansion without requiring proportional increases in the workforce
• Improved risk management – AI powered risk management tools analyze past project performance data to predict potential risks, helping the organization mitigate issues before they arise
• Agility and innovation – AI helps organizations stay ahead by providing real-time insights into market changes, allowing them to react faster to evolving business environments
• Cost reduction – AI reduces costs by automating labor-intensive tasks, optimizing resource use and ensuring smoother workflows
• Talent augmentation – AI does not replace human talent but complements it. By handling tedious or time-consuming tasks, AI enables employees to focus on creative, strategic work. This augmentation of human capabilities is especially valuable for knowledge workers in industries like project management
• Better project outcomes – AI’s abilities to provide accurate forecasts, optimize resource planning and monitor performance contribute to better project outcomes
• Competitive advantage – Organizations that adopt AI early position themselves as industry leaders driving innovation and efficiency. AI adoption is increasingly seen as a critical factor for companies seeking a sustainable competitive advantage

Understanding the AI Adoption Life Cycle

Exploration and Strategy (Initiation)

• support leadership in defining the business case for AI
• conduct initial feasibility studies, market research and risk assessment
• identify key stakeholders
• evaluate the availability and quality of data
• need for specialized AI talent
• develop mitigation strategies for AI specific risks:
  • data bias
  • model accuracy
  • regulatory compliance
• Consider:
  • ethical considerations
  • integration of ML models into existing systems
  • impact of AI models on business models and operations

Planning and Design (Planning)

• ensure clear goals, timelines, budgets and success metrics
• define scope
• design project charters
• resource allocation
• create AI adoption roadmaps
• plan integration of data pipelines
• development of AI models
• continuous monitoring and refinement of these models

Pilot and Implementation (Execution)

• monitor AI project execution against predefined goals
  • track progress
  • manage resources
  • ensure AI projects are delivered on time and within scope
• ensure AI project is continuously evaluated for it’s impact on data quality and model performance
• ensure project plans include strategies for:
  • data validation
  • model tuning
  • integration of AI into existing systems

Evaluation and Optimization (Monitoring and Controlling)

• conduct post-implementation reviews that specifically assess the AI model’s accuracy, reliability and ethical implications of AI models
• gather feedback from users to understand the real-world impact of AI and monitor the return of investment (ROI) using AI analytics to predict future performance and identify areas for optimization
• adjust plans and strategies based on these insights to ensure continuous improvement

Scaling and Institutionalization (Closure)

• manage documentation, knowledge transfer and lessons learned

Data-Related Challenges

Data Quality

• establishing standards, metrics and reporting processes
• check data is clean, accurate and consistent

Data Accessibility

• data could be stored in various areas and proper shareholders should have access to it
• remove barriers to data flow across departments

Data Governance

• establish and implement data governance policies and standards ensuring data quality, security and compliance are maintained throughout the project life cycle

Data Privacy

• enforce data privacy policies and compliance frameworks ensuring all AI projects adhere to legal and regulatory requirements

Data Integration

• cross department data integration efforts

Technological Issues

Model Complexity

• manage the development, testing and tuning of complex models
• focus on the strategic implications of model complexity making sure they are scalable and aligned with future AI initiatives

Scalability and Performance

• ensure AI systems can grow without compromising performance
• optimize models to ensure they can handle increased loads, larger data sets without sacrificing efficiency

Business Strategy

ROI Measurement

• establish metrics, track performance and ensure alignment with business objectives and long-term goals
• track project-specific outcomes, costs and benefits

Prioritization

• set criteria for evaluating AI projects and align with strategic goals
• assess timelines, resources and stakeholder expectations, helping to ensure high-impact AI projects are executed first

Adopting the Plan-Do-Check-Act Method for Continuous Improvement

Kef Activities:

• Plan improvements
• Implement changes
• Check results
• Act on findings
• Review impact
• Iterate continuously

The AI project maturity model

Aligning With Organizational Goals

• Conduct a thorough analysis of the organization’s strategic objectives and business priorities
• Identifying specific AI initiatives that can directly support organizational goals
• Engaging with key stakeholders
• Developing a clear roadmap
• Establishing a process to monitor and adjust the AI strategy

Leading Cros-Functional Teams to Drive AI Adoption

• Establishing clear roles and responsibilities
• Facilitating effective communication
• Setting clear goals and milestones
• Promoting collaboration and knowledge sharing
• Managing stakeholder expectations
• Addressing conflicts and challenges
• Monitoring and adjusting the project plan

Managing Stakeholder Expectations and Communication Throughout the AI Life Cycle

• Identifying key stakeholders
• Establishing clear communication channels
• Defining expectations early
• Regularly updating and reporting
• Addressing concerns and feedback
• Educating and informing
• Managing change
• Celebrating successes

AI Vendor Management

• Selecting vendors
• Creating clear contracts and SLAs that outline scope, deliverables, timelines, performance metrics and payment terms
• Ensuring vendor tool compatibility
• Onboarding and integrating
• Regularly communicating and meeting
• Monitoring performance and feedback against the agreed-upon SLAs
• Managing risk
• Collaborating and knowledge sharing
• Establishing a contract renewal and exit strategy

Identifying and prioritizing AI Use Cases

• Conducting a business needs assessment
• Evaluating organization’s current AI capabilities
• Identifying potential use cases
• Scoring and prioritizing use cases
• Conducting feasibility studies for high-priority use cases
  • technical assessments
  • cost-benefit analyses
  • risk evaluations
• Creating a roadmap
• Securing stakeholder buy-in
• Monitoring and adjusting

Upskilling and Reskilling the Workforce

• Conducting skills gap analysis
• Developing training programs
• Providing continuous learning opportunities
• Creating mentorship programs
• Encouraging cross-functional collaboration
• Measuring training effectiveness
• Aligning training with business objectives

Conclusion

Similar to most publications coming from PMI this book is also “dry” and theoretical and is trying to cover many cases and therefore not being specific. However it gives a list of things that are to be taken into account when transforming the organization to implement AI of which I have taken out the most important here.

I would like to start by highly emphasizing the following course from Udemy: The Product Management for AI & Data Science Course

It is giving a great beginners all round picture about AI and AI projects. Here are some 5 minutes read takeaways to inspire you to check the course:

AI projects should be viewed as R&D projects

Unlike typical software projects where we are used to more exact Scope, Timelines and Cost expectation, with AI projects that is harder the case. What represent them much more are:

• uncertain timeline
• uncertain budget
• uncertain result precision

The areas where AI is used are:

• Natural language processing
• Computer vision
• Speech recognition and
• Robotics

Types of AI

Machine learning uses statistics.
Deep Learning uses artificial neural networks.

Deep learning is more expensive, so with little amount of data use ML as it is cheaper and gives equal results

Supervised learning – show the machine what you want it to do
Unsupervised learning – models learn on their own without the need oof labeled data by finding patterns and commonalities

Supervised learning works best when you know what the outcome would be.

Unsupervised learning works best when you don’t know what the outcome will be

Reinforcement learning is when you give the end goal and set of rules, but no labeled data

When should AI be used

• When a human expert can give response in few seconds
• When it is hard to define exact rules of the problem solving
• When it is easy to get examples of the desired behavior – get labeled data

The AI Canvas (questions to answer)

• PREDICTION – What do you need to know to make the decision?
• JUDGEMENT – How do you value different outcomes and errors?
• ACTION – What are you trying to do?
• OUTCOME – What are your metrics for task success?
• INPUT – What data do you need to run the predictive algorithm?
• TRAINING – What data do you need to train the predictive algorithm?
• FEEDBACK – How can you use the outcomes to improve the algorithm?

Data collection methods

Next to a proper model second most important thing for AI project to succeed is the data the we train the model with.
These are the data collection methods:

• Open data
• Company data
• Crowdsourcing labeled data
• New feature data
• Acquisition or purchased data

There was also introduced a concept of MVD which stands for Minimum Viable Data (needed to train the model in order to develop an AI product)

Managing AI projects

An example of Kanban board states for an AI project:

• Data Backlog – when Datasets are ready
• Data Processing – preparing and cleaning the Data
• Data Modeling – building Models with clean data
• Model Training – train and retrain models to reach requirements
• Testing – test the trained Model
• Deployment – deploying the Model on production environment
• Monitoring – monitoring Model performance
• Done – celebration 🙂

Data & AI team roles

• Data/AI Product Manager – determines which products to build
• Data Scientist – builds AI models and conducts advanced data analysis
• ML Data Engineer – develops and maintains the ingestion of data, data infrastructure, data warehousing and data pipeline

The End

I hope you enjoyed the summary of this course and that it has inspired to you to check it out. It was very well worth it for understanding the basics of AI projects.

Which course or learning material would you like to share with me?

Connect with me on LinkedIn and let me know in a message https://vladimirrodic.com/about-me/