✦ RAGOps 90-Day Learning Path

What RAGOps means

RAGOps is the operational discipline for Retrieval-Augmented Generation systems in production. It covers the full lifecycle of a RAG pipeline: document ingestion, chunking strategy selection, embedding management, vector store operations and index refresh, retrieval quality evaluation, and observability for retrieval and generation components. RAGOps ensures that knowledge-grounded AI systems remain accurate as underlying data changes.

Who should follow this path

• AI engineers building enterprise knowledge assistants
• ML engineers operating production RAG pipelines
• Backend engineers integrating vector search into applications
• DevOps engineers supporting AI inference infrastructure
• Data engineers building document processing pipelines

Prerequisites

• Working experience with LLM APIs and Python
• Basic understanding of information retrieval concepts
• Familiarity with Docker and cloud storage
• Some knowledge of vector embeddings and cosine similarity
• Basic data pipeline experience

The 90-day plan

Daily study recommendation: 2-3 hours/day, six days a week. Consistency beats intensity — block the time in your calendar like a meeting.

Days 1–15: Foundation

• RAG architecture: naive RAG, advanced RAG, modular RAG patterns
• Document ingestion pipelines: PDF, HTML, Markdown, structured data
• Chunking strategies: fixed-size, recursive, semantic, late chunking
• Embedding model selection and trade-offs
• Vector database landscape: Pinecone, Weaviate, Qdrant, pgvector, Chroma

Outcome: Design and implement a document ingestion pipeline with a chosen chunking strategy and vector store.

Days 16–30: Core concepts

• Retrieval strategies: top-k, MMR, threshold-based
• Hybrid search: dense + sparse (BM25) fusion
• Reranking with cross-encoders (Cohere Rerank, BGE Reranker)
• Query expansion and hypothetical document embedding (HyDE)
• Metadata filtering and faceted retrieval

Outcome: Implement hybrid search with reranking and query expansion to measurably improve retrieval quality.

Days 31–45: Tools and workflows

• RAG evaluation with Ragas: faithfulness, context recall, precision
• ARES and TruLens evaluation frameworks
• Building a golden QA evaluation dataset
• Ablation studies: chunk size vs retrieval quality
• Automated regression testing for retrieval pipelines

Outcome: Build a Ragas evaluation pipeline that scores faithfulness and context recall on every pipeline change.

Days 46–60: Hands-on projects

• Index refresh pipelines: incremental vs full rebuild strategies
• Change detection and document update tracking
• Embedding drift when switching embedding models
• Multi-tenant vector store isolation patterns
• Access control in vector databases

Outcome: Implement an automated index refresh pipeline with change detection and tenant isolation.

Days 61–75: Advanced practices

• Production RAG observability: latency, retrieval quality, token cost
• Tracing with Langfuse and Arize Phoenix for RAG steps
• Failure mode analysis: retrieval misses, hallucinations, latency spikes
• Scaling vector stores: sharding, replication, HNSW parameters
• Cost optimization: embedding cache, query deduplication

Outcome: Deploy end-to-end RAG observability with per-trace scoring and cost attribution.

Days 76–90: Portfolio, interview & certification prep

• RAGOps portfolio project: enterprise knowledge assistant
• Interview preparation for AI engineer roles
• RAGOps interview questions
• Metrics: retrieval recall@k, faithfulness score, TTFT, cost-per-query
• Emerging: multi-modal RAG, agentic RAG, graph RAG

Outcome: Ship a production-grade RAG application with full eval coverage and be ready for AI engineer interviews.

Weekly outcomes at a glance

Phase	Outcome
Days 1–15	Design and implement a document ingestion pipeline with a chosen chunking strategy and vector store.
Days 16–30	Implement hybrid search with reranking and query expansion to measurably improve retrieval quality.
Days 31–45	Build a Ragas evaluation pipeline that scores faithfulness and context recall on every pipeline change.
Days 46–60	Implement an automated index refresh pipeline with change detection and tenant isolation.
Days 61–75	Deploy end-to-end RAG observability with per-trace scoring and cost attribution.
Days 76–90	Ship a production-grade RAG application with full eval coverage and be ready for AI engineer interviews.

Tools to learn

• LangChain
• LlamaIndex
• Pinecone
• Weaviate
• Qdrant
• Ragas
• Langfuse
• Arize Phoenix
• Cohere Rerank
• pgvector
• Hugging Face Transformers
• Apache Airflow

Labs to practice

• LLM RAG Application Basics
• MLOps Pipeline Basics
• Docker Build and Push

Mini projects

• Build a production RAG system with hybrid search, Cohere reranking, and Ragas evaluation CI pipeline
• Implement an automated index refresh pipeline tracking document changes in S3 and triggering incremental re-embedding
• Create a multi-tenant RAG platform with per-tenant isolation, access control, and per-query cost tracking in Langfuse

Interview questions to prepare

1. What are the trade-offs between different chunking strategies for RAG?
2. How does hybrid search improve over pure dense retrieval?
3. What is HyDE (hypothetical document embedding) and when would you use it?
4. How do you evaluate retrieval quality in a RAG pipeline?
5. Explain the difference between faithfulness and context recall in Ragas.
6. How do you handle index updates when source documents change frequently?
7. What HNSW parameters affect recall and query latency in a vector store?
8. How would you design a multi-tenant RAG system with data isolation?

Certification suggestions

• Hugging Face NLP Course Certificate — Hugging Face
• Databricks Generative AI Fundamentals — Databricks
• AWS Certified Machine Learning Specialty — AWS
• DeepLearning.AI LLMOps Specialization — DeepLearning.AI / Coursera

Browse the full certification registry for exam details and official links.

Free resources

• Ragas Documentation
• LlamaIndex Documentation
• Weaviate Academy (free)
• Langfuse Documentation
• Pinecone Learning Center

Related roadmaps

• LLMOps Engineer Roadmap
• MLOps Engineer Roadmap
• DataOps Engineer Roadmap

Related tool categories

• LLMOps Tools
• MLOps Tools
• DataOps Tools
• Workflow Orchestration Tools