Retrieval-Augmented Generation (RAG) - Pre-Quiz

Answer: B) Retrieval-Augmented Generation

RAG is a technique that combines information retrieval with language generation. It retrieves relevant documents/passages and uses them to augment the LLM’s response, grounding answers in factual sources.

Reference: Phase 8 - Introduction to RAG

Answer: B) LLM hallucinations and outdated knowledge

RAG addresses:

• Hallucinations: LLMs making up facts → RAG grounds responses in retrieved documents
• Outdated knowledge: LLMs trained on old data → RAG accesses current information
• Domain-specific needs: LLMs lack specialized knowledge → RAG retrieves from custom knowledge bases

Reference: Phase 8 - Why RAG?

Answer: B) Indexing, Retrieval, Generation

RAG Pipeline:

1. Indexing (Offline):

   • Split documents into chunks
   • Create embeddings
   • Store in vector database

2. Retrieval (Query time):

   • Embed user query
   • Find similar chunks
   • Retrieve top-k relevant passages

3. Generation (Query time):

   • Combine query + retrieved docs
   • Send to LLM
   • Generate grounded response

Reference: Phase 8 - RAG Architecture

Answer: B) A numerical representation of text in high-dimensional space

Vector Embedding:

• Converts text → array of numbers (e.g., 384, 768, or 1536 dimensions)
• Similar meaning → similar vectors (close in vector space)
• Enables semantic search (beyond keyword matching)

Example:

"dog" → [0.12, -0.34, 0.56, ..., 0.78]  # 384 dims
"puppy" → [0.15, -0.31, 0.54, ..., 0.81]  # Similar!
"car" → [-0.45, 0.67, -0.23, ..., -0.12]  # Different

Reference: Phase 8 - Embeddings & Vector Search

Answer: B) To break large documents into smaller, retrievable pieces

Why chunk?

• LLMs have token limits (can’t process entire books)
• Retrieve only relevant sections (not entire documents)
• Better precision (specific paragraphs vs. whole pages)

Chunking strategies:

• Fixed size: 500 tokens per chunk
• Sentence-based: Complete sentences only
• Semantic: Chunks by topic/section
• Overlap: Chunks share 50-100 tokens

Reference: Phase 8 - Document Processing

Answer: C) Cosine similarity

Cosine Similarity:

similarity = (A · B) / (||A|| * ||B||)

Range: -1 to +1

• +1: Identical direction
• 0: Orthogonal (unrelated)
• -1: Opposite direction

Why cosine?

• Measures angle, not magnitude
• Works well for high-dimensional text embeddings
• Efficient to compute

Alternatives:

• Dot product: Faster but magnitude-sensitive
• Euclidean: Can work but less common

Reference: Phase 8 - Similarity Search

Answer: B) A specialized database optimized for similarity search

Vector Databases:

• Store high-dimensional embeddings efficiently
• Perform fast approximate nearest neighbor (ANN) search
• Scale to millions/billions of vectors

Popular vector DBs:

• Pinecone (managed)
• Weaviate (open-source)
• Qdrant (open-source)
• Chroma (lightweight)
• FAISS (Facebook AI library)

Traditional databases can’t: Efficiently search “find most similar vectors” at scale

Reference: Phase 7 - Vector Databases

Answer: B) The maximum tokens the LLM can process

Context Window:

• Maximum tokens LLM can handle in one request
• Includes: system prompt + retrieved docs + user query + response

Example (GPT-3.5):

• Context window: 4,096 tokens
• System prompt: 500 tokens
• Retrieved docs: 2,000 tokens
• User query: 100 tokens
• Available for response: 1,496 tokens

Modern models:

• GPT-4: 8K-128K tokens
• Claude 3: 200K tokens
• Gemini 1.5: 1M tokens

RAG consideration: Must fit retrieved chunks within context window!

Reference: Phase 8 - LLM Context Limits

Answer: B) The number of documents to retrieve

Top-k Retrieval:

• Retrieve the k most similar chunks
• Common values: k = 3, 5, or 10

Example:

# Query: "What is RAG?"
# Vector DB contains 10,000 chunks
# top_k = 5
 
results = vector_db.search(query_embedding, top_k=5)
# Returns 5 most relevant chunks

Trade-offs:

• Small k (3): Fast, focused, but might miss context
• Large k (10): More context, but slower and noisier

Reference: Phase 8 - Retrieval Strategies

Answer: B) RAG retrieves external knowledge; fine-tuning updates model weights

When to use:

• RAG: Frequently changing data, need citations
• Fine-tuning: Specialized behavior, consistent style
• Both: Often combined for best results!

Reference: Phase 8 - RAG vs Fine-Tuning