33.3 Parallelizing retrieval and preprocessing

The Serial Trap

A naive RAG app often does this:

1. Receive User Query
2. (Wait) Generate Embedding for Query
3. (Wait) Search Vector DB
4. (Wait) Fetch Full Documents
5. (Wait) Call LLM
6. Response

This "waterfall" kills performance.

Parallelizing the Pipeline

You can do many things at once:

• Speculative Retrieval: Start searching your docs as soon as the user stops typing (debounce), before they even hit enter.
• Parallel Chunks: If you need to summarize 5 documents, send 5 separate requests to the model in parallel (map-reduce), rather than asking it to summarize them one by one.
• Hybrid Search: Run your keyword search (Elasticsearch) and vector search (Pinecone) at the same time, then merge results.

Where to go next

• 33.4 Caching: what to cache and what not to