This chapter added reranking to the search pipeline. Reranking is a second pass over an initial set of search results.
The first search retrieves a larger group of likely matches using Reciprocal Rank Fusion (RRF). A reranker then looks at those candidates more carefully, sorts them again, and returns only the requested number of results.
The search process now has two stages:
When reranking was enabled, we retrieved five times the requested limit. For example, a limit of 5 meant retrieving 25 candidates before reranking them and returning the best 5.
BM25 is based on keyword matching. Semantic search uses embeddings and cosine similarity. Both are useful for finding likely matches, but the top results may still not be in the best order.
A reranker compares the query with the candidate documents after retrieval. Since it only processes a small candidate set, it can spend more work deciding which results belong at the top.
With the individual method, the LLM scored each candidate document in a separate request. Each prompt included the query and one movie, and the model returned a relevance score from 0 to 10.
This let the LLM consider things like direct relevance, user intent, and content appropriateness. The results were then sorted from highest score to lowest.
The main downside was making one LLM request per document. It was slower and could run into API limits.
With the batch method, all candidate documents were included in one LLM prompt. Instead of scoring each document separately, the model returned the movie IDs in relevance order as a JSON list.
This used fewer API calls and let the model compare every candidate in the same context. It also avoided assigning separate scores that might not be consistent with each other.