File Search vs Custom RAG: When to Use Which

Objectives

• Understand OpenAI’s managed RAG (File Search)
• Compare File Search with a custom RAG pipeline
• Choose the right approach for your use case

File Search: OpenAI’s Managed RAG

You have just built every layer of a RAG pipeline, lesson by lesson. File Search does all of this for you. It is a tool built into the OpenAI Responses API that automatically handles chunking, embedding, vector storage, and search: you upload your files, and you query. This lesson is not a rejection of what came before, but an honest question you must ask before writing any code: what is the right level of abstraction for your project?

Everything starts with a vector store, the container that hosts your indexed documents. You create it, push files into it one by one, and then wait for indexing to finish before querying. This waiting loop is not cosmetic: indexing is asynchronous, and a query launched too early will see only part of the corpus.

from openai import OpenAI

client = OpenAI()

# 1. Create a vector store
vector_store = client.vector_stores.create(
    name="product-documentation"
)
print(f"Vector store created: {vector_store.id}")

# 2. Upload files
files = ["user_guide.pdf", "faq.md", "changelog.txt"]

for file in files:
    with open(file, "rb") as f:
        file_obj = client.files.create(file=f, purpose="assistants")
        client.vector_stores.files.create(
            vector_store_id=vector_store.id,
            file_id=file_obj.id
        )
        print(f"  Uploaded: {file}")

# Wait for indexing to complete
import time
while True:
    vs = client.vector_stores.retrieve(vector_store.id)
    if vs.file_counts.completed == vs.file_counts.total:
        break
    time.sleep(2)
print("Indexing complete")

Querying then takes a single call: you declare the file_search tool with the store ID, and the model decides when to retrieve information. The interesting part is reading the annotations, which contain the file citations. This lets you display sources to the user—the behavior you had to implement manually in the custom pipeline.

# Query with File Search
response = client.responses.create(
    model="gpt-5.6-terra",
    input="What are the new features in v3.2?",
    tools=[{
        "type": "file_search",
        "vector_store_ids": [vector_store.id]
    }]
)

print(response.output_text)

# Access cited sources
for annotation in response.output:
    if hasattr(annotation, "annotations"):
        for ann in annotation.annotations:
            if ann.type == "file_citation":
                print(f"  Source: file {ann.file_citation.file_id}")

Two settings let you fine-tune behavior without leaving the managed tool. max_num_results sets the number of chunks retrieved, and score_threshold filters out low-scoring results, exactly like the relevance threshold you applied after reranking.

response = client.responses.create(
    model="gpt-5.6-terra",
    input="How do I configure SSO?",
    tools=[{
        "type": "file_search",
        "vector_store_ids": [vector_store.id],
        "max_num_results": 10,         # Number of chunks to retrieve
        "ranking_options": {
            "ranker": "auto",           # Automatic ranker
            "score_threshold": 0.3      # Relevance threshold
        }
    }]
)

What File Search Gives You, and What It Takes Away

The decisive argument for File Search is the total absence of infrastructure: no vector database to provision, back up, or monitor. Chunking is done for you and optimized by OpenAI, updating the corpus is as simple as adding or removing a file, citations are generated automatically, and common formats are supported natively—PDF, DOCX, MD, TXT, HTML, JSON, among others. A team with no infrastructure expertise can get a knowledge base online in an afternoon.

The flip side of this convenience is the loss of control over every link in the chain. You do not choose the chunk size, so the precision/context trade-off from Lesson 12 is out of your hands. Search is purely semantic: no hybrid search, so product references and exact error codes remain fragile. You also cannot plug in Cohere or a homegrown cross-encoder for reranking. Add to this three more strategic considerations: vector storage is billed in addition to API calls, your data resides on OpenAI’s servers, and indexing covers text only.

A custom pipeline inverts this list term for term. You master chunking, embedding, search, reranking, and prompting; you combine semantic and lexical search; you mix model providers, or keep everything on-premise if your data requires it; and above all, you optimize each component independently by measuring its effect. This control comes at the cost of code to write and maintain, a vector database to administer, and expertise—chunking, evaluation, and reranking are not improvised, which is precisely what this course covers.

Decision Matrix

Hybrid Architecture: The Best of Both Worlds

The choice does not have to be definitive, and it should not be. The most cost-effective approach is to start with File Search to validate that your use case holds water, then switch to a custom pipeline the day you hit an identified limit. Encapsulating both behind a single respond method makes this switch painless for the rest of your application.

class HybridRAG:
    """Start with File Search, switch to custom if needed."""

    def __init__(self, mode: str = "file_search"):
        self.client = OpenAI()
        self.mode = mode

    def respond_file_search(self, question: str, vs_id: str) -> str:
        response = self.client.responses.create(
            model="gpt-5.6-terra",
            input=question,
            tools=[{
                "type": "file_search",
                "vector_store_ids": [vs_id]
            }]
        )
        return response.output_text

    def respond_custom(self, question: str, pipeline) -> str:
        result = pipeline.respond(question)
        return result["response"]

    def respond(self, question: str, **kwargs) -> str:
        if self.mode == "file_search":
            return self.respond_file_search(
                question, kwargs["vs_id"]
            )
        else:
            return self.respond_custom(
                question, kwargs["pipeline"]
            )

When migration becomes necessary, it follows a precise order:

1. Export your files from the vector store
2. Implement chunking adapted to your content
3. Generate embeddings with text-embedding-3-large
4. Index in a vector database (Qdrant, Pinecone, etc.)
5. Add reranking and hybrid search
6. Evaluate with your test dataset
7. Compare metrics with File Search before switching

The last step is the one most often skipped, and it is the only one that justifies everything else: without a quantitative comparison, you may have replaced a decent managed system with an inferior homegrown one, plus the maintenance overhead.

Summary

• File Search is ideal for rapidly prototyping a RAG system without infrastructure
• A custom RAG pipeline offers total control and optimizable performance
• Start with File Search, migrate to custom when you hit limits
• Decision criteria: control, sensitive data, cost, performance