RAG for beginners: build a Python retriever

What is retrieval-augmented generation?

Retrieval searches documents to find passages related to a question. Generation asks a language model to compose an answer using those passages. Evaluation checks if the retrieval was relevant and the answer was supported. A RAG app can still make mistakes even when retrieval succeeds.

This tutorial implements only retrieval, not the language-model part. Our output is a list of matching source passages, not a synthesized answer. That makes the first component easy to inspect and debug before connecting an AI model.

Build a small document retriever in Python

You need Python 3 and a text editor. Familiar with dictionaries, functions, loops, and tests? Continue below. Otherwise start with your first Python project and practice 10 beginner exercises.

Save this code as retrieval_demo.py. It compares useful words from a question to words in a collection of short documents. The score counts overlapping unique words; higher scores come first.

import re

DOCS = {
    "Python": "Python supports functions, loops and dictionaries.",
    "Retrieval": "Retrieval finds passages from documents for context.",
    "Evaluation": "Evaluation tests if answers match evidence.",
    "Deployment": "Deployment puts an application in front of users.",
}

STOP = {"a", "and", "do", "for", "from", "how",
        "i", "if", "in", "is", "of", "the", "to", "what"}

def terms(text):
    return set(re.findall(r"[a-z0-9]+", text.lower())) - STOP

def search(question, limit=2):
    found = []
    for title, passage in DOCS.items():
        score = len(terms(question) & terms(title + " " + passage))
        if score > 0:
            found.append((score, title, passage))
    found.sort(key=lambda item: (-item[0], item[1]))
    return [(title, text) for _, title, text in found[:limit]]

if __name__ == "__main__":
    for title, text in search("What is retrieval from documents?"):
        print(f"{title}: {text}")

Run python retrieval_demo.py (or python3 retrieval_demo.py). The top hit should be Retrieval. In Python, the & set operator finds common elements. The snippet uses only the standard library; the HTML entities above render as ordinary Python operators when copied from the page.

Run four repeatable retrieval tests

Create test_retrieval.py beside your first file. Run it with python test_retrieval.py. A failing assertion gives you something specific to investigate.

from retrieval_demo import search

assert search("What is retrieval?")[0][0] == "Retrieval"
assert search("Deployment for users")[0][0] == "Deployment"
assert search("unicorn banana spaceship") == []
assert len(search("Python dictionaries", limit=1)) == 1

print("All four retrieval tests passed.")

Next try five questions of your own and write down the correct document title for each before running the program. Count how often your expected document appears among the top two results. This miniature test measures your chosen example questions only; it does not prove search quality on real production data.

Where simple keyword retrieval goes wrong

• Synonyms: “launch my app” may fail to match “deployment”, even though the ideas are related.
• False matches: sharing words does not mean a passage supports the requested fact.
• Privacy: a real search service must filter out documents users are not allowed to access.
• Source tracking: our sample lacks stable document IDs, dates, permissions and citation URLs.
• No generation: our code does not run an LLM, create a natural-language answer or become a complete RAG chatbot.

Calling this tiny example semantic search, a vector database, or a full RAG system would be misleading.

From a beginner retriever to a real RAG application

Start by splitting larger documents into useful passages, storing stable source identifiers, and enforcing permissions. Evaluate retrieval with questions whose correct answers are known. Then compare a keyword baseline with BM25, embedding-based retrieval, or hybrid search; a more complex system is not automatically better.

If you later connect a language model, send only relevant and permitted passages. Ask it to distinguish supported statements from unknowns, and verify citations instead of blindly trusting the response. Treat retrieved text as untrusted data: a document may contain instructions intended to manipulate a model. Finally, monitor answer correctness, latency, token costs and changes in the underlying documents.

That is the beginning of the engineering discipline described in the full AI engineering roadmap, not a shortcut to production readiness.

What should you build next?

Replace the example notes with six of your own study notes and rerun your questions. Then build a rule-based FAQ chatbot and compare exact-match answers with document retrieval. Explore other small beginner projects before starting model APIs.

For language details, consult the official Python regular-expression documentation and Python data structures tutorial. This free website guide is available now; the Lumio mobile app remains in development. Continue learning with free seven-day Python-for-AI practice plan.