run-llama / llama_index

What This Repo Claims

A data framework for building LLM applications with Retrieval Augmented Generation (RAG) — the pattern where an AI searches through your documents to answer questions instead of relying on training data alone.

The core promise: index your documents and query them with natural language in a few lines of Python.

38k stars. Used in production by companies building document-aware AI applications.

Two primary install paths:

pip install llama-index          # full install
pip install llama-index-core     # minimal core only

What RAG Actually Does

RAG solves a specific problem: LLMs don't know about your documents.

If you ask Claude "what does our README say?", it doesn't know. RAG fixes this by:

• 1. Chunking your documents into pieces
• 2. Converting chunks to vector embeddings
• 3. Storing embeddings in an index
• 4. At query time: finding the most relevant chunks
• 5. Feeding those chunks to the LLM as context

llama_index handles steps 1-4 automatically.

What I Tested

Environment:

• macOS, MacBook Pro 14-inch M4 Pro, 24GB RAM
• Python 3.13.5, pip 25.3
• llama-index 0.14.21
• Fresh macOS user — no prior Python environment

Test 1: In-memory documents

from llama_index.core import VectorStoreIndex, Document

documents = [
    Document(text="Python is a high-level programming language..."),
    Document(text="PostgreSQL is an advanced open source database..."),
    Document(text="Redis is an in-memory data structure store..."),
    Document(text="Docker is a platform for running containers..."),
    Document(text="Railway is a deployment platform..."),
]

index = VectorStoreIndex.from_documents(documents)
query_engine = index.as_query_engine()

response = query_engine.query("What database is good for caching?")
# → "Redis is a good database for caching."

response = query_engine.query("What tool helps run applications in containers?")
# → "Docker"

Both answers correct. Semantic retrieval working — "caching" correctly mapped to Redis without exact keyword match.

Test 2: Loading real files from disk

from llama_index.core import VectorStoreIndex, SimpleDirectoryReader

reader = SimpleDirectoryReader("./docs")
documents = reader.load_data()
# → Loaded 1 document from disk

index = VectorStoreIndex.from_documents(documents)
query_engine = index.as_query_engine()

response = query_engine.query("How do I deploy to Railway?")
# → "Run /plan-eng-review first. Then /feature-dev to 
#    build. Then /review and /qa before deploying to Railway."

response = query_engine.query(
    "What tool stops Claude Code from making assumptions?"
)
# → "andrej-karpathy-skills"

Both answers extracted correctly from the markdown file.

Same query interface as in-memory documents — SimpleDirectoryReader is the only addition.

Key Observations

Accurate retrieval: Both test sets returned factually correct answers drawn from the indexed content — not hallucinated.

Same interface for files and strings: Whether you pass Document(text=...) or load from disk with SimpleDirectoryReader, the query interface is identical. This is the core design win.

Uses OpenAI by default: The default embedding and LLM configuration calls OpenAI's API. An OPENAI_API_KEY environment variable must be set for the default setup to work. Tests were run with an existing API key.

This is the most important caveat for developers wanting a fully local setup — you need to configure a local LLM (e.g. Ollama) and local embeddings explicitly. The README documents this but the default path requires OpenAI.

What I Did Not Test

• Local LLM configuration with Ollama
• Persistent vector stores (Chroma, Pinecone, etc.)
• PDF, Word, and other document format loading
• Large document sets (100+ files)
• Streaming responses
• Agents and tool use

This review covers the core RAG pattern — index documents, query with natural language. The library has significantly more surface area than tested here.

Verdict: Solid

Install works. Core claim is real. Five lines of Python to index documents and query them with natural language — exactly as documented.

The OpenAI default is the one thing to know before you start. If you want fully local operation, configure Ollama as the LLM and a local embedding model explicitly. The docs cover this but it is not the default path.

Combined with chroma (local vector store) and ollama (local LLM), llama_index enables a completely local RAG pipeline with no external API calls and no recurring cost.

Worth installing if you are building anything that needs to answer questions about your documents.