LlamaIndex Tutorial: Getting Started the Right Way

Install LlamaIndex, build a VectorStoreIndex over your documents, query it, and add an agent with document search. Hosted and local Ollama paths covered.

By Galactic News Editorial · Published 2026-11-11T13:20:33+00:00 · Updated 2026-11-11T13:20:33+00:00

LlamaIndex Tutorial: Getting Started the Right Way

This LlamaIndex tutorial takes you from install to a working RAG agent. You will load documents, build a vector index, ask questions over your own data, and add an agent that can search those documents. The steps follow the official LlamaIndex docs. Two paths are covered: hosted models with OpenAI, and local models with Ollama.

What LlamaIndex Does

LlamaIndex is a data framework for building LLM applications. It connects custom data sources to large language models. It provides tools for ingesting, indexing, and querying data. The result is an application that answers from your documents, not just from training data.

The core workflow has three stages. First, you load documents. Second, you build an index over them. Third, you query the index. A query engine handles retrieval and answer synthesis for you. Agents add a further step: they choose tools, including your index, to answer a question.

LlamaIndex suits RAG projects. If your app must answer questions about private notes, manuals, or research, it fits. If you only need a chatbot with no custom data, a plain model call is simpler. Pick the tool for the job.

LlamaIndex Tutorial: Install and Set Up

LlamaIndex is a collection of namespaced Python packages. The main package is a starter bundle. Extra integrations install separately as needed. Follow these steps.

1. Check your Python version. Use Python 3.10 or newer. Run python --version to confirm.
2. Create a virtual environment. Run python -m venv venv, then activate it. Isolated environments prevent package conflicts.
3. Install the starter bundle. Run pip install llama-index. The bundle includes llama-index-core, the OpenAI LLM integration, the OpenAI embeddings integration, and the file reader.
4. Set your OpenAI key for the hosted path. Run export OPENAI_API_KEY=your-key-here. The defaults use OpenAI models for text generation and embeddings.
5. Or install the local path instead. Run pip install llama-index-llms-ollama llama-index-embeddings-huggingface. This path needs Ollama installed, plus the llama3.1 model pulled with ollama pull llama3.1.
6. Verify the install. Run python -c "import llama_index; print(llama_index.__version__)". A version number means the install worked.

By default, LlamaIndex uses the gpt-3.5-turbo model for text generation and text-embedding-ada-002 for embeddings. For new projects, the docs recommend passing model="text-embedding-3-small" explicitly. Defaults change over time, so set your models in code rather than relying on them.

Build Your First Index and Query Engine

Five lines of LlamaIndex build a working RAG pipeline. Load documents, build the index, create a query engine, and ask a question. The code below uses the hosted OpenAI path.

from llama_index.core import VectorStoreIndex, SimpleDirectoryReader

documents = SimpleDirectoryReader("data").load_data()
index = VectorStoreIndex.from_documents(documents)
query_engine = index.as_query_engine()
response = query_engine.query("What did the author do in college?")
print(response)

Each line has a clear job. SimpleDirectoryReader loads every file in the data folder into Document objects. VectorStoreIndex.from_documents splits the documents into nodes, embeds them, and stores the vectors in memory. as_query_engine builds an engine that retrieves relevant nodes and synthesizes an answer. query runs the whole flow.

Start with a small folder of text files. A single essay works well for a first test. The official docs use a Paul Graham essay as the example corpus. Add more files only after the small case works end to end.

You can tune retrieval with similarity_top_k. It controls how many nodes feed the answer. A small value keeps answers focused. A larger value adds context but risks noise. Try a value of 3 to 5 first, then adjust.

Run LlamaIndex with Local Models

Local models keep your data on your own machine. The official starter tutorial uses Ollama for the language model and a Hugging Face embedding model. It needs a machine with at least about 32GB of RAM. The embedding model is BAAI/bge-base-en-v1.5. The language model is llama3.1 served through Ollama.

from llama_index.core import Settings
from llama_index.llms.ollama import Ollama
from llama_index.embeddings.huggingface import HuggingFaceEmbedding

Settings.llm = Ollama(
    model="llama3.1",
    request_timeout=360.0,
    context_window=8000,
)
Settings.embed_model = HuggingFaceEmbedding(
    model_name="BAAI/bge-base-en-v1.5"
)

Settings holds global defaults for the whole app. Set the LLM and embedding model once, and every index and query engine uses them. This avoids passing the same models into each call. It also makes switching providers a one-line change.

Local models run slower than hosted ones on most machines. That is normal. The trade is privacy and cost control against speed. For learning and small corpora, local models are a good fit.

Build an Agent That Can Search Your Documents

An index answers questions. An agent decides how to answer them. The official starter tutorial builds an agent with two tools: a calculator and a document search tool. The agent picks the right tool for each question. Follow these steps.

1. Define a simple tool. A plain function with a clear docstring works. The docstring tells the model when to use the tool.
2. Build a search tool from your index. Wrap the query engine in an async function that returns the answer as a string.
3. Create the agent workflow. Pass both tools and your LLM to the agent. Give it a system prompt that describes what it can do.
4. Run it with a mixed question. Ask something that needs both tools, such as a question about your documents plus a calculation.

import asyncio
from llama_index.core import VectorStoreIndex, SimpleDirectoryReader, Settings
from llama_index.core.agent.workflow import FunctionAgent
from llama_index.llms.ollama import Ollama
from llama_index.embeddings.huggingface import HuggingFaceEmbedding

Settings.embed_model = HuggingFaceEmbedding(model_name="BAAI/bge-base-en-v1.5")
Settings.llm = Ollama(model="llama3.1", request_timeout=360.0, context_window=8000)

documents = SimpleDirectoryReader("data").load_data()
index = VectorStoreIndex.from_documents(documents)
query_engine = index.as_query_engine()

async def search_documents(query: str) -> str:
    """Useful for answering questions about the documents."""
    response = await query_engine.aquery(query)
    return str(response)

def multiply(a: float, b: float) -> float:
    """Useful for multiplying two numbers."""
    return a * b

agent = FunctionAgent(
    tools=[multiply, search_documents],
    llm=Settings.llm,
    system_prompt="You are a helpful assistant that can multiply numbers and search through documents.",
)

async def main():
    response = await agent.run("What is 1234 * 4567?")
    print(str(response))

if __name__ == "__main__":
    asyncio.run(main())

The official example prints 5635678 for that multiplication. The agent chose the multiply tool, ran it, and returned the result. With the search tool added, the same agent can answer document questions too. It switches tools based on the question.

Agents can also remember earlier messages. The agent workflow keeps history in a Context object. Pass the same context to later runs, and the agent can answer follow-up questions such as "What is my name?" after you told it your name.

Save and Reload Your Index

Building an index takes time. Embedding every document on each run wastes it. Persist the index to disk after the first build. Load it back on later runs instead of rebuilding it.

# Save the index
index.storage_context.persist("storage")

# Later, load the index
from llama_index.core import StorageContext, load_index_from_storage

storage_context = StorageContext.from_defaults(persist_dir="storage")
index = load_index_from_storage(storage_context)
query_engine = index.as_query_engine()

Use the same embedding model when you reload. A different model produces different vectors, and the stored index will not match. Keep the model name in your project config so it stays consistent.

Common Mistakes to Avoid

Old LlamaIndex tutorials are a common trap. Class names such as GPTVectorStoreIndex and ServiceContext belong to early versions. Current code uses VectorStoreIndex and Settings. Check the import paths in any tutorial before you copy its code.

Bottom Line

Install the starter bundle, load a small set of documents, and build a VectorStoreIndex. Query it to confirm the pipeline works. Then persist the index, add an agent with a search tool, or switch to local models with Ollama. Each step builds on the last, and none of them needs a large codebase to start.