RAG Explained: How to Chat with Your Blog Posts

If you read the posts on this blog, you can see a small box: "Ask about this article". You write a question and an AI answers using the content of the post. You can also ask it to recommend other related posts.

Many people ask me how this works. The technique has a name: RAG, which means Retrieval-Augmented Generation. It sounds complex, but the idea is simple. In this post I explain it step by step.

The problem RAG solves#

An LLM like Claude or GPT knows a lot of things. But it doesn't know:

• What I wrote on my blog last week.
• The internal documentation of your company.
• The data in your database.

You have two options to give the model this knowledge:

1. Train or fine-tune the model with your data. This is expensive, slow and difficult to update.
2. Give the relevant information in the prompt at the moment of the question.

RAG is option two. The name explains the three steps:

• Retrieval: find the pieces of information that are relevant to the question.
• Augmented: add these pieces to the prompt.
• Generation: the model generates an answer using them.

In simple words: before you ask the model, you search your documents, and you give the model the best results. It is like an open-book exam.

Why not send everything?#

A good question: "My blog has only 20 posts. Why not send all of them in every prompt?"

Sometimes this is a valid option! Modern models accept very long contexts. If your content is small, sending everything (with prompt caching to reduce the cost) can be the simplest solution.

But when the content grows, problems appear:

• Cost: you pay for every token in every request.
• Speed: long prompts are slower.
• Quality: with too much text, the model can miss the important part.

RAG sends only what is needed. For a single-post chat, I send the current post. For questions like "Which posts talk about testing?", I need retrieval across all posts.

Step 1: Split the content into chunks#

The first step happens when you publish a post, not when the user asks. You split each post into smaller pieces, called chunks.

Why? Because when you search, you want to find the exact paragraph that answers the question, not a whole 2,000-word article.

How big should a chunk be? There is no perfect number. Some practical rules:

• Between 200 and 800 tokens is a good starting point.
• Split by structure when possible. Markdown headings (##) are natural limits. A section about "retries" should be one chunk, not half of two chunks.
• Add a small overlap (for example 10–15% of the text) between consecutive chunks, so you don't lose ideas that cross the border.
• Add context to each chunk: the post title, the section heading and the URL. A chunk that says "This is why you need a timeout" is not useful without knowing the topic.

This is what one chunk looks like in my database:

{
  "post_slug": "how-to-integrate-an-llm-into-your-backend",
  "post_title": "How to Integrate an LLM into Your Backend",
  "section": "Retries: yes, but carefully",
  "category": "AI develop",
  "tags": ["llm", "reliability"],
  "content": "LLM APIs return temporary errors: rate limits..."
}

Step 2: Create embeddings#

Now we need a way to search by meaning, not only by exact words. If a user asks "How do I avoid paying twice for the same request?", I want to find the chunk about idempotency, even if the word "idempotency" is not in the question.

For this we use embeddings. An embedding is a list of numbers (a vector) that represents the meaning of a text. An embedding model converts text into this vector, for example 1,024 numbers.

The magic property is this: texts with similar meaning have similar vectors. "How to avoid duplicate payments" and "Idempotency-Key prevents double charges" will be close to each other, even with different words.

So, for each chunk:

1. Send the text to an embedding model (Voyage, OpenAI, Cohere, or a local model).
2. Save the vector next to the chunk.

Step 3: Store vectors#

You need a place to store the vectors and search them quickly. Options:

• PostgreSQL with pgvector. My favourite option for small and medium projects. You keep everything in the same database you already use.
• Dedicated vector databases like Qdrant, Weaviate or Pinecone. Useful when you have millions of vectors.
• SQLite with an extension, for very small projects.

With pgvector, the table can be very simple:

CREATE TABLE post_chunks (
    id         BIGSERIAL PRIMARY KEY,
    post_slug  TEXT NOT NULL,
    section    TEXT,
    content    TEXT NOT NULL,
    embedding  VECTOR(1024) NOT NULL
);

Step 4: Search when the user asks#

Now the user writes a question. The flow is:

1. Create an embedding of the question, with the same model you used for the chunks.
2. Find the chunks with the most similar vectors.
3. Take the top 3–8 results.

With pgvector, the query looks like this:

SELECT post_slug, section, content
FROM post_chunks
ORDER BY embedding <=> :question_embedding
LIMIT 5;

The <=> operator calculates the cosine distance. Smaller distance means more similar meaning.

Hybrid search: the secret ingredient#

Embeddings are great for meaning, but sometimes they fail with exact words: names of tools, error codes, acronyms like "CQRS". Classic keyword search (full-text search or BM25) is very good at this.

Hybrid search combines both and mixes the results. In my experience, this improves the quality a lot, and it is not difficult to add if you use PostgreSQL, because it has full-text search built in.

Step 5: Build the prompt#

Now we have the question and the best chunks. We build the prompt for the LLM. This is a simplified version of what I use:

You are an assistant for the blog "IA Backend Journey".
Answer the question using ONLY the information in <context>.
If the answer is not in the context, say you don't know.
When useful, recommend related posts using their title and URL.
Write in simple English (B1 level).

<context>
<chunk post="how-to-integrate-an-llm-into-your-backend" section="Retries">
...
</chunk>
<chunk post="use-idempotency-key-request" section="Store the original response">
...
</chunk>
</context>

<question>
How do I avoid paying twice when I retry an LLM call?
</question>

Some important details:

• "Use only the context" reduces invented answers (hallucinations).
• "Say you don't know" gives the model permission to not answer. Without this, models try to be helpful and invent things.
• Tags around chunks separate data from instructions and help with prompt injection.
• Post metadata (slug, title) lets the model recommend posts with real links.

If you want the answer in a fixed format (for example, an answer plus a list of recommended posts), use structured outputs. I explain this in Structured Outputs and Prompts as Code.

Step 6: Recommend related posts#

The recommendation feature is a nice extra. I use two sources of information:

1. The retrieved chunks. If the top chunks come from different posts, those posts are probably related to the question.
2. Metadata in the front matter. Each post has tags, category and a related_posts list that I write by hand. The model receives this list, so it can suggest posts even when the retrieval doesn't find them.

Manual links are simple but very effective. Not everything needs AI.

Keep the index updated#

When you publish or edit a post, the chunks and embeddings must be updated. In Laravel, I do this with an event and a queued job:

PostPublished event  →  ReindexPost job (queue)
   1. delete old chunks for this post
   2. split the new content
   3. create embeddings (in batch)
   4. insert new chunks

Doing it in a queue is important: creating embeddings calls an external API, and the editor should not wait. This is the same pattern I describe in How to Integrate an LLM into Your Backend.

Also, save the embedding model name with each vector. If you change the model one day, the old vectors are not compatible with the new ones, and you need to re-index everything.

Common problems#

The answer is wrong but the model sounds sure. Usually the problem is retrieval, not the model. Check which chunks were sent. If the right chunk is not there, the model can't answer well.

Chunks without context. A chunk like "Yes, this is the most important part" is useless alone. Always add the title and section.

Too many chunks. More context is not always better. Five good chunks are better than twenty mediocre ones.

No evaluation. You change the chunk size and you "feel" it is better. Feelings are not data. Create a small list of questions with the expected posts and measure. I explain how in Testing AI Features with Evals.

Summary#

RAG is a simple idea with many small details:

1. Split your content into chunks with context.
2. Embed each chunk to represent its meaning.
3. Store the vectors (pgvector is a great start).
4. Search with the question's embedding, ideally hybrid search.
5. Build a prompt with the best chunks and clear rules.
6. Generate the answer, and recommend related content.

Try it now: open the chat box on this post and ask "Which post explains idempotency?". You will see RAG working.