Michael Limberger

Normal RAG is the straightforward pipeline. Search, stuff the top hits into context, generate. It works until it does not, and the next pages show where it quietly breaks.

It is also locked to that app, quality varies, it often runs in the cloud, and it has no standard interface. The RAG lives inside the app. That is what makes everything fragile.

How most people meet RAG

Most RAG today lives at the application layer. You open a chat app that has a document feature, you upload your files, you ask questions. The RAG happens inside that app.

This is real RAG. It works. It is also where almost everyone starts, including me. The point of this part is not to mock that approach. It is to show its shape clearly, so the cracks in part 8 and the alternative in part 9 make sense.

The shape of normal RAG

Step	Plain English
Ask	You type a question into the chat app.
Built-in RAG	The app has its own retrieval system bolted in. It searches your uploaded docs.
Answer	The app replies using whatever its internal RAG found.

It is convenient. It is one tool. For a lot of people it is the finish line, and that is fine.

What counts as a "normal RAG" app? Anything with a built-in document or knowledge feature. OpenWebUI's Knowledge feature. The cloud chat assistants with file upload. Most of the commercial AI tools. If the RAG is a feature inside the chat product, it is this category.

Locked to one app

The RAG lives inside that specific application. Your documents, your index, your retrieval, all of it belongs to that app. Switch to a different chat tool and the RAG does not come with you. You rebuild from scratch.

This sounds small. It is not. It means the work you do to build a good knowledge base is captured by the app. Vendor lock-in shows up in many forms. This is one of the quietest.

Quality varies

Every app implements RAG differently, and most hide the settings. You do not control chunking (how documents are split into searchable pieces), the embedding model (the part that turns text into searchable meaning-vectors), retrieval depth (how many passages get pulled in), re-ranking (whether the top results are reordered for relevance), or hybrid search (whether keyword search runs alongside meaning search).

You get whatever the app's developers chose, and it is often the "make it work everywhere" default, not the "make it good for your use case" choice.

Often cloud-based

Many built-in RAG features run their retrieval in the cloud. Your documents get uploaded to the vendor's servers, indexed there, and searched there. For anyone with PHI rules, NDAs, or any sensitive content, that is the exact problem from part 2 wearing a different hat.

Some built-in RAGs are local. OpenWebUI's is, which is why we use it as the example in the next part. The "often cloud-based" limitation applies to the category in general, not every member of it.

No shared standard

There is no shared standard for how app-layer RAG works. Each app is its own island. You cannot script it from the terminal. You cannot trigger re-indexing from a cron job. You cannot use the same knowledge base from another tool. You cannot inspect what got retrieved before it went to the model. You cannot swap the retrieval logic without forking the app.

Whatever you build inside the app stays inside the app, forever.

The RAG runs inside the app

If you take one architectural insight from this whole night, take this one. That is the whole problem.

Not because the app is bad. Because RAG-as-a-feature can never be RAG-as-infrastructure. A feature is something you use. Infrastructure is something you build on top of. You cannot reuse a feature. You can reuse infrastructure.

Once you see that distinction, the rest of the night reads as the consequences. Part 7 is the most popular built-in RAG, working. Part 8 is exactly where it breaks. Part 9 is what to do instead.

Feature versus infrastructure shows up everywhere in software, not just RAG. Authentication can be a feature in one app or a service many apps share. Search the same. Logging the same. The decision is always: how reusable does this need to be, and how separate from the consuming application?

A fair note on the built-in approach

For a hobbyist with one chat app and one set of docs, built-in RAG is genuinely the right answer. Less to run. Less to maintain. Works out of the box.

The argument for moving the RAG out gets strong when you have multiple knowledge domains (HR docs, technical docs, policies, each a separate brain), multiple tools that all want to ask questions of the same knowledge, automation needs (re-index from a script, evaluate retrieval quality in CI), or hard data-control requirements: PHI, IP, NDA, regulated content.

That is the Faculty of Dentistry's situation, and it is where the proxy earns its keep.

Ownership is the fragility

Built-in RAG works, but everything about it (your docs, your index, your retrieval logic) belongs to the one app. That ownership is what makes it fragile.