From One Tenant to a Thousand: Secure RAG Design

From One Tenant to a Thousand: Multi-Tenant AI and RAG Architecture

Scaling from one tenant to a thousand means designing an AI application so each customer’s information stays within the intended boundary. In a multi-tenant RAG product, the final answer alone does not show whether retrieval respected that boundary.

Choose an isolation model deliberately, then apply tenant filters and query-time permissions during retrieval. The isolation model organizes tenant data, while retrieval controls limit the chunks each identity can retrieve.

Quick answer

Choose shared tables, separate schemas, or separate databases based on the isolation requirements of your customers and the operational work your team can support. For RAG, test the retrieved chunks as well as the answer the model produces.

• Shared schema: tenant-scoped rows, where strict query scoping is essential.
• Separate schemas: logical separation, with migrations across tenant schemas to manage.
• Separate databases: stronger isolation, with more provisioning and operational work.

From one tenant to a thousand: choose the isolation model

Choose shared tables, separate schemas, or separate databases based on the isolation your customers need and the operations your team can run. The choice affects infrastructure monitoring, migration work, and operating complexity as the system grows.

What a tenant boundary must protect in an AI/RAG system

In this article, a tenant boundary separates one customer’s data from another’s. For a RAG application, the important test is whether retrieval stays within the boundary assigned to the requesting identity.

Option 1: shared database, shared schema with tenant_id and row-level security

In a shared-database, shared-schema model, tenant data lives in the same tables and is distinguished by tenant_id on each row. This is the cheapest and fastest model to build and operate, but it requires strict tenant scoping in every query. A missing scope in application or ORM logic can expose another customer’s data.

Option 2: one schema per tenant in a shared database

In a shared-database, separate-schema model, each tenant has its own schema within one database instance. It provides stronger logical separation than a shared schema while retaining shared infrastructure. The tradeoff is that migrations must run across every tenant schema, which becomes more difficult as tenant counts grow.

Option 3: one database per tenant

In a database-per-tenant model, each customer has dedicated infrastructure. This model provides the strongest isolation, but provisioning, backups, monitoring, and migrations must be handled for each tenant.

Decision table: isolation strength, operating cost, migration burden, scaling behavior, and suitable use cases

How tenant isolation relates to document access during retrieval

A tenant data model describes how tenant data is separated. For RAG, apply tenant filters and query-time permissions during retrieval to limit the chunks each identity can retrieve.

Implement shared-schema tenant isolation and trusted authorization

A tenant_id column is not enough on its own. Centralize tenant scoping in the data-access layer so the application does not depend on every individual query remembering a filter.

[!note] A tenant_id column distinguishes shared-schema rows, but strict tenant scoping is still required on every query. For RAG, use tenant filters and query-time permissions to limit the chunks available for retrieval.

Prerequisites and example project structure

Start with an identity provider, a database access layer, and database roles appropriate for your chosen policy design. Keep the boundary clear:

src/auth/  src/tenants/  src/repos/  db/migrations/

Step 1: model tenants, users, memberships, documents, and chunks with tenant_id

Put tenant_id on every tenant-owned table, including documents and chunks. Add foreign keys where your database supports them, plus indexes such as (tenant_id, document_id) for retrieval scope.

create table memberships (
  user_id uuid not null,
  tenant_id uuid not null,
  role text not null,
  primary key (user_id, tenant_id)
);

Step 2: authenticate the user and resolve tenant membership on the server

Resolve the requesting identity and the selected workspace in server-side request handling. Use membership data to determine whether that identity may access the workspace before continuing with the request.

Step 3: validate role or permission before issuing a retrieval request

Apply the relevant permission and tenant constraints at retrieval time. Where documents have different visibility rules, build the retrieval scope from those rules before searching for chunks.

Step 4: create a transaction-scoped trusted tenant context for row-level security

For a transaction-local context, issue the context-setting statement and the database operation it affects through the same transaction callback:

select set_config('app.tenant_id', $1, true);

In this illustrative pattern, the database operation must run inside the transaction that establishes the request context. A callback that only sets transaction-local context ends before a later operation can use it. Verify the behavior of your database, driver, and pool configuration before relying on this design.

Step 5: write RLS policies and repository methods that cannot accept a caller-supplied tenant ID

Use the context in the policy:

create policy tenant_chunks on chunks
using (tenant_id = current_setting('app.tenant_id')::uuid);

Repository methods can take an authorized request context rather than treating a client parameter as the source of tenant scope. Shared schemas need strict scoping on every query, and centralized access code helps make that requirement consistent.

How to verify that Tenant A cannot retrieve Tenant B rows

For the illustrated policy design, test direct document reads and chunk retrieval with separate tenant identities. Inspect the retrieved chunks as well as the final answer to confirm that tenant filters and query-time permissions are limiting retrieval.

Debugging: missing tenant context, incorrect policy predicates, service-role bypasses, and ORM queries outside the transaction

When debugging, check policy predicates, database-role behavior, and whether ORM queries run in the transaction that establishes request context. Add tests for absent or incorrect context so the resulting behavior is explicit and verified.

Step-by-step: enforce tenant context before retrieval

1. Authenticate the user and resolve the tenant associated with the request.
2. Validate that the user belongs to that tenant and has permission for the requested data.
3. Establish the tenant scope in server-side request handling after resolving the identity and membership.
4. Apply tenant filters and query-time permission constraints to database and retrieval operations.
5. Pass only authorized retrieved content to the model, and inspect retrieved chunks during testing.
6. Test with separate identities and synthetic tenant documents. Verify that each identity cannot retrieve the other's content.

[!tip] Treat the retrieval results as the boundary-test artifact: record which chunk identifiers each test identity receives without logging chunk text.

Implement schema-per-tenant and database-per-tenant paths

Stronger isolation is not free. Separate schemas add migration work, while dedicated databases require per-tenant provisioning, backups, monitoring, and migrations, as Mavani Solution’s Multi-Tenant SaaS Database Design: Shared, Isolated, or Hybrid explains.

Schema-per-tenant: prerequisites, tenant schema naming rules, provisioning flow, migration runner, and safe schema routing

Use one shared PostgreSQL instance and a server-owned tenant registry. Generate schema names from an internal ID, such as tenant_<uuid_without_dashes>, never from a customer name or request parameter.

create schema tenant_a1b2;
set search_path to tenant_a1b2, public;

Provision in order: create the registry row, create the schema, run migrations, then mark the tenant active. Resolve the schema only after authentication and membership checks.

Schema-per-tenant folder structure and execution-order code samples

db/migrations/  src/tenants/registry.ts  src/db/router.ts

const schema = await registry.activeSchemaFor(requestContext);
await db.transaction(async (tx) => tx.execute(`set local search_path to ${schema}, public`));

Don't interpolate an unvalidated schema name. Resolve a server-owned registry value that matches a strict generated-name pattern, then validate it before constructing an identifier. In the execution sequence, the schema-routing statement and the routed query must appear in the same transaction callback; setting a transaction-local search path by itself does not route a later query after that transaction ends.

Database-per-tenant: tenant registry, provisioning workflow, connection resolution, migration automation, backups, and monitoring

For a database-per-tenant design, automate database creation, migrations, backups, and monitoring before treating a tenant setup as complete. Database-per-tenant provides the strongest isolation model, but it is also the most operationally demanding.

Database-per-tenant folder structure and execution-order code samples

src/tenants/registry.ts  src/db/provision.ts  db/migrations/

const database = await registry.activeDatabaseFor(requestContext);
const client = poolFor(database);
await migrate(client);

Plan connection handling as part of the operating model, and monitor database usage as the number of tenant databases grows.

Expected results and debugging: connection pool exhaustion, failed fan-out migrations, missing tenant registry entries, and tenant routing errors

Test tenant routing with representative identities, record migration status per tenant, and investigate missing tenant configuration before serving requests. Automated provisioning and migration tooling are especially important when database operations must be repeated for every tenant.

When to use a hybrid model instead of moving every tenant to dedicated infrastructure

Use shared storage for most tenants and move only accounts with compliance, residency, or uneven workload needs. That avoids dedicated-infrastructure costs where they aren't worth it.

Build tenant-aware RAG retrieval without data leakage

For tenant-aware RAG, apply tenant filters and query-time permissions during retrieval so only permitted chunks are used in model input. The final answer alone cannot show whether retrieval respected those limits.

Retrieval sequence: identity to prompt assembly

Resolve the requesting identity and its permitted retrieval scope, then apply tenant filters and query-time permissions during search. Inspect retrieved chunks in testing before using them in model input.

Metadata contract for chunks

Carry access metadata on every chunk so retrieval can apply tenant filters and query-time permissions. The metadata should include the attributes needed by the access rules your application uses, such as tenant and document identifiers or visibility information.

Execution-order code samples

const scope = await authorizedDocuments(user, tenant);
const hits = await vectorSearch({ tenantId: tenant.id, documentIds: scope });
const safe = hits.filter(hit => hit.tenantId === tenant.id && scope.has(hit.documentId));
const prompt = buildPrompt(safe);

Handle an empty verified result set explicitly before prompt assembly. In a DevBrows post on multi-tenant RAG testing, the publisher recommends inspecting retrieved chunks before they reach the model.

Why client filters are untrusted

Apply tenant filters and query-time permissions during retrieval. Build the retrieval scope from the requesting identity and the access rules that apply to the requested content.

GigaRAG integration checklist

When connecting GigaRAG, document how agents reach its central Hive Mind through MCP or the API and configure its access controls for what each agent can use. Before writing integration code, verify the applicable MCP installation, authentication, API configuration, tool names, and knowledge boundaries in the official product documentation.

Connection validation and application responsibility

Test the integration connection with a low-risk request and document the access controls configured for it. Use the boundary tests below to validate how the surrounding application limits returned chunks for different identities.

Test tenant boundaries before you scale

Executable test cases

Use two test identities and synthetic documents assigned to separate tenants. Request retrieval under each identity, inspect the returned chunks before model use, and confirm that query-time permissions and tenant filters exclude documents assigned to the other tenant.

Concurrent-request test

Run Tenant A and Tenant B requests through the same pool at once, then repeat under load. Verify that each request uses its intended request-scoped configuration and inspect retrieval results for cross-tenant content.

What to log safely

Log tenant resolution, policy decision, retrieval count, and denied-access reason. Don't log chunk text, prompts, document titles, or credentials.

Final comparison reminder

Make cross-tenant retrieval tests part of the release process, not a one-time architecture check. Re-run them when changing policies, routing, connection pooling, migrations, or retrieval integrations, and inspect returned chunk identifiers for unexpected cross-tenant results.

Frequently asked questions

What is the safest starting architecture for a multi-tenant RAG application?

The safest starting architecture is the one that meets the required isolation level while your team can operate it reliably. A shared schema can suit high-volume SaaS when every query is strictly tenant-scoped; separate schemas provide stronger logical separation on shared infrastructure, while separate databases fit requirements that need the strongest isolation.

Why is a tenant_id column alone not enough to prevent cross-tenant access?

A tenant_id column distinguishes rows, but shared-schema access still depends on strict tenant scoping in every query. A missed filter can expose another tenant's data.

What is the difference between schema-per-tenant and database-per-tenant isolation?

Schema-per-tenant keeps tenant schemas within a shared database instance. Database-per-tenant gives each customer a separate database, with greater isolation and more per-tenant operational work.

Should every vector chunk include tenant and access metadata?

Yes. Carry the access metadata needed by your retrieval rules on each vector chunk so tenant filters and query-time permission checks can be applied during retrieval.

Why must authorization happen before RAG retrieval rather than only before showing the final answer?

A final answer does not reveal whether unauthorized documents were retrieved. Enforce permissions at retrieval time so disallowed chunks do not reach the model.

How do you test whether a RAG system can retrieve another tenant's documents?

In an authorized test environment, use two test identities and synthetic documents assigned to different tenants. Inspect each identity's retrieved chunks and confirm the other tenant's documents are excluded.

When should a SaaS team move a tenant from a shared schema to stronger isolation?

Consider moving a tenant when a requirement such as compliance or data residency calls for stronger isolation, or when tenant-specific usage makes the shared setup unsuitable. Plan and automate the migration path.

What should a tenant-aware RAG boundary test inspect?

Test with separate identities and synthetic documents assigned to different tenants. Inspect the chunks returned for each identity before model use, and confirm that documents assigned to the other tenant are excluded.

About GigaRAG

GigaRAG is a shared memory platform for AI agents that helps developers and teams keep project knowledge available across sessions and tools.

Agents can connect to a central Hive Mind through MCP or the API, with access controls that manage what each agent can use. Configure those controls alongside the surrounding application’s tenant model and retrieval tests.