Tenant isolation testing: three gaps beyond IDOR

This is a scripted Python and curl experiment with downloadable source and responses. No AI agent or model ran the test.

Define tenant isolation and the test scope

Tenant isolation, also called multi-tenant data isolation, means a user from one customer organization, called a tenant, cannot read or change another tenant's data, even where code, storage and caches are shared. A cross-tenant data leak is the failure: a request authenticated as one tenant returns data owned by another. The simple form is one object read by ID, covered in authorization testing for APIs and the IDOR walkthrough. This post covers three places where the tenant check is often still missing after that read is fixed.

The OWASP Multi-Tenant Security Cheat Sheet names cross-tenant data leakage, tenant context injection and shared resource poisoning among its key risks. It says to treat client-supplied tenant identifiers as selectors only, and to include the tenant identifier in every cache key whose value varies by tenant. The export gap below is an object-level check, the class in OWASP API1:2023, Broken Object Level Authorization.

• Target: two short-lived HTTP servers bound to IPv4 loopback, 127.0.0.1, on ports the operating system chooses.
• Tenants and identities: tenants acme (users alice and amy) and globex (users bob and bea). The bearer values, such as demo-acme-alice, are public demo strings.
• Data: four synthetic invoices held in memory, two per tenant. No customer records, external services or databases.
• Requests: 38 in total, 19 per mode, using GET and POST on /invoices, /reports/usage and /exports.

The vulnerable mode is intentionally insecure and the server cannot bind to any other address. For a staging engagement, agree hosts, tenants, accounts and stopping conditions with the system owner first. The Acceptable Use Policy describes authorized use, and this article gives no permission to test another system.

Run the downloadable fixture

Download fixture.py and verify.py into one directory. You need Python 3.9 or later and curl. The Python code uses only the standard library. Inspect the source, then run:

python3 verify.py --output rerun

The output directory must be new. The runner starts the vulnerable server, sends 19 real curl requests, saves each response, stops it, and repeats the sequence against a fixed server. Each request bypasses proxies, has a one-second connection timeout and a three-second total timeout, and follows no redirects.

Our recorded command was python3 verify.py --output recorded-run. It exited with {"checks": 38, "passed": true, "failures": []}, which means every request matched its expected outcome, including the intentionally vulnerable ones. The run on 10 November 2026 used fixture version 1.0.0, Python 3.15.2 and curl 8.7.1 on macOS ARM64, with the vulnerable server on port 53743 and the fixed server on port 53763. results.json holds the exact versions, UTC timestamps and source SHA-256 hashes.

Three isolation gaps, one at a time

List filter. GET /invoices?q=inv&tenant_id=globex is a search endpoint. In vulnerable mode the server uses the tenant_id parameter as the scope and falls back to the session's tenant only when the parameter is absent. A caller can therefore name any tenant.

Cache key. GET /reports/usage?period=2026-09 builds a report once and stores it under the path and query string. The key does not contain the tenant. Whichever tenant asks first fills the entry, and the other tenant receives it with an X-Cache: HIT header. Authentication still runs first, so an anonymous request gets a 401.

Export by job ID. POST /exports creates a job for the caller's tenant and returns an ID such as exp-0001. GET /exports/exp-0001 returns the job's rows. The vulnerable lookup checks only that the job exists. The fixture uses sequential IDs for clarity. The OWASP cheat sheet notes that opaque, random identifiers can reduce enumeration but are not authorization controls, so random IDs would not remove this gap.

Observed results: six cross-tenant requests and thirteen controls

For each gap the runner sent the request in both directions, once as an acme user and once as a globex user. The cache cases depend on order: one tenant fills the entry for a period, then the other tenant requests the same URL.

The 13 control requests per mode behave the same in both modes. Each user reads their own tenant's list. Alice repeating her own report request gets a HIT with her own tenant's data. Alice fetches her own export job, and amy, a second acme user, fetches it too with 200, so the fix checks the tenant and does not require the same user. An unknown job returns 404, and anonymous requests to all three endpoints return 401. A fix that denied everything would pass the six attack rows and fail these.

Verify a cross-tenant data leak before you report it

The recorded vulnerable request below is bob's read of the report that alice had already requested. This is the executed command, reformatted across lines:

curl --noproxy '*' --silent --show-error --include \
  --http1.1 --connect-timeout 1 --max-time 3 --request GET \
  --header 'Authorization: Bearer demo-globex-bob' \
  'http://127.0.0.1:53743/reports/usage?period=2026-09'

The recorded response has this status line, cache header and body; other headers are omitted:

HTTP/1.0 200 OK
X-Cache: HIT

{"generated_for": "alice", "period": "2026-09", "tenant": "acme", "total_cents": 20800}

The 200 status alone proves little. In fixed mode, bob's list request naming acme also returns 200, with globex rows, and a scanner that flags every 200 would report a false positive. Compare the data in the body with the session's tenant instead. Here the body says tenant: acme and generated_for: alice while bob, a globex user, sent the request. The HIT header explains the cause but is not the evidence, because a HIT on the caller's own data is normal. For exports, the same fields are tenant and created_by in the job.

A useful finding records the caller's identity and tenant, the request sequence (the cache leak appears only after the other tenant fills the entry), the other tenant's identifying fields in the response, and the intended policy. The impact in this fixture is disclosure of synthetic invoices and user names. A production severity rating needs the real data and the real tenant model.

Fix tenant isolation: session tenant, tenant in the key, check on lookup

The fixture holds both behaviours in one source file, so the fixed mode is a configuration switch that activates real server-side checks. It is not a generated patch or a production deployment. The list fix derives the scope from the session and ignores the parameter:

if self.server.mode == "vulnerable":
    scope = query.get("tenant_id", [tenant])[0]
else:
    scope = tenant  # tenant comes from the session only; the parameter is ignored

The cache fix puts the tenant in every key. The fixture checks the session before it reads the cache. The OWASP cheat sheet asks for authorization before a protected cache read, because key separation does not replace authorization:

key = url.path + "?" + url.query
if self.server.mode == "fixed":
    key = "tenant:" + tenant + ":" + key

The cheat sheet also says to put any other attribute that changes the result, such as the user, in the key. The cached report carries a generated_for user name, so a second acme user would see alice's name on a hit. We did not test that case.

The export fix checks the tenant on the lookup. A job that belongs to another tenant gets the same 404 body as an unknown job, so a caller cannot learn which IDs exist. A 403 is also reasonable; choose one by policy and test for it:

if job is None or (self.server.mode == "fixed" and job["tenant"] != tenant):
    return self.reply(404, {"error": "not_found"})

The retest repeated the identical 19-request sequence against a new server on port 53763. A tenant by endpoint matrix keeps the retest honest, because each gap can be fixed on one endpoint and stay open on the next. Each cell shows the fixed-mode result, with the vulnerable-mode result in brackets:

Because the fixed list ignores tenant_id, a request that names another tenant gets the caller's own rows. That fits this fixture, where each user belongs to one tenant. Users who belong to several tenants need an explicit selector that the server checks against current membership, as the cheat sheet describes. To inspect the servers by hand, run python3 fixture.py --mode vulnerable --port 8765 and python3 fixture.py --mode fixed --port 8766 in two terminals and stop each with Ctrl-C.

Inspect the evidence bundle

• Machine-readable results: all 38 cases, identities, expected and observed outcomes, the matrix, response filenames and hashes.
• Exact recorded curl commands: the ports refer to the finished run. Use the runner to create fresh servers.
• SHA-256 manifest: compare downloaded sources and evidence. This is an integrity check, not independent attestation.
• Fixture instructions and limitations, server source and verification source.

All bearer values and bodies are synthetic public demo data. The bundle holds the raw HTTP response text, including dates and server headers. No production credentials or customer data were involved.

What this experiment does not cover

One in-process cache, one list endpoint and one export endpoint demonstrate three missing checks. We did not test shared HTTP caches or CDNs, where Vary and Cache-Control matter, tenant claims in tokens or headers, users with several tenants, database-level isolation such as row-level security, queue workers, file storage, tenant offboarding or noisy-neighbor load. A clean result from the fixed fixture says nothing about another system. The wider class is in broken access control.

For a real engagement, build the matrix from the application's tenant model, keep the successful and denied controls, inspect returned data and repeat the matrix after each change. Excession Labs runs this kind of test in its security assessments. See how a cross-tenant finding looks in a sample report.