Proposed Single-Instance Multi-Tenant Architecture

This article describes the proposed architecture where a single Fly.io application serves all groups, with tenant resolution middleware determining which group's database and storage to use per request.

Architecture Diagram

flowchart TD
    D1@{ icon: "ngx:user", label: "group-a-ramblers.org.uk", pos: "b", h: 48 }
    D2@{ icon: "ngx:user", label: "group-b-ramblers.org.uk", pos: "b", h: 48 }
    DN@{ icon: "ngx:user", label: "...up to 500 domains", pos: "b", h: 48 }

    D1 --> CF
    D2 --> CF
    DN --> CF

    CF@{ icon: "ngx:cloudflare", label: "Cloudflare DNS", pos: "b", h: 48 }
    CF --> FLY

    subgraph FLY["Fly.io Application - 2 to 4 instances"]
        TM["Tenant Resolution Middleware<br/>Host header to groupCode"]
        CACHE["Domain-to-group cache<br/>refreshed periodically"]
        DBR["Database Router<br/>groupCode to mongoose.useDb"]
        TM --> CACHE
        TM --> DBR
    end

    DBR --> MDB1@{ icon: "ngx:mongodb", label: "Group A DB", pos: "b", h: 48 }
    DBR --> MDB2@{ icon: "ngx:mongodb", label: "Group B DB", pos: "b", h: 48 }
    DBR --> MDBN@{ icon: "ngx:mongodb", label: "Group N DB", pos: "b", h: 48 }

    MDB1 --> S3
    MDB2 --> S3
    MDBN --> S3

    S3@{ icon: "ngx:aws", label: "Single S3 Bucket<br/>/{groupCode}/images/", pos: "b", h: 48 }

    style FLY fill:#E8F5EE,stroke:#9BC8AB,stroke-width:2px,rx:12,ry:12,color:#404143

Can a Single App Serve Multiple Domains?

Yes, this is technically feasible. Fly.io (and most reverse proxies) support binding multiple custom domains and TLS certificates to a single application. The server would inspect the Host header on each request to determine which group context to use. This is a well-established pattern (multi-tenant SaaS).

A user navigating to group-a-ramblers.org.uk vs group-b-ramblers.org.uk would see completely different branding, events, members, and content — unaware they hit the same server.

At 500 groups: Fly.io supports hundreds of custom certificates per app. TLS termination is handled at their edge, so this scales.

Request Lifecycle

sequenceDiagram
    participant Browser
    participant Fly as Fly.io Edge
    participant MW as Tenant Middleware
    participant Route as Route Handler
    participant DB as MongoDB (Group A)
    participant SPA as Angular SPA

    Browser->>Fly: GET https://group-a-ramblers.org.uk/events
    Fly->>MW: TLS terminated, Host: group-a-ramblers.org.uk
    MW->>MW: lookup("group-a-ramblers.org.uk") to "group-a"
    MW->>MW: req.groupCode = "group-a"
    MW->>MW: req.db = mongoose.useDb("group-a")
    MW->>MW: req.s3Prefix = "group-a"
    MW->>Route: GET /api/database/events
    Route->>DB: Query events (Group A DB only)
    DB-->>Route: Group A events
    Route-->>Browser: JSON response
    Browser->>SPA: Render with Group A config
    SPA->>Route: GET /api/database/config
    Route->>DB: Query config (Group A DB)
    DB-->>Route: Group A branding, settings
    Route-->>SPA: Group A config
    SPA-->>Browser: Group A branded UI

Coexistence: Running Both Models on the Same Codebase

A critical requirement is that the multi-tenant architecture must coexist with the current per-group deployment model — both running from the same codebase simultaneously. This is achievable using a feature-flagged middleware approach.

flowchart TD
    subgraph existing["Existing Groups - per-app deployment"]
        E1@{ icon: "logos:fly-icon", label: "Group A app", pos: "b", h: 48 }
        E2@{ icon: "logos:fly-icon", label: "Group B app", pos: "b", h: 48 }
        E3@{ icon: "logos:fly-icon", label: "Group C app", pos: "b", h: 48 }
        E1 --> ED1@{ icon: "ngx:mongodb", label: "Group A DB", pos: "b", h: 48 }
        E2 --> ED2@{ icon: "ngx:mongodb", label: "Group B DB", pos: "b", h: 48 }
        E3 --> ED3@{ icon: "ngx:mongodb", label: "Group C DB", pos: "b", h: 48 }
    end

    subgraph multitenant["New Groups - multi-tenant instance"]
        MT@{ icon: "logos:fly-icon", label: "Multi-tenant app", pos: "b", h: 48 }
        MT --> MD1@{ icon: "ngx:mongodb", label: "Group D DB", pos: "b", h: 48 }
        MT --> MD2@{ icon: "ngx:mongodb", label: "Group E DB", pos: "b", h: 48 }
        MT --> MDN@{ icon: "ngx:mongodb", label: "Group N DB", pos: "b", h: 48 }
    end

    IMG@{ icon: "logos:docker-icon", label: "Same Docker Image", pos: "b", h: 48 }
    IMG --> existing
    IMG --> multitenant

    style existing fill:#E8F5EE,stroke:#9BC8AB,stroke-width:2px,rx:12,ry:12,color:#404143
    style multitenant fill:#E8F5EE,stroke:#9BC8AB,stroke-width:2px,rx:12,ry:12,color:#404143

How it works:

• An environment variable (MULTI_TENANT_ENABLED) controls whether the tenant resolution middleware is active
• When off (existing groups): the app behaves exactly as today — connects to a single database via MONGODB_URI, serves one group
• When on (multi-tenant instance): the middleware inspects the Host header, resolves the group, and switches database context per request
• Both modes run the same codebase and the same Docker image — the only difference is the environment variable and which secrets are configured

Migration path:

1. Add tenant middleware behind the feature flag — off by default, zero impact on existing groups
2. Deploy a new Fly.io app with MULTI_TENANT_ENABLED=true and a shared admin database containing the domain-to-group lookup table
3. Onboard new groups onto the multi-tenant instance
4. Optionally migrate existing groups one at a time by pointing their DNS to the multi-tenant instance and adding their domain to the lookup table
5. Decommission the old per-group app once migrated

This approach means there is no big-bang migration — existing groups continue running undisturbed, and the multi-tenant capability is introduced incrementally alongside them.

What Would Be Required

1. Request-Scoped Tenant Resolution (HIGH)

Every incoming HTTP request must be mapped to a group context based on the Host header:

• A domain-to-group lookup table (stored in a shared admin database or in-memory cache, refreshed periodically)
• Express middleware that resolves the group before any route handler runs
• The resolved group context must flow through the entire request lifecycle

At 500 groups: The lookup table is small (500 domain to groupCode pairs). An in-memory Map refreshed every 60 seconds from a shared admin database is sufficient.

2. Database Architecture (HIGH)

Option A: Single database, collection-level isolation

• All groups share one MongoDB database
• Every document gains a groupCode field
• Every query gains a groupCode filter
• Risk: a missing filter leaks data across groups — this risk multiplies with 500 groups

Option B: Shared connection pool, database-per-group (recommended)

• Keep separate databases per group (as today)
• Use mongoose.connection.useDb(groupDbName) to route queries per request
• Models are compiled once and reused across databases
• Benefit: maintains current data isolation model, zero risk of cross-group data leakage

At 500 groups (Option B):

• MongoDB Atlas supports thousands of databases per cluster
• useDb() reuses the existing connection pool — it does not create new connections
• Connection pool size: ~50–100 connections shared across all groups

3. S3 Storage Consolidation (MEDIUM)

• Keep separate buckets: proven model, but managing 500 IAM users/policies is operationally burdensome
• Single bucket with path prefixes: {groupCode}/images/... — simpler at scale, recommended for 500 groups

4. Authentication Changes (MEDIUM)

• Recommended: Single shared AUTH_SECRET but include groupCode in the JWT payload
• Token verification middleware checks that the token's groupCode matches the request's resolved groupCode
• A member logged into one group's domain must NOT be authenticated on another group's domain

5. Background Jobs (MEDIUM)

Current: Each of the 13 Fly.io apps runs its own independent cron jobs.

Proposed: A single scheduler manages jobs for all 500 groups with controlled concurrency.

flowchart TD
    SCHED["Job Scheduler"]
    Q["Job Queue<br/>event-sync, email-send"]
    SCHED --> Q
    Q --> W1@{ icon: "logos:fly-icon", label: "Group A sync", pos: "b", h: 48 }
    Q --> W2@{ icon: "logos:fly-icon", label: "Group B sync", pos: "b", h: 48 }
    Q --> W3@{ icon: "logos:fly-icon", label: "Group C sync", pos: "b", h: 48 }
    Q --> WN["... x 500 groups<br/>concurrency: 3-5<br/>staggered schedule"]

    style SCHED fill:#E8F5EE,stroke:#9BC8AB,stroke-width:2px,rx:12,ry:12,color:#404143

6. Frontend Changes (LOW)

• No frontend code changes needed — the Angular app already loads all config at runtime from the API
• The initial /api/database/config call would return different data based on which domain was accessed

7. Admin and Onboarding (EXISTING — extend for multi-tenant)

An environment management admin UI already exists at /admin/environment-management/setup. It provides a multi-step wizard for provisioning new groups:

1. Group Selection — create new, clone from existing, or modify an existing environment
2. Environment and Services — configure app name, database, and service integrations
3. Admin and Options — create the initial admin user
4. Review and Validate — confirm all settings before deployment
5. Deploy — provisions the Fly.io app, database, S3 bucket, DNS, and seeds initial data

For the multi-tenant model, this wizard would need to be extended to support provisioning groups within the shared instance (database creation + domain-to-group mapping) rather than creating a new Fly.io app. The core onboarding workflow already exists — it just needs a multi-tenant deployment target alongside the existing per-app target.