14-Apr-2026 — distributed Serenity upload worker with S3 report storage #114

build 602 — commit a741945

Offloads the browser-driven Ramblers walks upload from the main website app to a dedicated Fly machine so the website no longer needs Serenity, Chrome or a 460MB Node heap reserved for it. The website enqueues jobs over an HTTP/HMAC-signed contract; the worker runs the playwright scenario, streams progress back, uploads the generated Serenity HTML report to the website's own S3 bucket, and posts a final result callback with the report key.

Worker infrastructure

• New Fly app ngx-ramblers-upload-worker sharing the same Docker image as the website but running npm run worker-server.
• Worker build + deploy gated on dorny/paths-filter so only worker-relevant file changes trigger a rebuild; standalone workflow_dispatch workflow kept as a manual escape hatch.
• Single GitHub secret RAMBLERS_UPLOAD_WORKER_SECRETS_JSON carries the HMAC shared secret and AES encryption key; surfaced in admin UI under Global Settings and atomically synced alongside CONFIGS_JSON.
• Per-environment routing done via RAMBLERS_UPLOAD_WORKER_URL, RAMBLERS_UPLOAD_WORKER_SHARED_SECRET and RAMBLERS_UPLOAD_WORKER_ENCRYPTION_KEY on each env's secrets.

Upload lifecycle

• website dispatches to /api/ramblers-upload-worker/jobs with encrypted walks-manager credentials and encrypted AWS credentials for the report bucket, plus the target bucket/region/keyPrefix.
• Worker decrypts, runs npm run serenity, streams stdout as STANDARD_OUT progress, test-step events as TEST_STEP progress, and new LIFECYCLE events ("Scenario execution complete", "Serenity report generation starting", "Serenity report generation complete, uploading to S3").
• On subprocess exit, worker uploads the full target/site/serenity directory to S3 under ramblers-upload-reports/<fileName> and posts /result with reportKeyPrefix + reportBucket.
• website records a final SUCCESS audit "Serenity report stored at s3://..." carrying the reportKeyPrefix, broadcast over the existing upload websocket so the audit appears live without a page reload.

Login path

• Login screenplay task no longer wraps its full body in mongooseClient.execute; when RAMBLERS_USERNAME / RAMBLERS_PASSWORD are present in the env (as on the worker) it skips the Mongo lookup entirely. website in-process runs still fall back to systemConfig via Mongo as before.

Report path resolution

• serenityReportLocalPath() now resolves from process.cwd() rather than __dirname + folderNavigationsUp, so the Serenity report directory is found on the worker regardless of NODE_ENV or compiled output layout.

Walk export UI

• Per-row "View Report" buttons removed; single "View Report" button now sits next to "Back To Walks Admin", driven by the currently selected upload session's report audit. The getter scans the full unfiltered audits list so report rows beyond the 500-row cap are still found.
• Button restyled to btn-primary (NGX yellow) to match the page.

Local developer ergonomics

• ngx-cli local dev now re-applies server/.env overrides for RAMBLERS_UPLOAD_WORKER_URL / _CALLBACK_BASE_URL / _SHARED_SECRET / _ENCRYPTION_KEY after building the env-secret-derived env, so developers can toggle between local worker (http://localhost:5002) and the Fly worker purely via .env edits without touching code.
• Worker websocket-server log decodes test-step-reporter payloads and prints Serenity step: "<name>" → SUCCESS / outcome code, replacing the old Message received of size: N b for type: test-step-reporter noise.

Shared content-type mapping

• contentTypeForExtension + fileContentTypeMappings consolidated in content-metadata.model.ts as a single data-structure-driven lookup used by both the server's aws-utils.contentTypeFrom and the S3 directory uploader. Covers images, web assets, fonts, archives and documents; falls back to application/octet-stream. Backed by aws-utils.spec.ts unit tests.