04-May-2026 — make Create Environment wizard work end-to-end closes #258

build 637 — commit 8fb1097

The Environment Setup wizard at /admin/environment-management/setup did not work on any deployed machine. Five failure modes stacked on top of each other depending on how far a run got. This change addresses all of them so that clicking Create Environment, on a deployed machine, ends up with a fully working Fly app, and clicking Resume on a half-completed environment finishes cleanly with no manual flyctl intervention.

Failure mode 1: persistent non-vcs/fly-io/configs.json read

The wizard ran inside the production Fly machine. non-vcs/ is gitignored and never makes it into the Docker image, so the deploy died on ENOENT and the residual code path then called process.exit(1), taking the whole production server with it. Both wizard entry points (handleEnvironmentCreate and handleEnvironmentSetup) hit the same read at deployToFlyio:141.

What changed

• Deleted server/lib/shared/configs-json.ts outright.
• Deleted server/deploy/manage-configs.ts (the only remaining writer of configs.json outside the wizard) and its npm script.
• server/lib/cli/commands/fly.ts now reads dockerImage from the ConfigKey.ENVIRONMENTS document via a new dockerImageFromDatabase() helper. After a successful deploy it persists the env entry through upsertEnvironmentInDatabase instead of writing a JSON file.
• Added optional dockerImage and region fields to EnvironmentsConfig plus a DEPLOYMENT_DEFAULTS constant (nbarrett36/ngx-ramblers:latest / lhr) matching what the published image actually is and the region the user has been deploying to. region is wired through buildDeploymentConfig so the legacy CLI deploy path also reads it from the DB.
• server/lib/cli/commands/destroy.ts lost its --skip-configs flag and now removes the env entry through removeEnvironmentFromDatabase.
• server/lib/environments/environments-config.ts rewritten: loadFromFiles() and the interactive "fall back to file-based configs.json?" prompt removed; configuredEnvironments() throws on a missing DB document instead of exiting; added upsertEnvironmentInDatabase and removeEnvironmentFromDatabase.
• server/lib/backup/config-initializer.ts seeds from the DB.
• server/deploy/config-loader.ts and server/deploy/mongo-cli.ts read configs through the DB-backed loadConfigs() helper.
• server/deploy/deploy-to-environments.ts and server/deploy/deploy-integration-worker.ts had duplicate hard-coded image strings. Both now read from DEPLOYMENT_DEFAULTS.DOCKER_IMAGE.

Failure mode 2: errors never reached the browser

readConfigFile and several other helpers called process.exit(1) on failure. From inside an HTTP/WebSocket request handler that killed the production process before the wizard's sendError(ws, ...) could fire. Fly then restarted the machine.

What changed

• readConfigFile removed entirely. runCommand lost its throwOnError boolean and now always throws. Fly volume helpers throw rather than exit.
• configuredEnvironments() and findEnvironmentFromDatabase() throw on missing DB config instead of exiting.
• Audited every process.exit(...) call in server/lib and server/deploy. The remaining exits are all in CLI-only entry points; none reachable from request handlers.

Failure mode 3: AUTH_SECRET (and other generated secrets) were not persisted

The wizard generated AUTH_SECRET and a full secrets bundle, wrote them to a local file under non-vcs/secrets/ (useless on a deployed machine), and passed them in-memory to deployToFlyio. The first deploy worked because flyctl secrets import ran with that in-memory bundle, but the resume flow loaded secrets from the DB - and the DB record never had AUTH_SECRET or the rest of the bundle written into it.

What changed

• updateEnvironmentsConfig in server/lib/environment-setup/environment-setup-service.ts now persists the full secrets bundle to envConfig.secrets. The local writeSecretsFile call is gone, and the writeSecretsFile and secretsPath imports with it.
• deployToFlyio performs a REQUIRED_SECRETS pre-flight check before running flyctl deploy. If any required secret is missing it throws with a clear list of missing keys, which the wizard relays through sendError(...).
• The Fly secrets-import file is now an os.tmpdir() temp file unlinked in a finally block; no per-app secrets file lingers on disk.
• The CLI ngx-cli fly deploy <name> action uses loadSecretsWithFallback so it works on a deployed machine where the local file no longer exists.

Failure mode 4: NODE_ENV-dependent path resolver caused silent SPA 404s

Discovered while testing on a freshly resumed environment. After the AUTH_SECRET pre-flight passed, the deployed Fly app booted but every request to / returned Cannot GET /. Root cause: server/lib/shared/path-utils.ts had a resolveClientPath whose output depended on a runtime check of NODE_ENV === "production". The wizard's buildSecretsConfig always sets NODE_ENV: "production" for new environments, but environments created during the original bug window had an incomplete persisted secrets bundle and the resume flow imported only the keys it could reconstruct, which did not include NODE_ENV. The Fly app booted with NODE_ENV empty, the resolver took the dev branch, the resolved distFolder did not exist, the static-asset middleware was skipped, and Express fell through to default 404.

What changed

• Rewrote server/lib/shared/path-utils.ts to anchor on the project root by walking up from __dirname looking for fly.toml - a stable marker present in dev (at the repo root) and in the deployed image (at /usr/src/app). The navigateUpFromCurrentExecutionDirectory helper and the isProduction() branch are gone. resolveClientPath returns paths relative to the project root, resolveServerPath returns paths relative to <projectRoot>/server. Same input produces the same output regardless of NODE_ENV.
• Added NODE_ENV to REQUIRED_SECRETS in server/lib/shared/secrets.ts. The path resolver no longer cares about NODE_ENV, but the surrounding app code does (env-config, isProduction guards elsewhere) and a missing NODE_ENV is a real misconfiguration worth catching at deploy time rather than as a silent runtime regression.

Failure mode 5: brevo template migration ENOENT shared the same root cause

The migration server/lib/mongo/migrations/database/20260120000000-update-brevo-transactional-template.ts reads brevo HTML templates via localTemplatePath in server/lib/brevo/templates/local-template-reader.ts, which calls resolveClientPath. On the deployed container during the wizard's initial bootstrap, NODE_ENV was unset, so resolveClientPath returned /usr/src/app/server/ts-gen/projects/ngx-ramblers/src/brevo/templates/fully-automated-text-body.html - a path that does not exist in the image. ENOENT, migration FAILED, wizard's "Migration Failed" page rendered. A retry succeeded only because by then the user was running locally, where the resolver landed somewhere that happened to have the files.

What changed

The failure mode 4 fix is the entire fix here - localTemplatePath now lands at <projectRoot>/projects/ngx-ramblers/src/brevo/templates/<name>.html consistently, regardless of NODE_ENV or which machine the migration runs on. No changes to the migration or the brevo resolver were needed.

Cleanup

• Removed the dead duplicate generateAuthSecret function in server/lib/cli/commands/fly.ts. The active one is in environment-setup-service.ts.

How they slipped through

All five failure modes share the same shape: deployment-environment-only bugs that dev laptops never exercise, because dev laptops happen to satisfy the fragile assumptions by accident - they have the local non-vcs/ files, they always run with NODE_ENV set in the shell, and they have the source tree sitting at the resolved paths regardless of how the resolver computes them. Wizard / migration changes need at least one CI run that exercises a clean container - no host filesystem, no inherited shell env - before being declared green.

Verification

• npm run lint clean
• npm run lint:server clean
• npx tsc --noEmit for server/tsconfig.json and the frontend tsconfig clean
• npm run test:server passes 619 mocha tests
• grep -rn "configs\.json" server/ (excluding tooling JSONs) and grep -rn "non-vcs/fly-io" server/ both empty
• grep -rn "writeSecretsFile" server/lib/environment-setup/ empty
• grep -rn "nbarrett.*/ngx-ramblers" server/ projects/ returns exactly one match: the single DEPLOYMENT_DEFAULTS.DOCKER_IMAGE declaration
• grep -rn "navigateUpFromCurrentExecutionDirectory\|isProduction" server/lib/shared/ empty - the path resolver no longer reads NODE_ENV

End-to-end behaviour

Clicking Create Environment in the wizard, on a deployed machine, ends up with a fully working Fly app: the env record and full secrets bundle (including NODE_ENV) are written to the DB, dockerImage is read from the DB, the deploy runs with secrets passed in-memory and imported to Fly, the env entry is upserted with the deploy token at the end, the SPA serves /, and migrations that read bundled assets resolve them from the project root regardless of NODE_ENV state. Clicking Resume on a half-completed env reads the full secrets bundle back from the DB and produces a working app without any manual flyctl secrets set. A simulated failure inside deployToFlyio (bad Fly API token, missing required secret) produces a red error in the wizard UI with the failure details, and the production server stays up.

Out of scope

• The vestigial EnvironmentSetupResult.configsJsonUpdated boolean was left as-is. Renaming it touches the wizard component and websocket payload shape, which would broaden the patch.