{"id":"69fb87632d3be50e28f7c738","title":"2026 05 04","path":"how-to/committee/release-notes/2026-05-04","contentMarkdown":"# 04-May-2026 — make Create Environment wizard work end-to-end [closes #258](https://github.com/nbarrett/ngx-ramblers/issues/258)\n\n## [build 637](https://github.com/nbarrett/ngx-ramblers/actions/runs/25450246053) — [commit 8fb1097](https://github.com/nbarrett/ngx-ramblers/commit/8fb1097b0777db28a6903d0592796a9f70bd357b)\n\n_____\n\nThe Environment Setup wizard at `/admin/environment-management/setup`\ndid not work on any deployed machine. Five failure modes stacked on\ntop of each other depending on how far a run got. This change addresses\nall of them so that clicking Create Environment, on a deployed machine,\nends up with a fully working Fly app, and clicking Resume on a\nhalf-completed environment finishes cleanly with no manual `flyctl`\nintervention.\n\n## Failure mode 1: persistent `non-vcs/fly-io/configs.json` read\n\nThe wizard ran inside the production Fly machine. `non-vcs/` is\ngitignored and never makes it into the Docker image, so the deploy\ndied on `ENOENT` and the residual code path then called\n`process.exit(1)`, taking the whole production server with it. Both\nwizard entry points (`handleEnvironmentCreate` and\n`handleEnvironmentSetup`) hit the same read at `deployToFlyio:141`.\n\n### What changed\n\n- Deleted `server/lib/shared/configs-json.ts` outright.\n- Deleted `server/deploy/manage-configs.ts` (the only remaining writer\nof `configs.json` outside the wizard) and its npm script.\n- `server/lib/cli/commands/fly.ts` now reads `dockerImage` from the\n`ConfigKey.ENVIRONMENTS` document via a new\n`dockerImageFromDatabase()` helper. After a successful deploy it\npersists the env entry through `upsertEnvironmentInDatabase` instead\nof writing a JSON file.\n- Added optional `dockerImage` and `region` fields to\n`EnvironmentsConfig` plus a `DEPLOYMENT_DEFAULTS` constant\n(`nbarrett36/ngx-ramblers:latest` / `lhr`) matching what the\npublished image actually is and the region the user has been\ndeploying to. `region` is wired through `buildDeploymentConfig` so\nthe legacy CLI deploy path also reads it from the DB.\n- `server/lib/cli/commands/destroy.ts` lost its `--skip-configs` flag\nand now removes the env entry through\n`removeEnvironmentFromDatabase`.\n- `server/lib/environments/environments-config.ts` rewritten:\n`loadFromFiles()` and the interactive \"fall back to file-based\nconfigs.json?\" prompt removed; `configuredEnvironments()` throws on\na missing DB document instead of exiting; added\n`upsertEnvironmentInDatabase` and `removeEnvironmentFromDatabase`.\n- `server/lib/backup/config-initializer.ts` seeds from the DB.\n- `server/deploy/config-loader.ts` and `server/deploy/mongo-cli.ts`\nread configs through the DB-backed `loadConfigs()` helper.\n- `server/deploy/deploy-to-environments.ts` and\n`server/deploy/deploy-integration-worker.ts` had duplicate\nhard-coded image strings. Both now read from\n`DEPLOYMENT_DEFAULTS.DOCKER_IMAGE`.\n\n## Failure mode 2: errors never reached the browser\n\n`readConfigFile` and several other helpers called `process.exit(1)`\non failure. From inside an HTTP/WebSocket request handler that killed\nthe production process before the wizard's `sendError(ws, ...)` could\nfire. Fly then restarted the machine.\n\n### What changed\n\n- `readConfigFile` removed entirely. `runCommand` lost its\n`throwOnError` boolean and now always throws. Fly volume helpers\nthrow rather than exit.\n- `configuredEnvironments()` and `findEnvironmentFromDatabase()`\nthrow on missing DB config instead of exiting.\n- Audited every `process.exit(...)` call in `server/lib` and\n`server/deploy`. The remaining exits are all in CLI-only entry\npoints; none reachable from request handlers.\n\n## Failure mode 3: AUTH_SECRET (and other generated secrets) were not persisted\n\nThe wizard generated `AUTH_SECRET` and a full secrets bundle, wrote\nthem to a local file under `non-vcs/secrets/` (useless on a deployed\nmachine), and passed them in-memory to `deployToFlyio`. The first\ndeploy worked because `flyctl secrets import` ran with that in-memory\nbundle, but the resume flow loaded secrets from the DB - and the DB\nrecord never had `AUTH_SECRET` or the rest of the bundle written\ninto it.\n\n### What changed\n\n- `updateEnvironmentsConfig` in\n`server/lib/environment-setup/environment-setup-service.ts` now\npersists the full `secrets` bundle to `envConfig.secrets`. The\nlocal `writeSecretsFile` call is gone, and the `writeSecretsFile`\nand `secretsPath` imports with it.\n- `deployToFlyio` performs a `REQUIRED_SECRETS` pre-flight check\nbefore running `flyctl deploy`. If any required secret is missing\nit throws with a clear list of missing keys, which the wizard\nrelays through `sendError(...)`.\n- The Fly secrets-import file is now an `os.tmpdir()` temp file\nunlinked in a `finally` block; no per-app secrets file lingers on\ndisk.\n- The CLI `ngx-cli fly deploy ` action uses\n`loadSecretsWithFallback` so it works on a deployed machine where\nthe local file no longer exists.\n\n## Failure mode 4: NODE_ENV-dependent path resolver caused silent SPA 404s\n\nDiscovered while testing on a freshly resumed environment. After the\nAUTH_SECRET pre-flight passed, the deployed Fly app booted but every\nrequest to `/` returned `Cannot GET /`. Root cause:\n`server/lib/shared/path-utils.ts` had a `resolveClientPath` whose\noutput depended on a runtime check of `NODE_ENV === \"production\"`.\nThe wizard's `buildSecretsConfig` always sets `NODE_ENV: \"production\"`\nfor new environments, but environments created during the original bug\nwindow had an incomplete persisted secrets bundle and the resume flow\nimported only the keys it could reconstruct, which did not include\n`NODE_ENV`. The Fly app booted with `NODE_ENV` empty, the resolver took\nthe dev branch, the resolved `distFolder` did not exist, the\nstatic-asset middleware was skipped, and Express fell through to\ndefault 404.\n\n### What changed\n\n- Rewrote `server/lib/shared/path-utils.ts` to anchor on the project\nroot by walking up from `__dirname` looking for `fly.toml` - a\nstable marker present in dev (at the repo root) and in the deployed\nimage (at `/usr/src/app`). The `navigateUpFromCurrentExecutionDirectory`\nhelper and the `isProduction()` branch are gone. `resolveClientPath`\nreturns paths relative to the project root, `resolveServerPath`\nreturns paths relative to `/server`. Same input\nproduces the same output regardless of `NODE_ENV`.\n- Added `NODE_ENV` to `REQUIRED_SECRETS` in\n`server/lib/shared/secrets.ts`. The path resolver no longer cares\nabout `NODE_ENV`, but the surrounding app code does (env-config,\nisProduction guards elsewhere) and a missing `NODE_ENV` is a real\nmisconfiguration worth catching at deploy time rather than as a\nsilent runtime regression.\n\n## Failure mode 5: brevo template migration ENOENT shared the same root cause\n\nThe migration\n`server/lib/mongo/migrations/database/20260120000000-update-brevo-transactional-template.ts`\nreads brevo HTML templates via `localTemplatePath` in\n`server/lib/brevo/templates/local-template-reader.ts`, which calls\n`resolveClientPath`. On the deployed container during the wizard's\ninitial bootstrap, `NODE_ENV` was unset, so `resolveClientPath`\nreturned `/usr/src/app/server/ts-gen/projects/ngx-ramblers/src/brevo/templates/fully-automated-text-body.html` -\na path that does not exist in the image. ENOENT, migration FAILED,\nwizard's \"Migration Failed\" page rendered. A retry succeeded only\nbecause by then the user was running locally, where the resolver\nlanded somewhere that happened to have the files.\n\n### What changed\n\nThe failure mode 4 fix is the entire fix here - `localTemplatePath`\nnow lands at `/projects/ngx-ramblers/src/brevo/templates/.html`\nconsistently, regardless of `NODE_ENV` or which machine the migration\nruns on. No changes to the migration or the brevo resolver were\nneeded.\n\n## Cleanup\n\n- Removed the dead duplicate `generateAuthSecret` function in\n`server/lib/cli/commands/fly.ts`. The active one is in\n`environment-setup-service.ts`.\n\n## How they slipped through\n\nAll five failure modes share the same shape: deployment-environment-only\nbugs that dev laptops never exercise, because dev laptops happen to\nsatisfy the fragile assumptions by accident - they have the local\n`non-vcs/` files, they always run with `NODE_ENV` set in the shell,\nand they have the source tree sitting at the resolved paths regardless\nof how the resolver computes them. Wizard / migration changes need at\nleast one CI run that exercises a clean container - no host\nfilesystem, no inherited shell env - before being declared green.\n\n## Verification\n\n- `npm run lint` clean\n- `npm run lint:server` clean\n- `npx tsc --noEmit` for `server/tsconfig.json` and the frontend\ntsconfig clean\n- `npm run test:server` passes 619 mocha tests\n- `grep -rn \"configs\\.json\" server/` (excluding tooling JSONs) and\n`grep -rn \"non-vcs/fly-io\" server/` both empty\n- `grep -rn \"writeSecretsFile\" server/lib/environment-setup/` empty\n- `grep -rn \"nbarrett.*/ngx-ramblers\" server/ projects/` returns\nexactly one match: the single `DEPLOYMENT_DEFAULTS.DOCKER_IMAGE`\ndeclaration\n- `grep -rn \"navigateUpFromCurrentExecutionDirectory\\|isProduction\" server/lib/shared/`\nempty - the path resolver no longer reads `NODE_ENV`\n\n## End-to-end behaviour\n\nClicking Create Environment in the wizard, on a deployed machine,\nends up with a fully working Fly app: the env record and full secrets\nbundle (including `NODE_ENV`) are written to the DB, `dockerImage` is\nread from the DB, the deploy runs with secrets passed in-memory and\nimported to Fly, the env entry is upserted with the deploy token at\nthe end, the SPA serves `/`, and migrations that read bundled assets\nresolve them from the project root regardless of `NODE_ENV` state.\nClicking Resume on a half-completed env reads the full secrets bundle\nback from the DB and produces a working app without any manual\n`flyctl secrets set`. A simulated failure inside `deployToFlyio` (bad\nFly API token, missing required secret) produces a red error in the\nwizard UI with the failure details, and the production server stays\nup.\n\n## Out of scope\n\n- The vestigial `EnvironmentSetupResult.configsJsonUpdated` boolean\nwas left as-is. Renaming it touches the wizard component and\nwebsocket payload shape, which would broaden the patch.","contentHtml":"

04-May-2026 — make Create Environment wizard work end-to-end closes #258

\n

build 637 — commit 8fb1097

\n
\n

The Environment Setup wizard at /admin/environment-management/setup\ndid not work on any deployed machine. Five failure modes stacked on\ntop of each other depending on how far a run got. This change addresses\nall of them so that clicking Create Environment, on a deployed machine,\nends up with a fully working Fly app, and clicking Resume on a\nhalf-completed environment finishes cleanly with no manual flyctl\nintervention.

\n

Failure mode 1: persistent non-vcs/fly-io/configs.json read

\n

The wizard ran inside the production Fly machine. non-vcs/ is\ngitignored and never makes it into the Docker image, so the deploy\ndied on ENOENT and the residual code path then called\nprocess.exit(1), taking the whole production server with it. Both\nwizard entry points (handleEnvironmentCreate and\nhandleEnvironmentSetup) hit the same read at deployToFlyio:141.

\n

What changed

\n
    \n
  • Deleted server/lib/shared/configs-json.ts outright.
    • \n
  • Deleted server/deploy/manage-configs.ts (the only remaining writer\nof configs.json outside the wizard) and its npm script.
    • \n
  • server/lib/cli/commands/fly.ts now reads dockerImage from the\nConfigKey.ENVIRONMENTS document via a new\ndockerImageFromDatabase() helper. After a successful deploy it\npersists the env entry through upsertEnvironmentInDatabase instead\nof writing a JSON file.
    • \n
  • Added optional dockerImage and region fields to\nEnvironmentsConfig plus a DEPLOYMENT_DEFAULTS constant\n(nbarrett36/ngx-ramblers:latest / lhr) matching what the\npublished image actually is and the region the user has been\ndeploying to. region is wired through buildDeploymentConfig so\nthe legacy CLI deploy path also reads it from the DB.
    • \n
  • server/lib/cli/commands/destroy.ts lost its --skip-configs flag\nand now removes the env entry through\nremoveEnvironmentFromDatabase.
    • \n
  • server/lib/environments/environments-config.ts rewritten:\nloadFromFiles() and the interactive "fall back to file-based\nconfigs.json?" prompt removed; configuredEnvironments() throws on\na missing DB document instead of exiting; added\nupsertEnvironmentInDatabase and removeEnvironmentFromDatabase.
    • \n
  • server/lib/backup/config-initializer.ts seeds from the DB.
    • \n
  • server/deploy/config-loader.ts and server/deploy/mongo-cli.ts\nread configs through the DB-backed loadConfigs() helper.
    • \n
  • server/deploy/deploy-to-environments.ts and\nserver/deploy/deploy-integration-worker.ts had duplicate\nhard-coded image strings. Both now read from\nDEPLOYMENT_DEFAULTS.DOCKER_IMAGE.
    • \n
\n

Failure mode 2: errors never reached the browser

\n

readConfigFile and several other helpers called process.exit(1)\non failure. From inside an HTTP/WebSocket request handler that killed\nthe production process before the wizard's sendError(ws, ...) could\nfire. Fly then restarted the machine.

\n

What changed

\n
    \n
  • readConfigFile removed entirely. runCommand lost its\nthrowOnError boolean and now always throws. Fly volume helpers\nthrow rather than exit.
    • \n
  • configuredEnvironments() and findEnvironmentFromDatabase()\nthrow on missing DB config instead of exiting.
    • \n
  • Audited every process.exit(...) call in server/lib and\nserver/deploy. The remaining exits are all in CLI-only entry\npoints; none reachable from request handlers.
    • \n
\n

Failure mode 3: AUTH_SECRET (and other generated secrets) were not persisted

\n

The wizard generated AUTH_SECRET and a full secrets bundle, wrote\nthem to a local file under non-vcs/secrets/ (useless on a deployed\nmachine), and passed them in-memory to deployToFlyio. The first\ndeploy worked because flyctl secrets import ran with that in-memory\nbundle, but the resume flow loaded secrets from the DB - and the DB\nrecord never had AUTH_SECRET or the rest of the bundle written\ninto it.

\n

What changed

\n
    \n
  • updateEnvironmentsConfig in\nserver/lib/environment-setup/environment-setup-service.ts now\npersists the full secrets bundle to envConfig.secrets. The\nlocal writeSecretsFile call is gone, and the writeSecretsFile\nand secretsPath imports with it.
    • \n
  • deployToFlyio performs a REQUIRED_SECRETS pre-flight check\nbefore running flyctl deploy. If any required secret is missing\nit throws with a clear list of missing keys, which the wizard\nrelays through sendError(...).
    • \n
  • The Fly secrets-import file is now an os.tmpdir() temp file\nunlinked in a finally block; no per-app secrets file lingers on\ndisk.
    • \n
  • The CLI ngx-cli fly deploy <name> action uses\nloadSecretsWithFallback so it works on a deployed machine where\nthe local file no longer exists.
    • \n
\n

Failure mode 4: NODE_ENV-dependent path resolver caused silent SPA 404s

\n

Discovered while testing on a freshly resumed environment. After the\nAUTH_SECRET pre-flight passed, the deployed Fly app booted but every\nrequest to / returned Cannot GET /. Root cause:\nserver/lib/shared/path-utils.ts had a resolveClientPath whose\noutput depended on a runtime check of NODE_ENV === "production".\nThe wizard's buildSecretsConfig always sets NODE_ENV: "production"\nfor new environments, but environments created during the original bug\nwindow had an incomplete persisted secrets bundle and the resume flow\nimported only the keys it could reconstruct, which did not include\nNODE_ENV. The Fly app booted with NODE_ENV empty, the resolver took\nthe dev branch, the resolved distFolder did not exist, the\nstatic-asset middleware was skipped, and Express fell through to\ndefault 404.

\n

What changed

\n
    \n
  • Rewrote server/lib/shared/path-utils.ts to anchor on the project\nroot by walking up from __dirname looking for fly.toml - a\nstable marker present in dev (at the repo root) and in the deployed\nimage (at /usr/src/app). The navigateUpFromCurrentExecutionDirectory\nhelper and the isProduction() branch are gone. resolveClientPath\nreturns paths relative to the project root, resolveServerPath\nreturns paths relative to <projectRoot>/server. Same input\nproduces the same output regardless of NODE_ENV.
    • \n
  • Added NODE_ENV to REQUIRED_SECRETS in\nserver/lib/shared/secrets.ts. The path resolver no longer cares\nabout NODE_ENV, but the surrounding app code does (env-config,\nisProduction guards elsewhere) and a missing NODE_ENV is a real\nmisconfiguration worth catching at deploy time rather than as a\nsilent runtime regression.
    • \n
\n

Failure mode 5: brevo template migration ENOENT shared the same root cause

\n

The migration\nserver/lib/mongo/migrations/database/20260120000000-update-brevo-transactional-template.ts\nreads brevo HTML templates via localTemplatePath in\nserver/lib/brevo/templates/local-template-reader.ts, which calls\nresolveClientPath. On the deployed container during the wizard's\ninitial bootstrap, NODE_ENV was unset, so resolveClientPath\nreturned /usr/src/app/server/ts-gen/projects/ngx-ramblers/src/brevo/templates/fully-automated-text-body.html -\na path that does not exist in the image. ENOENT, migration FAILED,\nwizard's "Migration Failed" page rendered. A retry succeeded only\nbecause by then the user was running locally, where the resolver\nlanded somewhere that happened to have the files.

\n

What changed

\n

The failure mode 4 fix is the entire fix here - localTemplatePath\nnow lands at <projectRoot>/projects/ngx-ramblers/src/brevo/templates/<name>.html\nconsistently, regardless of NODE_ENV or which machine the migration\nruns on. No changes to the migration or the brevo resolver were\nneeded.

\n

Cleanup

\n
    \n
  • Removed the dead duplicate generateAuthSecret function in\nserver/lib/cli/commands/fly.ts. The active one is in\nenvironment-setup-service.ts.
    • \n
\n

How they slipped through

\n

All five failure modes share the same shape: deployment-environment-only\nbugs that dev laptops never exercise, because dev laptops happen to\nsatisfy the fragile assumptions by accident - they have the local\nnon-vcs/ files, they always run with NODE_ENV set in the shell,\nand they have the source tree sitting at the resolved paths regardless\nof how the resolver computes them. Wizard / migration changes need at\nleast one CI run that exercises a clean container - no host\nfilesystem, no inherited shell env - before being declared green.

\n

Verification

\n
    \n
  • npm run lint clean
    • \n
  • npm run lint:server clean
    • \n
  • npx tsc --noEmit for server/tsconfig.json and the frontend\ntsconfig clean
    • \n
  • npm run test:server passes 619 mocha tests
    • \n
  • grep -rn "configs\\.json" server/ (excluding tooling JSONs) and\ngrep -rn "non-vcs/fly-io" server/ both empty
    • \n
  • grep -rn "writeSecretsFile" server/lib/environment-setup/ empty
    • \n
  • grep -rn "nbarrett.*/ngx-ramblers" server/ projects/ returns\nexactly one match: the single DEPLOYMENT_DEFAULTS.DOCKER_IMAGE\ndeclaration
    • \n
  • grep -rn "navigateUpFromCurrentExecutionDirectory\\|isProduction" server/lib/shared/\nempty - the path resolver no longer reads NODE_ENV
    • \n
\n

End-to-end behaviour

\n

Clicking Create Environment in the wizard, on a deployed machine,\nends up with a fully working Fly app: the env record and full secrets\nbundle (including NODE_ENV) are written to the DB, dockerImage is\nread from the DB, the deploy runs with secrets passed in-memory and\nimported to Fly, the env entry is upserted with the deploy token at\nthe end, the SPA serves /, and migrations that read bundled assets\nresolve them from the project root regardless of NODE_ENV state.\nClicking Resume on a half-completed env reads the full secrets bundle\nback from the DB and produces a working app without any manual\nflyctl secrets set. A simulated failure inside deployToFlyio (bad\nFly API token, missing required secret) produces a red error in the\nwizard UI with the failure details, and the production server stays\nup.

\n

Out of scope

\n
    \n
  • The vestigial EnvironmentSetupResult.configsJsonUpdated boolean\nwas left as-is. Renaming it touches the wizard component and\nwebsocket payload shape, which would broaden the patch.
    • \n
\n"}