# 30-Apr-2026 — make Fly post-deploy scaling self-healing

## [build 632](https://github.com/nbarrett/ngx-ramblers/actions/runs/25156041311) — [commit acc3049](https://github.com/nbarrett/ngx-ramblers/commit/acc3049898948d7727a497b009b3c8f11fc18094)

_____

Two of sixteen environments (medway, sevenoaks) failed in the
all-environments deploy of 4cae672e with:

Error: could not update machine ...: failed to update VM ...:
aborted: machine is replacing: concurrent update in progress

After flyctl deploy --strategy rolling returned success, the script
fired flyctl scale memory immediately. Fly's machine-replacement state
machine was still finalising the rolling swap behind the scenes - the
scale call collided with that, and the existing 3-attempt 2s/4s retry
backoff was too short to outlast the replacement window. By the second
retry the old machine was gone and the new one had not yet appeared in
the 'app' process group, producing the secondary "No active machines"
error.

Two changes in server/lib/fly/fly-commands.ts that the deploy loop now
relies on instead of issuing flyctl scale unconditionally:

- listAppMachines reads flyctl machines list -j and surfaces id,
state, and current memory_mb. ensureScale compares this to the
target scaleCount and memory from the environment config. When
both already match (the common case - memory rarely changes
between deploys) the scale commands are skipped entirely, which
on its own would have rescued today's two failures.

- When scaling IS needed, waitForStableMachines polls flyctl
machines list -j every 3s (90s timeout) until the expected count
of machines are all in 'started' state with no transition states
visible, then issues the scale command. Removes the race instead
of retrying through it. After a count change the wait is repeated
before the memory call so the two scale operations cannot collide
with each other either.

deploy-to-environments.ts now calls ensureScale once per environment
in place of the back-to-back flyctl scale count and flyctl scale memory
calls.