{"id":"6aa13eaba7b06e82ff1a557f","title":"2026 09 09 Issue 300","path":"how-to/committee/release-notes/2026-09-09-issue-300","contentMarkdown":"# 09-Sep-2026 — hold the worker awake for the whole job so the overrun stop can fire, and add admin [#300](https://github.com/nbarrett/ngx-ramblers/issues/300)\n\n## [build 885](https://github.com/nbarrett/ngx-ramblers/actions/runs/34342625474) — [commit 570e098](https://github.com/nbarrett/ngx-ramblers/commit/570e098ca87114b3dcdcf7df8eee2afcc36bf3ab)\n\n_____\n\n## The problem\n\nThe 45-minute overrun stop is a timer inside the worker's Node process, but\nFly autosuspends the machine on excess capacity (zero concurrent requests),\nwhich freezes the VM and stops that timer advancing. The previous keep-alive\npinged the health address every 60s, which left the machine idle almost the\nwhole minute, so Fly suspended it anyway. A running OS Maps conversion then\nfroze for hours, the queue jammed behind it, and the overrun stop never fired.\n\n## What is new\n\n- The keep-alive now holds a single streaming connection open to the worker\nfor as long as a job runs (reopening if it drops), so Fly always sees\nconcurrency and does not suspend mid-job. Idle suspend is unchanged, so the\ncost saving stays. The overrun watchdog remains the backstop and can now\nactually run because the VM is never frozen while working.\n- A GET /api/integration-worker/hold endpoint the keep-alive holds open.\n- Admin controls for the worker queue, on the System Settings Diagnostics tab:\nsee the active job and anything queued, stop the active job, or clear the\nqueue, so a stuck job can be ended and someone else can run something without\na machine restart.\n\n## How it works\n\n- Worker: the queue gains clearQueued(); the runner registers a cancel for the\nactive job (the same SIGKILL path as the watchdog) and exposes\ncancelActiveWorkerJob(); signed /queue/status, /queue/cancel-active and\n/queue/clear endpoints on the worker.\n- Website: admin-only /api/health/worker-queue/* routes sign and proxy to the\nworker, and an app-integration-worker-queue panel drives them.","contentHtml":"<h1>09-Sep-2026 — hold the worker awake for the whole job so the overrun stop can fire, and add admin <a href=\"https://github.com/nbarrett/ngx-ramblers/issues/300\">#300</a></h1>\n<h2><a href=\"https://github.com/nbarrett/ngx-ramblers/actions/runs/34342625474\">build 885</a> — <a href=\"https://github.com/nbarrett/ngx-ramblers/commit/570e098ca87114b3dcdcf7df8eee2afcc36bf3ab\">commit 570e098</a></h2>\n<hr>\n<h2>The problem</h2>\n<p>The 45-minute overrun stop is a timer inside the worker&#39;s Node process, but\nFly autosuspends the machine on excess capacity (zero concurrent requests),\nwhich freezes the VM and stops that timer advancing. The previous keep-alive\npinged the health address every 60s, which left the machine idle almost the\nwhole minute, so Fly suspended it anyway. A running OS Maps conversion then\nfroze for hours, the queue jammed behind it, and the overrun stop never fired.</p>\n<h2>What is new</h2>\n<ul>\n<li>The keep-alive now holds a single streaming connection open to the worker\nfor as long as a job runs (reopening if it drops), so Fly always sees\nconcurrency and does not suspend mid-job. Idle suspend is unchanged, so the\ncost saving stays. The overrun watchdog remains the backstop and can now\nactually run because the VM is never frozen while working.</li>\n<li>A GET /api/integration-worker/hold endpoint the keep-alive holds open.</li>\n<li>Admin controls for the worker queue, on the System Settings Diagnostics tab:\nsee the active job and anything queued, stop the active job, or clear the\nqueue, so a stuck job can be ended and someone else can run something without\na machine restart.</li>\n</ul>\n<h2>How it works</h2>\n<ul>\n<li>Worker: the queue gains clearQueued(); the runner registers a cancel for the\nactive job (the same SIGKILL path as the watchdog) and exposes\ncancelActiveWorkerJob(); signed /queue/status, /queue/cancel-active and\n/queue/clear endpoints on the worker.</li>\n<li>Website: admin-only /api/health/worker-queue/* routes sign and proxy to the\nworker, and an app-integration-worker-queue panel drives them.</li>\n</ul>\n"}