02-Jul-2026 — perf+fix(server+admin): memory economies, restart reporting when session is lost, CSV d... #304
What's new
Forensic memory work driven by a production site hovering at 89% of its 259MB heap limit, plus two small fixes from production testing. The server was paying for admin-only features on every boot: playwright (28MB), jsdom (29MB) and xlsx (13MB) all loaded eagerly even though they are only used for static-site migration, document conversion and member bulk load. They now load on first use, cutting roughly 70MB from the boot baseline of every machine. The area-map GeoJSON caches, which could hold up to three 50MB copies of the full-UK district file forever after a single admin visit to area configuration, now evict after 10 minutes idle and share one parse instead of two. The machine restart button no longer misreports a successful restart as a failure when the login session does not survive the restart, and a CSV exported from the walk export page can now be re-imported through the walk import page without date errors.
At a glance
- Server boot memory drops by roughly 70MB on every environment; the biggest saving on 512MB machines.
- Area-map admin pages no longer pin ~100MB of UK district geometry in memory for the life of the process
- the data loads on demand and is released after 10 minutes unused.
- Campaign recipient exports cache at most 3 campaigns instead of growing without limit.
- The unused googleapis dependency (200MB of node_modules, ~60MB of heap if ever imported) is removed; google-auth-library, which the inbox OAuth flow actually uses, is now declared directly and a lint rule stops googleapis creeping back in.
- Restarting a machine from Memory Diagnostics now shows "Machine restarted" with a prompt to log in again, instead of "Restart failed", when the restart logs you out.
- Walk CSV files import with dates in either yyyy-MM-dd or dd/MM/yyyy form, so NGX-exported CSVs round-trip cleanly.
- Fly machine history gains 15m and 30m ranges alongside 1h-7d, showing per-scrape (15s) detail for close-up analysis of restarts and upload runs, plus its own Refresh button beside the Range toggle; the history now lives in its own Fly Machine History frame rather than inside Memory Diagnostics.
Technical changes
- playwright, jsdom, mammoth, pdf-parse and xlsx move from top-level imports to dynamic imports at their point of use; verified none of them load when the owning route modules are imported.
- New shared idleCached() helper (single-value async cache with idle-TTL eviction and clear()); the bundled GeoJSON, S3 GeoJSON and district index in areas.ts use it, replacing four permanent module-level caches, and the serving path reuses the bundled parse rather than re-reading the file.
- idleCached() clears via a generation counter so a load in flight when clear() is called cannot re-cache its stale result.
- Brevo campaign recipients rowsCache evicts oldest entries beyond 3 campaigns.
- googleapis removed from server dependencies (nothing imports it; Gmail and Google Cloud go via REST with fetch); google-auth-library promoted to a direct dependency; no-restricted-imports lint rule bans googleapis with a pointer to the fetch pattern.
- The npm "fs" security-holding package removed from dependencies (require("fs") always resolves to the Node builtin); all server package.json versions now pinned exact.
- The restart poll treats a 401 from the machine-state endpoint as restart-complete-but-session-expired (environment target only
- a worker-target 401 keeps polling): polling stops and a success alert explains the figures cannot refresh until a new login.
- parseCsvDate() tries ISO first and falls back to the Walks Manager CSV export format (dd/MM/yyyy), aligning the import path with what the export writes.