{"id":"69da90a1b519c70a7a3ad3e8","title":"2026 04 11 Build 586","path":"how-to/committee/release-notes/2026-04-11-build-586","contentMarkdown":"# 11-Apr-2026 — Incremental S3 backup aligned with MongoDB backup timestamps [#187](https://github.com/nbarrett/ngx-ramblers/issues/187)\n\n## [build 586](https://github.com/nbarrett/ngx-ramblers/actions/runs/24288444538) — [commit 37ea501](https://github.com/nbarrett/ngx-ramblers/commit/37ea501064ae59aba57af2a779a3667283b3b803)\n\n_____\n\n## Consolidate admin UI, per-object diagnostics, shared-bucket dedupe and manifest analysis\n\n### What changed for users\n\n- Backup admin tabs reduced from six to four. The standalone **S3 Backup** and **Backups** tabs are gone; their capabilities now live where they logically belong: the consolidated Backup/Restore flow covers S3 object sync automatically, bulk deletion of stored Mongo backup files is a collapsible **Manage stored backups** panel on the Restore tab, and the S3 snapshot analysis and deletion actions are merged directly onto the matching rows in Backup/Restore History\n- **One row per backup operation** in History. Each session row now shows the matching S3 snapshot's actions inline — View Logs, Analyse S3 snapshot, Delete S3 snapshot — so a Mongo backup and its aligned S3 manifest no longer duplicate each other. Sessions without a matching manifest (e.g. restores, failed S3 phases, Mongo-only backups) still render with just the View Logs button\n- **Analyse S3 snapshot** opens an inline panel with three tables: breakdown by top-level prefix, breakdown by file extension, and the twenty largest individual objects. Surfaces what's actually driving a bucket's size so a ballooning backup is easy to diagnose\n- **Environment and Collections selectors now carry their own label** that reads e.g. \"Environments (3 of 14 selected)\" and updates live as the selection changes, replacing the previous static \"Environment(s)\" heading and the redundant \"n environments selected\" helper text\n- **History tab gains a single environment filter** that applies to both Mongo sessions and (when shown) S3 manifest data. The filter uses the same select-all-capable multi-select component as Backup, and an empty filter shows everything with an explicit \"No filter applied — showing sessions for all environments\" helper line\n- **Backup and restore operations stream periodic S3 progress** into the session log (every few seconds and at completion), so the admin UI History row reveals how many of how many objects have been processed rather than a silent gap until the phase finishes\n- **Shared S3 buckets** across multiple environments no longer produce duplicate manifest chains or risk destructive in-place restores. When two environments point at the same source bucket, the backup runs once under the alphabetically-first (\"primary\") site with a note in the session log, and the restore applies the snapshot once to the shared physical bucket\n- **\"In progress\" session status pill** no longer wraps onto two lines when the Actions column is narrow; the spinner and text stay on one line\n- **Expanded logs on an in-progress session** now show a \"Waiting for log output...\" placeholder until the first log line arrives, instead of a visually empty panel\n- **Auto-refresh** on the History tab now reloads S3 manifests alongside sessions, so a newly completed backup's Analyse and Delete actions appear on its row without the user having to leave and re-enter the tab\n- **CLI `backup s3 analyse `** command runs the same analysis (prefix, extension, top-N) from the terminal with optional `--timestamp`, `--depth` and `--top` flags, giving operators a quick way to inspect a snapshot without opening the admin UI\n\n### Technical — s3-backup-service\n\n- Add a `ProgressCallback` type and thread it through `performIncrementalBackup`, `performRestore`, `startS3Backup` and `startS3Restore` so callers can subscribe to aggregate progress messages without touching the stderr debug log\n- Enable the service's `debug()` logger and emit per-object lines: `COPY START`, `COPY DONE`, `COPY FAIL`, `COPY DRY-RUN`, `SKIPPED`, `ALREADY-OK`, `COPY FROM snapshot`, `COPY FROM earlier`, `MISS`, `DELETE-EXTRA` — each with the `[site] [N/total]` prefix, file size and eTag deltas where relevant\n- Throttle progress callbacks via `PROGRESS_EVERY_MS` (3 seconds) plus a guaranteed final message, so the session log gets useful but not spammy updates\n- Wrap every `CopyObjectCommand` in `copyObjectWithETagGuard`, a recursive async helper that passes `CopySourceIfMatch: expectedETag` and catches 412 Precondition Failed by re-heading the object, updating the eTag and retrying up to 3 times — prevents a silent integrity mismatch when an object is mutated (e.g. resized) between list and copy\n- Record the eTag that was physically copied rather than the one seen at listing time, so the manifest entry is always consistent with the stored bytes\n- Extract `rawSiteConfigs()` and `dedupeSiteConfigsByBucket()`, make `siteConfigs()` return one entry per unique source bucket with a new `sharedWith?: string[]` field populated with the aliased environment names, and teach `siteConfigFor()` to resolve either the primary or any alias to the primary's config\n- Teach `manifestByTimestamp()` to resolve aliases to the primary site before querying so cross-environment restore through a shared bucket finds the right manifest\n\n### Technical — consolidated backup/restore flow\n\n- `BackupAndRestoreService.executeBackup` and `executeRestore` now pass `(message) => this.addLog(sessionId, message)` as the `ProgressCallback` into `startS3Backup` / `startS3Restore`, so aggregate S3 progress flows into the same session log that the admin UI streams\n- Cross-environment restore now looks up the source environment from the backup file name (via the existing `extractSourceEnvironmentFromBackupName` helper) and passes it alongside the target, so a restore triggered through the main Restore tab automatically performs a cross-environment S3 restore when the selected backup came from a different source\n\n### Technical — manifest analysis\n\n- New pure module `projects/ngx-ramblers/src/app/functions/s3-manifest-analysis.ts` exports `groupEntriesByPrefix(entries, depth)`, `groupEntriesByExtension(entries)` and `topLargestEntries(entries, count)`. Shared verbatim by the CLI analyse command and by the admin UI expand panel, so the three analyses live in exactly one place\n- `S3ManifestBreakdown` interface moved from the functions module to `backup-session.model.ts` in line with the codebase rule that exported interfaces live in model files\n\n### Technical — CLI\n\n- New `backup s3 analyse ` command (alias `analyze`) with `--timestamp`, `--depth` and `--top` flags. Prints manifest summary, prefix and extension breakdowns, and the top N largest objects. Reuses the shared analysis helpers so behaviour matches the admin UI exactly\n- Retired the local `PrefixBucket` helpers in the CLI command in favour of the shared module\n\n### Technical — admin UI\n\n- `BackupRestoreTab` enum reduced to `BACKUP`, `RESTORE`, `HISTORY`, `SETTINGS`. The `S3_BACKUP` and `BACKUPS` entries are gone, as are their entire tab templates and all the state and methods that only served the standalone S3 backup form (`s3BackupAllSites`, `s3BackupDryRun`, `s3BackupInProgress`, `s3BackupResults`, `s3SelectedSite`, `s3RestoreResults`, `startS3Backup()`, `filteredS3Manifests()`)\n- Restore tab gains a **Manage stored backups** collapsible button that reveals the Mongo backup bulk selector and delete UI previously living in the Backups tab\n- History tab's session row template now includes `manifestForSession(session)` lookup and conditionally renders Analyse and Delete buttons plus the inline expand panel when a matching manifest exists. Matching is by timestamp parsed from `session.sessionId` (format `backup-{YYYY-MM-DD-HH-mm-ss}-{env}-{db}`) against `manifest.mongoTimestamp` and `manifest.site === session.environment`\n- `handleTabChange` now loads S3 manifests when entering the History tab (was previously the S3 Backup tab) and the auto-refresh subscription also reloads manifests on every tick so newly completed backups surface their actions without the user leaving and returning to the tab\n- `EnvironmentSelectComponent` gains a `label?: string` input and an internal `labelText()` helper that renders the label with a live `\"(X of Y selected)\"` suffix in multiple mode. The four existing call sites in the admin component drop their external `