# 08-Jun-2026 — legacy site URL redirect mapping with scraping, auto-mapping, and admin UI [#202](https://github.com/nbarrett/ngx-ramblers/issues/202)

## [build 671](https://github.com/nbarrett/ngx-ramblers/actions/runs/27146990563) — [commit 92f4c30](https://github.com/nbarrett/ngx-ramblers/commit/92f4c3010b26acd2ad99ecd4b8f6bac0c8e42414)

_____

When migrating a legacy site (e.g. www.kentramblers.org.uk) to NGX Ramblers, external
organisations and search engines hold links to specific URLs on the old site. This feature
ensures those links continue to work by automatically discovering, mapping, and redirecting
legacy URLs to their new equivalents.

Navigate to **Admin > Legacy Redirects** and select the **Scrape** tab.

- Enter the legacy domain (e.g. `www.kentramblers.org.uk`)
- Configure options: max pages to crawl, delay between requests, and whether to respect robots.txt
- Click **Run Scrape** to start the crawler
- Real-time progress is displayed via WebSocket as the crawler discovers pages
- Fragment identifiers (e.g. `#WW`) are recorded as separate mappable URLs
- Metadata is captured for each URL: page title, HTTP status, and content type

The scraper can be re-run at any time to discover new or changed pages. Existing URLs are
preserved and only new ones are added.

After scraping, click **Auto-Map** to run the mapping algorithm. It matches legacy URLs
to new NGX pages using multiple strategies:

- **Walk URL match** (high confidence) — matches against imported walk/event URLs
- **Slug match** (high confidence) — compares normalised path slugs against CMS page paths
- **Title similarity** (medium confidence) — token overlap between scraped titles and page names
- **Path pattern** (medium confidence) — recognises common patterns like `/walks/*`, `/gallery/*`, `/contact*`

Each mapping receives a confidence score (high, medium, low, or unmapped).

The **Mappings** tab shows all discovered URLs with their proposed targets:

- **Filter** by status (pending, accepted, ignored) or confidence level
- **Search** across URLs and titles
- **Sort** by any column
- **Accept** a mapping to activate the redirect
- **Edit** the target URL with autocomplete against all known NGX pages and walks
- **Ignore** URLs that don't need redirects (e.g. admin pages)
- **Bulk Accept** all high-confidence mappings with one click
- **Delete** unwanted entries

The **Summary** tab shows a dashboard with:

- Total URLs discovered
- Mapped vs unmapped counts
- Progress bar showing accepted/pending/ignored breakdown
- Breakdown by confidence level and status

Once mappings are accepted, the system serves **301 permanent redirects** for matching
incoming requests. This happens automatically via Express middleware — no additional
configuration is needed.

- Redirects are cached in memory and refreshed every 5 minutes
- Redirect hit counts are tracked for monitoring (batched writes every 60 seconds)
- API requests (`/api/*`) are never redirected

Each NGX site can have its own set of legacy URL mappings. To configure which legacy
domains should trigger redirect lookups, add a `legacy-redirect` config key in the
system settings with:

```json
{
"legacyDomains": ["www.kentramblers.org.uk", "kentramblers.org.uk"],
"cacheRefreshMinutes": 5,
"hitFlushIntervalSeconds": 60
}
```

Multiple legacy domains are supported per site.

---

**Shared Model:**
- `projects/ngx-ramblers/src/app/models/legacy-url-redirect.model.ts` — interfaces and enums

**Server — Database:**
- `server/lib/mongo/models/legacy-url-mapping.ts` — Mongoose schema (collection: legacyUrlMappings)
- `server/lib/mongo/models/legacy-scrape-run.ts` — Mongoose schema (collection: legacyScrapeRuns)

**Server — Routes:**
- `server/lib/mongo/routes/legacy-url-mapping.ts` — CRUD + /bulk-update-status, /auto-map, /summary, /target-urls
- `server/lib/mongo/routes/legacy-scrape-run.ts` — CRUD routes

**Server — Core Logic:**
- `server/lib/legacy-redirect/redirect-middleware.ts` — Express middleware with in-memory cache and batched hit counting
- `server/lib/legacy-redirect/auto-mapper.ts` — multi-strategy mapping algorithm
- `server/lib/legacy-redirect/legacy-url-scraper.ts` — starts at the home page and follows links outward to find every internal page, fetching each one via the integration worker
- `server/lib/legacy-redirect/legacy-redirect-ws-handler.ts` — WebSocket handler for long-running scrape operations

**Frontend — Services:**
- `projects/ngx-ramblers/src/app/services/legacy-redirect/legacy-url-mapping.service.ts`
- `projects/ngx-ramblers/src/app/services/legacy-redirect/legacy-scrape-run.service.ts`

**Frontend — Admin Component:**
- `projects/ngx-ramblers/src/app/pages/admin/legacy-redirects/legacy-redirects.component.ts`
- `projects/ngx-ramblers/src/app/pages/admin/legacy-redirects/legacy-redirects.component.sass`

- `projects/ngx-ramblers/src/app/models/config.model.ts` — added LEGACY_REDIRECT ConfigKey
- `projects/ngx-ramblers/src/app/models/websocket.model.ts` — added LEGACY_URL_SCRAPE EventType
- `projects/ngx-ramblers/src/app/modules/admin/admin-routing.module.ts` — added legacy-redirects route
- `server/lib/mongo/controllers/extended-group-event.ts` — exported convertTitleToSlug for reuse
- `server/lib/server.ts` — registered routes and redirect middleware
- `server/lib/websockets/websocket-server.ts` — registered WebSocket handler

- `legacyUrlMappings` — compound unique index on (legacyDomain, legacyPath, legacyFragment)
- `legacyScrapeRuns` — scrape operation history with audit logs

- `GET/POST/PUT/DELETE /api/database/legacy-url-mapping` — standard CRUD
- `POST /api/database/legacy-url-mapping/bulk-update-status` — bulk status changes
- `POST /api/database/legacy-url-mapping/auto-map` — trigger auto-mapping
- `GET /api/database/legacy-url-mapping/summary` — aggregate counts
- `GET /api/database/legacy-url-mapping/target-urls` — autocomplete data
- `GET/POST/PUT/DELETE /api/database/legacy-scrape-run` — scrape history CRUD
- WebSocket: `LEGACY_URL_SCRAPE` event type for long-running scrape+map operations