08-Jun-2026 — legacy site URL redirect mapping with scraping, auto-mapping, and admin UI #202

build 671 — commit 92f4c30

When migrating a legacy site (e.g. www.kentramblers.org.uk) to NGX Ramblers, external organisations and search engines hold links to specific URLs on the old site. This feature ensures those links continue to work by automatically discovering, mapping, and redirecting legacy URLs to their new equivalents.

Navigate to Admin > Legacy Redirects and select the Scrape tab.

• Enter the legacy domain (e.g. www.kentramblers.org.uk)
• Configure options: max pages to crawl, delay between requests, and whether to respect robots.txt
• Click Run Scrape to start the crawler
• Real-time progress is displayed via WebSocket as the crawler discovers pages
• Fragment identifiers (e.g. #WW) are recorded as separate mappable URLs
• Metadata is captured for each URL: page title, HTTP status, and content type

The scraper can be re-run at any time to discover new or changed pages. Existing URLs are preserved and only new ones are added.

After scraping, click Auto-Map to run the mapping algorithm. It matches legacy URLs to new NGX pages using multiple strategies:

• Walk URL match (high confidence) — matches against imported walk/event URLs
• Slug match (high confidence) — compares normalised path slugs against CMS page paths
• Title similarity (medium confidence) — token overlap between scraped titles and page names
• Path pattern (medium confidence) — recognises common patterns like /walks/*, /gallery/*, /contact*

Each mapping receives a confidence score (high, medium, low, or unmapped).

The Mappings tab shows all discovered URLs with their proposed targets:

• Filter by status (pending, accepted, ignored) or confidence level
• Search across URLs and titles
• Sort by any column
• Accept a mapping to activate the redirect
• Edit the target URL with autocomplete against all known NGX pages and walks
• Ignore URLs that don't need redirects (e.g. admin pages)
• Bulk Accept all high-confidence mappings with one click
• Delete unwanted entries

The Summary tab shows a dashboard with:

• Total URLs discovered
• Mapped vs unmapped counts
• Progress bar showing accepted/pending/ignored breakdown
• Breakdown by confidence level and status

Once mappings are accepted, the system serves 301 permanent redirects for matching incoming requests. This happens automatically via Express middleware — no additional configuration is needed.

• Redirects are cached in memory and refreshed every 5 minutes
• Redirect hit counts are tracked for monitoring (batched writes every 60 seconds)
• API requests (/api/*) are never redirected

Each NGX site can have its own set of legacy URL mappings. To configure which legacy domains should trigger redirect lookups, add a legacy-redirect config key in the system settings with:

{
"legacyDomains": ["www.kentramblers.org.uk", "kentramblers.org.uk"],
"cacheRefreshMinutes": 5,
"hitFlushIntervalSeconds": 60
}

Multiple legacy domains are supported per site.

Shared Model:

• projects/ngx-ramblers/src/app/models/legacy-url-redirect.model.ts — interfaces and enums

Server — Database:

• server/lib/mongo/models/legacy-url-mapping.ts — Mongoose schema (collection: legacyUrlMappings)
• server/lib/mongo/models/legacy-scrape-run.ts — Mongoose schema (collection: legacyScrapeRuns)

Server — Routes:

• server/lib/mongo/routes/legacy-url-mapping.ts — CRUD + /bulk-update-status, /auto-map, /summary, /target-urls
• server/lib/mongo/routes/legacy-scrape-run.ts — CRUD routes

Server — Core Logic:

• server/lib/legacy-redirect/redirect-middleware.ts — Express middleware with in-memory cache and batched hit counting
• server/lib/legacy-redirect/auto-mapper.ts — multi-strategy mapping algorithm
• server/lib/legacy-redirect/legacy-url-scraper.ts — starts at the home page and follows links outward to find every internal page, fetching each one via the integration worker
• server/lib/legacy-redirect/legacy-redirect-ws-handler.ts — WebSocket handler for long-running scrape operations

Frontend — Services:

• projects/ngx-ramblers/src/app/services/legacy-redirect/legacy-url-mapping.service.ts
• projects/ngx-ramblers/src/app/services/legacy-redirect/legacy-scrape-run.service.ts

Frontend — Admin Component:

• projects/ngx-ramblers/src/app/pages/admin/legacy-redirects/legacy-redirects.component.ts

• projects/ngx-ramblers/src/app/pages/admin/legacy-redirects/legacy-redirects.component.sass

• projects/ngx-ramblers/src/app/models/config.model.ts — added LEGACY_REDIRECT ConfigKey

• projects/ngx-ramblers/src/app/models/websocket.model.ts — added LEGACY_URL_SCRAPE EventType

• projects/ngx-ramblers/src/app/modules/admin/admin-routing.module.ts — added legacy-redirects route

• server/lib/mongo/controllers/extended-group-event.ts — exported convertTitleToSlug for reuse

• server/lib/server.ts — registered routes and redirect middleware

• server/lib/websockets/websocket-server.ts — registered WebSocket handler

• legacyUrlMappings — compound unique index on (legacyDomain, legacyPath, legacyFragment)

• legacyScrapeRuns — scrape operation history with audit logs

• GET/POST/PUT/DELETE /api/database/legacy-url-mapping — standard CRUD

• POST /api/database/legacy-url-mapping/bulk-update-status — bulk status changes

• POST /api/database/legacy-url-mapping/auto-map — trigger auto-mapping

• GET /api/database/legacy-url-mapping/summary — aggregate counts

• GET /api/database/legacy-url-mapping/target-urls — autocomplete data

• GET/POST/PUT/DELETE /api/database/legacy-scrape-run — scrape history CRUD

• WebSocket: LEGACY_URL_SCRAPE event type for long-running scrape+map operations