08-Jun-2026 — legacy site URL redirect mapping with scraping, auto-mapping, and admin UI #202
build 671 — commit 92f4c30
When migrating a legacy site (e.g. www.kentramblers.org.uk) to NGX Ramblers, external organisations and search engines hold links to specific URLs on the old site. This feature ensures those links continue to work by automatically discovering, mapping, and redirecting legacy URLs to their new equivalents.
Navigate to Admin > Legacy Redirects and select the Scrape tab.
- Enter the legacy domain (e.g.
www.kentramblers.org.uk) - Configure options: max pages to crawl, delay between requests, and whether to respect robots.txt
- Click Run Scrape to start the crawler
- Real-time progress is displayed via WebSocket as the crawler discovers pages
- Fragment identifiers (e.g.
#WW) are recorded as separate mappable URLs - Metadata is captured for each URL: page title, HTTP status, and content type
The scraper can be re-run at any time to discover new or changed pages. Existing URLs are preserved and only new ones are added.
After scraping, click Auto-Map to run the mapping algorithm. It matches legacy URLs to new NGX pages using multiple strategies:
- Walk URL match (high confidence) — matches against imported walk/event URLs
- Slug match (high confidence) — compares normalised path slugs against CMS page paths
- Title similarity (medium confidence) — token overlap between scraped titles and page names
- Path pattern (medium confidence) — recognises common patterns like
/walks/*,/gallery/*,/contact*
Each mapping receives a confidence score (high, medium, low, or unmapped).
The Mappings tab shows all discovered URLs with their proposed targets:
- Filter by status (pending, accepted, ignored) or confidence level
- Search across URLs and titles
- Sort by any column
- Accept a mapping to activate the redirect
- Edit the target URL with autocomplete against all known NGX pages and walks
- Ignore URLs that don't need redirects (e.g. admin pages)
- Bulk Accept all high-confidence mappings with one click
- Delete unwanted entries
The Summary tab shows a dashboard with:
- Total URLs discovered
- Mapped vs unmapped counts
- Progress bar showing accepted/pending/ignored breakdown
- Breakdown by confidence level and status
Once mappings are accepted, the system serves 301 permanent redirects for matching incoming requests. This happens automatically via Express middleware — no additional configuration is needed.
- Redirects are cached in memory and refreshed every 5 minutes
- Redirect hit counts are tracked for monitoring (batched writes every 60 seconds)
- API requests (
/api/*) are never redirected
Each NGX site can have its own set of legacy URL mappings. To configure which legacy
domains should trigger redirect lookups, add a legacy-redirect config key in the
system settings with:
{
"legacyDomains": ["www.kentramblers.org.uk", "kentramblers.org.uk"],
"cacheRefreshMinutes": 5,
"hitFlushIntervalSeconds": 60
}
Multiple legacy domains are supported per site.
Shared Model:
projects/ngx-ramblers/src/app/models/legacy-url-redirect.model.ts— interfaces and enums
Server — Database:
server/lib/mongo/models/legacy-url-mapping.ts— Mongoose schema (collection: legacyUrlMappings)server/lib/mongo/models/legacy-scrape-run.ts— Mongoose schema (collection: legacyScrapeRuns)
Server — Routes:
server/lib/mongo/routes/legacy-url-mapping.ts— CRUD + /bulk-update-status, /auto-map, /summary, /target-urlsserver/lib/mongo/routes/legacy-scrape-run.ts— CRUD routes
Server — Core Logic:
server/lib/legacy-redirect/redirect-middleware.ts— Express middleware with in-memory cache and batched hit countingserver/lib/legacy-redirect/auto-mapper.ts— multi-strategy mapping algorithmserver/lib/legacy-redirect/legacy-url-scraper.ts— starts at the home page and follows links outward to find every internal page, fetching each one via the integration workerserver/lib/legacy-redirect/legacy-redirect-ws-handler.ts— WebSocket handler for long-running scrape operations
Frontend — Services:
projects/ngx-ramblers/src/app/services/legacy-redirect/legacy-url-mapping.service.tsprojects/ngx-ramblers/src/app/services/legacy-redirect/legacy-scrape-run.service.ts
Frontend — Admin Component:
projects/ngx-ramblers/src/app/pages/admin/legacy-redirects/legacy-redirects.component.tsprojects/ngx-ramblers/src/app/pages/admin/legacy-redirects/legacy-redirects.component.sassprojects/ngx-ramblers/src/app/models/config.model.ts— added LEGACY_REDIRECT ConfigKeyprojects/ngx-ramblers/src/app/models/websocket.model.ts— added LEGACY_URL_SCRAPE EventTypeprojects/ngx-ramblers/src/app/modules/admin/admin-routing.module.ts— added legacy-redirects routeserver/lib/mongo/controllers/extended-group-event.ts— exported convertTitleToSlug for reuseserver/lib/server.ts— registered routes and redirect middlewareserver/lib/websockets/websocket-server.ts— registered WebSocket handlerlegacyUrlMappings— compound unique index on (legacyDomain, legacyPath, legacyFragment)legacyScrapeRuns— scrape operation history with audit logsGET/POST/PUT/DELETE /api/database/legacy-url-mapping— standard CRUDPOST /api/database/legacy-url-mapping/bulk-update-status— bulk status changesPOST /api/database/legacy-url-mapping/auto-map— trigger auto-mappingGET /api/database/legacy-url-mapping/summary— aggregate countsGET /api/database/legacy-url-mapping/target-urls— autocomplete dataGET/POST/PUT/DELETE /api/database/legacy-scrape-run— scrape history CRUDWebSocket:
LEGACY_URL_SCRAPEevent type for long-running scrape+map operations