{"id":"6939c5bfab8ed1a68f99cd8e","title":"2025 12 10","path":"how-to/technical-articles/2025-12-10","contentMarkdown":"# Content Migration Guide\n\n## Overview\n\nThe Content Migration system provides a template-based workflow for migrating pages from external websites into your NGX-Ramblers site structure. Instead of hard-coded parsing logic or abstract transformation steps, you:\n\n1. **Build a visual template** using the page editor\n2. **Configure data mappings** for each element (where content comes from)\n3. **Apply the template** to migrate multiple pages automatically\n\n### Key Features\n\n- **Visual template creation**: What you build is what migrated pages will look like\n- **Declarative mappings**: Configure extraction rules without writing code\n- **Dynamic content generation**: Automatically create nested rows from scraped content\n- **Reusable templates**: One template can migrate hundreds of pages\n- **Parent page support**: Create index pages with action buttons\n- **Pattern matching**: Extract specific content using text/image patterns\n\n### Template Types\n\n- **Shared Fragment**: Reusable components (headers, footers, sidebars) inserted into multiple pages\n- **User Template**: Custom page layouts for quickly building new pages with consistent structure\n- **Migration Template**: Templates with data mappings for migrating content from external sources\n\n## Quick Start\n\n### 1. Create a Migration Template\n\n1. Navigate to a page in site edit mode (e.g., `/routes/example-route`)\n2. Build your ideal structure using the page editor:\n - Add rows (Text, Location, Map, Index)\n - Arrange them exactly as you want migrated pages to look\n - Configure row settings (height, columns, visibility)\n3. Enable **Template Mode**:\n - Toggle \"Template options\" switch (top right)\n - Click the **\"Migration template\"** badge button\n4. Enable **Mapping Mode**:\n - Toggle **\"Mapping mode\"** switch\n - This reveals mapping configuration UI for each row\n5. Configure mappings (see sections below)\n6. Click **\"Create template\"** to publish\n\n### 2. Configure Parent Pages\n\nNavigate to **Admin → System Settings → Migration → Settings tab**\n\n1. Add a parent page for your migration\n2. Configure the settings (see Parent Page Configuration section)\n3. Select your migration template from the \"Override Template\" dropdown\n4. Save configuration\n\n### 3. Run Migration\n\n1. Stay in Migration Settings\n2. Configure which pages to migrate\n3. Click **Run migration**\n4. Monitor progress in the Activity tab\n\n## Parent Page Configuration\n\nEach migration site can have multiple **Parent Pages** - these are index/category pages that contain lists of child pages to migrate.\n\n### Parent Page Settings\n\nFor each parent page, configure:\n\n**URL Pattern**: The parent page URL\n- Example: `/walks/routes`\n- The source page containing links to child pages\n\n**Path Prefix**: Prefix for child page paths\n- Example: `walks/routes/`\n- Used to construct destination paths for migrated pages\n\n**Link Selector**: CSS selector for child page links (optional)\n- Defaults to content area if empty\n- Example: `.route-list a` to target specific links\n\n**Max Child Pages**: Limit number of children to migrate (optional)\n- Example: `50` to migrate only first 50 pages\n- Leave empty to migrate all found pages\n\n### Migrate Parent Page Options\n\nControls what happens to the parent page itself during migration:\n\n#### Not migrated (default)\n\n- The parent page **itself is skipped**\n- Child pages **are still migrated**\n- Use when: Parent page doesn't exist or you'll create it manually\n- Example: Skip `/walks/routes` but migrate `/walks/routes/wealdway`\n\n#### As-is\n\n- Parent page **is migrated** with its original content\n- Content extracted and migrated like any other page\n- Use when: Parent page has intro text or useful content to preserve\n- Example: `/walks/routes` has introductory text about routes\n\n#### With Links as Action Buttons Row\n\n- Parent page **is migrated** with child page links converted to action buttons\n- Creates an **auto-generated index page**\n- Use when: You want a navigation/directory page\n- Example: `/walks/routes` becomes a page with [Weald Way] [North Downs Way] [Thames Path] buttons\n\n### Override Template (optional)\n\nBy default, child pages use the site's default migration template. Use this field to specify a **different template** for children under this parent page.\n\n**Use cases:**\n- Different page structures for different sections\n - Routes use \"routes-template\"\n - Regional routes use \"regional-routes-template\"\n- Special layouts for specific categories\n - Index pages with summaries\n - Detail pages with maps\n\n**How it works:**\n- Select a template from the dropdown\n- All child pages under this parent use the selected template\n- If empty, child pages use the site's default template\n- Template selection shows all available migration templates from your `fragments/templates/` folder\n\n**Example configuration:**\n```\nParent Page: /walks/routes\n├─ Migrate parent page: With Links as Action Buttons Row\n├─ Override Template: routes-template\n└─ Max child pages: 50\n\nResult:\n- /walks/routes → Index page with action buttons\n- /walks/routes/wealdway → Uses routes-template\n- /walks/routes/north-downs-way → Uses routes-template\n```\n\n## Template Creation Workflow\n\n### UI Components\n\n#### Template Options Panel\n\nLocated at top of page editor when \"Template options\" toggle is ON:\n\n```\n[Shared fragment][User template][Migration template] [Remove] [Create template] [Mapping mode ☑]\n```\n\n- **Type badges**: Select template type\n- **Remove**: Clear template configuration\n- **Create template**: Publish to fragment library\n- **Mapping mode**: Toggle migration mapping UI (migration templates only)\n\n#### Template Library Panel\n\nShows published templates for reuse:\n\n```\nTemplate library\n├─ Dropdown (shows fragment paths)\n└─ [Replace] [Append]\n```\n\n- **Replace**: Replace current page content with template\n- **Append**: Append template to current page\n\n#### Paste Page Content\n\nThe page editor includes a **Paste page content** badge next to save/revert buttons. Use this to:\n\n- Clone complex templates between environments\n- Try out example content (like `docs/routes-template-page-content.json`)\n- Restore earlier snapshots from API explorer\n\nThe parser automatically unwraps API response shapes, so you can paste output from `/api/page-content?path=...` directly.\n\n## Data Mapping Configuration\n\n### Row-Level Mappings\n\nWhen Mapping Mode is enabled, each row shows configuration based on its type.\n\n#### Location Row Mapping\n\n```\nSource type: [Extract] [Static] [Metadata]\n☑ Extract from content\n☑ Hide location row\nDefault location: [____________]\n```\n\n**Extract from content**: Automatically extract location from narrative text\n- Looks for postcodes, grid references, place names\n- Geocodes to latitude/longitude\n\n**Hide location row**: Make the row invisible\n- Row data still available for Map rows via `useLocationFromRow`\n- Use when location is only for map positioning, not display\n\n**Default location**: Fallback location if extraction fails\n- Example: \"Tonbridge, Kent\" for Kent-based routes\n- Ensures map always has a center point\n\n#### Map Row Mapping\n\n```\n☑ Extract GPX from content\nGPX file path: [____________]\nUse location from row: [0]\n```\n\n**Extract GPX from content**: Pull GPX file from source page\n- Searches for .gpx file links in source HTML\n- Downloads and processes route data\n\n**GPX file path**: Static GPX file path (alternative to extraction)\n- Example: `/assets/routes/wealdway.gpx`\n\n**Use location from row**: Row index to get location coordinates from\n- Example: `0` references first row (Location row)\n- Map centers on this location\n- Supports nested row references (e.g., `0.1` = row 0, nested row 1)\n\n#### Metadata Row Mapping\n\n```\nSource type: [Metadata ▼]\nField: [Title | Path | Migration note]\nPrefix: [Automatically migrated from]\nDate format: [yyyy-LL-dd HH:mm]\n```\n\n**Field options:**\n- `Title`: Page title from scraped metadata\n- `Path`: Source page URL/path\n- `Migration note`: \"Migrated from [URL] on [date]\" footer\n\n**Customization:**\n- `metadataPrefix`: Custom prefix text (default: \"Migrated from\")\n- `metadataDateFormat`: Luxon format string (default: \"yyyy-LL-dd HH:mm\")\n\n#### Index Row Mapping\n\nIndex rows use existing Index row settings:\n- Content types (walks, routes, social events)\n- Render modes (carousel, grid, list)\n- Map settings (clustering, markers)\n\nNo additional mapping configuration required.\n\n### Column-Level Mappings\n\nFor rows with columns containing nested rows, column-level mapping configuration appears.\n\n#### Column Mapping UI\n\n```\nColumn 2 Mapping\n─────────────────────────────────────\nSource Type: [Extract | Static | Not mapped]\nContent Type: [Text | Image | Mixed]\n\nNested Rows Configuration\n─────────────────────────────────────\nContent source: [Extract from content ▼]\nPacking behavior: [One row per item ▼]\n☑ Group text with images\n\nDocumentation notes\n[Notes about what this mapping does...]\n```\n\n#### Content Source Options\n\n**Not mapped**\n- Column stays exactly as designed in template\n- Use for static content (headers, footers, shared fragments)\n\n**Extract from content**\n- Populate from scraped page data\n- Requires Content Type and packing configuration\n\n**Static content**\n- Use template content as-is\n- Allows adding documentation notes\n\n#### Content Type (what to extract)\n\n**Remaining images**\n- Images not yet consumed by other mappings\n- Recommended for galleries to avoid duplication\n- Each column gets unique images\n\n**Remaining text**\n- Text segments (paragraphs, headings) still unused\n- Recommended catch-all for narrative text\n- Prevents double-rendering content\n\n**All content**\n- Every text and image segment in source order\n- Use when column should mirror original story exactly\n\n**All images**\n- Every image, even if already used elsewhere\n- Use when multiple galleries need same images\n\n**Pattern match**\n- Only segments matching custom pattern\n- Supports regex, filename glob, CSS selector\n- Pattern input appears when selected\n\n#### Packing Behavior (when extracting)\n\n**One row per item**\n- Create separate nested row for each extracted item\n- Example: One nested row per image\n- Repeats until no matches remain\n\n**All in one row**\n- Put all extracted content in single nested row\n- Example: All narrative text in one row\n- Produces single combined row\n\n**Collect with breaks**\n- Group content based on document structure\n- Smart breaking on headings, images, paragraphs\n- Configurable break points\n\n#### Additional Options\n\n**Group text with images** (`groupTextWithImage`)\n- Associates caption text with each image\n- Looks for text immediately following image\n- Critical for preserving image captions\n\n**Hide row if empty** (`hideIfEmpty`)\n- Removes entire row when no content extracted\n- Prevents blank placeholders\n- Useful for optional sections\n\n**Documentation notes**\n- Explain what this mapping does\n- Visible in mapping mode\n- Example: \"Points-of-interest sidebar with remaining images\"\n\n### Dynamic Target Rows\n\nWhen Mapping mode is enabled, you can mark any nested row as a **Dynamic target** using the badge button in the nested row header (stacked-layers icon).\n\n**What it does:**\n- Marks which nested row will be cloned and populated with extracted content\n- The selected row becomes the template for dynamically generated rows\n- Only one nested row can be the dynamic target per column\n\n**How it works:**\n1. Content Source specifies **what** to extract and **how** to pack it\n2. Dynamic target button specifies **which row structure** to use\n3. When migration runs:\n - Extracts content based on Content Type setting\n - Clones the target nested row\n - Fills clone with extracted content\n - Repeats if packing behavior is \"one-per-item\"\n4. Other nested rows (without button) are static and added after dynamic rows\n\n**Example:**\n\n```\nNested Rows Configuration:\n├─ Content source: remaining-images\n├─ Packing: one-per-item\n│\n├─ Nested Row 1 (Text) ← \"Use as dynamic target\" (active)\n│ └─ Column: image + caption placeholder\n│\n└─ Nested Row 2 (Shared Fragment) ← Static row\n └─ Shared fragment reference\n```\n\n**Result:**\n- Nested row 1 cloned for each remaining image\n- Each clone filled with image + caption\n- Nested row 2 (shared fragment) added once at end\n\n### Column Processing Order\n\n**Important:** When a row has multiple columns with dynamic collection, order matters:\n\n1. **Columns with `remaining-images`** processed **first**\n - Allows images to capture their caption text\n2. **Columns with `remaining-text`** processed **second**\n - Gets narrative text after images claimed captions\n\nThis ensures images grab caption text before remaining-text column consumes all text.\n\n**Implementation:** The system automatically sorts column mappings by priority:\n1. Pattern-match images (specific content)\n2. Remaining-images with `groupTextWithImage: true`\n3. Other content types\n4. Remaining-images without text grouping\n\n## Content Matchers & Patterns\n\n### Text Patterns\n\nUsed when Content Type is `TEXT`:\n\n**ALL_TEXT_UNTIL_IMAGE**\n- Extract all text from current position until first image\n- Commonly used for intro paragraphs\n- Stops at first image boundary\n\n**ALL_TEXT_AFTER_HEADING**\n- Extract heading and following text until next heading or image\n- Good for section extraction\n- Captures semantic blocks\n\n**REMAINING_TEXT**\n- Extract all remaining unused text content\n- Use at end of transformation to capture everything left\n- Ensures no content is lost\n\n**PARAGRAPH**\n- Extract single paragraph or text segment\n- Advances to next segment on each use\n- For granular text extraction\n\n**STARTS_WITH_HEADING**\n- Extract text beginning with markdown heading (# through ######)\n- Filters to heading-led sections\n- Useful for structured content\n\n**CUSTOM_REGEX**\n- Match text using custom regular expression\n- Requires `customRegex` field\n- Powerful for complex extraction patterns\n- Test patterns using regex101.com\n\n### Image Patterns\n\nUsed when Content Type is `IMAGE`:\n\n**FIRST_IMAGE**\n- Extract first unused image\n- Most commonly used pattern\n- Marks image as used\n\n**REMAINING_IMAGES**\n- Extract ALL remaining unused images\n- Marks all as used\n- Use when consuming all remaining images\n\n**ALL_IMAGES**\n- Extract ALL images regardless of used status\n- Does NOT mark as used\n- Useful for showing images in multiple places (carousel + gallery)\n\n**FILENAME_PATTERN**\n- Match images by filename using wildcards\n- Requires `filenamePattern` field\n- Example: `*route-map*.jpg`\n- Case-sensitive matching\n\n**ALT_TEXT_PATTERN**\n- Match images by alt text using regex\n- Requires `altTextPattern` field\n- Example: `^Map of.*`\n- Useful for semantic image selection\n\n### Pattern Usage Table\n\n| Pattern | Use Case | Marks as Used? |\n|---------|----------|----------------|\n| `FIRST_IMAGE` | Single hero image, featured image | Yes |\n| `REMAINING_IMAGES` | Gallery consuming all remaining | Yes |\n| `ALL_IMAGES` | Show all images in carousel, then detail | No |\n| `FILENAME_PATTERN` | Specific image like route map | Yes |\n| `ALT_TEXT_PATTERN` | Images with specific alt text | Yes |\n\n### Understanding \"Used\" vs \"Unused\" Content\n\nThe transformation engine tracks which content has been consumed:\n\n**Text Content:**\n- Starts with all markdown text segments marked as \"unused\"\n- When text pattern matches and extracts content, segments marked as \"used\"\n- `REMAINING_TEXT` extracts all segments still marked \"unused\"\n\n**Image Content:**\n- All scraped images start as \"unused\"\n- `FIRST_IMAGE` extracts first unused image and marks it used\n- `REMAINING_IMAGES` extracts ALL unused images and marks all used\n- `ALL_IMAGES` extracts ALL images but does NOT mark them used (allows reuse)\n\n### Best Practices for Content Extraction\n\n**1. Extract in Order of Specificity**\n```\nStep 1: Extract specific content (e.g., route map by filename)\nStep 2: Extract structured content (e.g., intro text until image)\nStep 3: Extract remaining content (e.g., remaining-text, remaining-images)\n```\n\n**2. Use REMAINING Patterns Last**\nAlways use `REMAINING_TEXT` or `REMAINING_IMAGES` in final rows to ensure no content is lost.\n\n**3. Test Custom Regex Incrementally**\nStart simple and add complexity:\n- Test: `^###` (just headings)\n- Then: `^###\\\\s+.+` (heading with text)\n- Finally: `^###\\\\s+.+[\\\\s\\\\S]*?(?=###|$)` (heading with all text until next heading)\n\n**4. Filename Patterns Are Case-Sensitive**\n- Use lowercase patterns: `*route-map*` not `*Route-Map*`\n- Check actual filenames in Activity log output\n- Wildcards: `*` matches any characters\n\n**5. Extract Images Before Text**\nWhen both use `remaining-*` patterns, ensure images process first to capture captions before text consumes them.\n\n## Common Template Patterns\n\n### Pattern: Two-Column Layout with Featured Image\n\n```\nRow 1: Full-width intro text\nRow 2: Two columns\n ├─ Column 1 (8 cols): Featured image\n └─ Column 2 (4 cols): Remaining content in nested rows\n```\n\n**Mapping:**\n- Row 1, Column 1: Text pattern = `ALL_TEXT_UNTIL_IMAGE`\n- Row 2, Column 1: Image pattern = `FIRST_IMAGE`\n- Row 2, Column 2: Content source = `remaining-text`, Packing = `all-in-one`\n\n### Pattern: Route Page with Hidden Location\n\n```\nRow 1: Location (hidden)\nRow 2: Map (references Row 1)\nRow 3: Two columns (narrative + sidebar)\n ├─ Column 1 (8 cols): Main narrative text\n └─ Column 2 (4 cols): Points of interest images\n```\n\n**Mapping:**\n- Row 1: Extract from content ☑, Hide row ☑\n- Row 2: Use location from row = `0`\n- Row 3, Column 1: Content source = `remaining-text`, Packing = `all-in-one`\n- Row 3, Column 2: Content source = `remaining-images`, Packing = `one-per-item`, Group text with images ☑\n\n### Pattern: Photo Gallery with Map\n\n```\nRow 1: Two columns\n ├─ Column 1 (6 cols): Map showing photo locations\n └─ Column 2 (6 cols): Dynamic gallery (one row per photo)\n```\n\n**Mapping:**\n- Row 1, Column 1: Static (map configured in template)\n- Row 1, Column 2: Content source = `remaining-images`, Packing = `one-per-item`\n\n### Pattern: Index Page with Action Buttons\n\n```\nRow 1: Intro text\nRow 2: Index row (maps + clustering)\nRow 3: Action buttons for child pages\n```\n\n**Parent Page:**\n- Migrate parent page: `With Links as Action Buttons Row`\n- This automatically creates Row 3 with buttons\n\n## Advanced Topics\n\n### Nested Row Mappings Data Model\n\n```typescript\ninterface NestedRowMappingConfig {\n contentSource: 'remaining-images' | 'remaining-text' | 'all-content' | 'all-images' | 'pattern-match';\n packingBehavior: 'one-per-item' | 'all-in-one' | 'collect-with-breaks';\n breakOn?: 'image' | 'heading' | 'paragraph';\n stopOn?: 'image' | 'heading';\n groupTextWithImage?: boolean;\n filenamePattern?: string;\n textPattern?: string;\n imagePattern?: string;\n}\n```\n\n### Column-Level Mappings Data Model\n\n```typescript\ninterface ColumnMappingConfig {\n columnIndex: number;\n sourceType?: 'extract' | 'static' | 'metadata';\n contentType?: 'text' | 'image' | 'mixed';\n extractPreset?: string;\n extractPattern?: string;\n nestedRowMapping?: NestedRowMappingConfig;\n notes?: string;\n}\n```\n\n### Row-Level Mappings Data Model\n\n```typescript\ninterface MigrationTemplateMapping {\n targetRowIndex: number;\n sourceType?: 'extract' | 'static' | 'metadata';\n\n // Type-specific mappings\n location?: MigrationTemplateLocationMapping;\n map?: MigrationTemplateMapMapping;\n index?: MigrationTemplateIndexMapping;\n\n columnMappings?: ColumnMappingConfig[];\n hideIfEmpty?: boolean;\n notes?: string;\n}\n```\n\n### Bootstrap Grid System\n\nRemember the 12-column grid:\n- Full width: 12\n- Half width: 6\n- Two-thirds: 8, one-third: 4\n- Three columns: 4, 4, 4\n\nColumns in a row should sum to 12 for best results.\n\n### Image Styling\n\nImages support:\n- `imageBorderRadius`: Border radius in pixels (default: 6)\n- `alt`: Alt text (auto-populated from scraped alt)\n- `imageSource`: URL to image file\n\n### Content Exclusions\n\nBefore transformation, filter content using site config:\n- `excludeSelectors`: Remove HTML elements before scraping\n- `excludeTextPatterns`: Remove text matching regex\n- `excludeMarkdownBlocks`: Remove exact markdown blocks\n- `excludeImageUrls`: Don't use specific images\n\n## Troubleshooting\n\n### Template Issues\n\n**Template not appearing in library**\n- Check template was published (click \"Create template\")\n- Verify \"Template options\" toggle is ON\n- Check fragment path doesn't conflict with existing content\n\n**Mapping mode not showing**\n- Ensure \"Migration template\" type is selected\n- Toggle \"Mapping mode\" switch\n\n### Content Extraction Issues\n\n**Content not appearing**\n- Check Activity log for matcher debug output\n- Adjust pattern or use \"all\" to see all content first\n- Verify content source (remaining vs all)\n\n**Wrong image selected**\n- Filename pattern too broad or too narrow\n- Check image filenames in scraped content\n- Use more specific patterns\n\n**Images show placeholder text instead of captions**\n- Set `groupTextWithImage: true` in nested row mapping\n- Ensure images processed before remaining-text\n\n**Narrative column includes image caption headings**\n- System auto-sorts to process images first\n- Verify column processing order in mappings\n\n**Nested rows not populating**\n- Check targetRow and targetColumn index (zero-based)\n- Verify dynamic target row is marked\n- Check content source pattern matches content\n\n**Text split incorrectly**\n- Try different text patterns\n- `ALL_TEXT_UNTIL_IMAGE` stops at first image\n- `REMAINING_TEXT` takes everything unused\n\n**Custom regex not matching**\n- Test regex using regex101.com\n- Escape special characters\n- Use `[\\\\s\\\\S]*?` for multiline matching\n- Common patterns:\n - Match heading + text: `^###\\\\s+.+[\\\\s\\\\S]*?(?=###|$)`\n - Match paragraph: `^[^#\\\\n].+(?:\\\\n[^#\\\\n].+)*`\n\n**Filename pattern not matching images**\n- Check actual filenames in scraped content\n- Use wildcards correctly: `*route*` matches any file containing \"route\"\n- Case-sensitive by default\n- Patterns match against full URL path, not just filename\n\n**No nested rows generated**\n- Content type pattern didn't match anything\n- Check debug logs in Activity tab\n- Verify content source (e.g., \"remaining-images\" vs \"all-images\")\n\n**All content in one nested row when expecting multiple**\n- Packing behavior set to \"all-in-one\"\n- Change to \"one-per-item\" for separate rows\n\n### Location & Map Issues\n\n**Location extraction not working**\n- Verify narrative contains postcode or grid reference\n- Check extraction logs in browser console\n- Use default location as fallback\n\n**Map not centering on location**\n- Check `useLocationFromRow` index is correct\n- Verify location row has valid coordinates\n- Review location extraction success in logs\n\n**Blank maps showing**\n- No valid location data extracted\n- Maps now automatically hidden when no coordinates\n- Check location row configuration\n\n## Example Workflows\n\n### Example 1: Migrating Route Pages\n\n**Goal:** Migrate old route pages with location extraction, map display, and narrative content.\n\n**Template Structure:**\n```\nRow 0: Location (hidden, extracted from content)\nRow 1: Map (500px height, uses location from Row 0)\nRow 2: Two columns\n ├─ Column 1 (8 cols): Main narrative text\n └─ Column 2 (4 cols): Points of interest images with captions\nRow 3: Migration note (metadata)\n```\n\n**Mappings:**\n1. Row 0 (Location):\n - Extract from content: ☑\n - Hide row: ☑\n - Default location: \"Kent, England\"\n\n2. Row 1 (Map):\n - Use location from row: `0`\n - Auto-fit bounds: ☑\n\n3. Row 2, Column 1:\n - Source type: Extract\n - Content type: Text\n - Nested rows: Content source = `remaining-text`, Packing = `all-in-one`\n\n4. Row 2, Column 2:\n - Source type: Extract\n - Content type: Mixed\n - Nested rows: Content source = `remaining-images`, Packing = `one-per-item`, Group text ☑\n - Notes: \"Points-of-interest sidebar with image captions\"\n\n5. Row 3 (Metadata):\n - Source type: Metadata\n - Field: Migration note\n - Prefix: \"Migrated from\"\n\n**Parent Page Config:**\n- URL: `/walks/routes`\n- Migrate parent page: `With Links as Action Buttons Row`\n- Override template: `routes-template`\n- Max children: 100\n\n**Result:** All route pages get consistent structure with extracted locations, centered maps, narrative text, and image galleries with preserved captions.\n\n### Example 2: Photo Gallery Migration\n\n**Goal:** Complex gallery with map showing photo locations and dynamic nested rows.\n\n**Template Structure:**\n```\nRow 0: Title and description\nRow 1: Two columns\n ├─ Column 1 (6 cols):\n │ ├─ Nested Row 0: Description text\n │ ├─ Nested Row 1: Location (hidden)\n │ └─ Nested Row 2: Map (references nested row 1)\n └─ Column 2 (6 cols): Dynamic gallery rows (one per photo)\n```\n\n**Mappings:**\n1. Row 1, Column 1:\n - Source type: Static\n - Nested rows configured in template\n\n2. Row 1, Column 2:\n - Source type: Extract\n - Content source: `remaining-images`\n - Packing: `one-per-item`\n - Mark Nested Row 0 as dynamic target\n\n**Result:** Gallery page with map on left, dynamically generated photo rows on right (one row per image).\n\n## Implementation Status\n\n### ✅ Completed\n\n- Template type system (Shared/User/Migration)\n- Template UI with badge button groups\n- Template publishing to fragment library\n- Location row mapping UI\n- Map row mapping UI\n- Metadata row mapping UI\n- Column-level mapping UI for nested rows\n - Content source selection\n - Packing behavior configuration\n - Content type selection\n - Pattern matching support\n- Dynamic target row selection\n- Template descriptions\n- Paste page content functionality\n- Basic template transformation engine (`transformWithTemplate`)\n - Static row cloning\n - Metadata population\n - Location/Map row handling\n- Column mapping data model\n- Migration runtime integration\n - Site-level template selection\n - Parent page template override\n - Template priority over legacy transformations\n- Dynamic nested row extraction\n - Column mappings drive row generation\n - Shared fragments preserved\n - `hideIfEmpty` support\n- Column processing order optimization\n - Images processed before text\n - Caption preservation for image galleries\n\n### 🚧 In Progress\n\n- GPX file extraction from content\n- Advanced pattern matching for text extraction\n- Enhanced location narrative parsing\n\n### 📋 Planned\n\n- Template preview in migration settings\n- Batch migration with templates\n- Template validation\n- Visual mapping UI with drag-and-drop\n- Template marketplace for sharing\n- Migration preview/diff view\n- Rollback capability\n- Template versioning\n- Real-time migration progress tracking\n- Error recovery and retry logic\n\n## File Locations\n\n### Frontend\n\n**Models:**\n- `projects/ngx-ramblers/src/app/models/content-text.model.ts` - Template types, mapping interfaces\n\n**Components:**\n- `projects/ngx-ramblers/src/app/modules/common/dynamic-content/dynamic-content-site-edit.ts` - Template UI, mapping mode\n- `projects/ngx-ramblers/src/app/modules/common/dynamic-content/dynamic-content-site-edit-text-row.ts` - Column mapping UI\n- `projects/ngx-ramblers/src/app/pages/admin/system-settings/migration/migration-settings.ts` - Parent page configuration\n\n**Services:**\n- `projects/ngx-ramblers/src/app/pages/admin/data-population.service.ts` - Template descriptions\n\n### Backend\n\n**Migration Engine:**\n- `server/lib/migration/page-transformation-engine.ts` - Template transformation engine\n - `transformWithTemplate()` method (line ~1600)\n - Column mapping processing (line ~1750)\n - Content extraction and nested row generation\n\n**Migrations:**\n- `server/lib/mongo/migrations/database/20251121090000-rename-fragment-index-to-content-templates.ts` - Template library setup\n\n**Migration Service:**\n- `server/lib/migration/migrate-static-site-engine.ts` - Site migration orchestration\n - Template loading and caching\n - Parent page template override logic\n\n## Routes Template Example\n\nThe repository includes `docs/routes-template-page-content.json` as a reference implementation containing:\n\n- Full `fragments/templates/routes-template` structure\n- Populated rows based on example walk\n- Mapping definitions for title, location, map, narrative, and sidebar\n- Clean markdown with helper text removed\n\n**To use:**\n1. Open `/admin/page-content/fragments/templates/routes-template`\n2. Click **Paste page content**\n3. Paste JSON from `docs/routes-template-page-content.json`\n4. Click **Apply pasted content**\n5. Save template\n\nThis gives you a working baseline to experiment with mappings and configurations.\n\n## Related Documentation\n\n- **Database Migrations**: See `docs/admin/MIGRATIONS.md` for MongoDB schema migrations (different topic)\n- **Phase 7 Overview**: Historical context in `docs/MIGRATION_PHASE_7.md` (superseded by this guide)\n","contentHtml":"

Content Migration Guide

\n

Overview

\n

The Content Migration system provides a template-based workflow for migrating pages from external websites into your NGX-Ramblers site structure. Instead of hard-coded parsing logic or abstract transformation steps, you:

\n
    \n
  1. Build a visual template using the page editor
    2. \n
  2. Configure data mappings for each element (where content comes from)
    3. \n
  3. Apply the template to migrate multiple pages automatically
    4. \n
\n

Key Features

\n\n

Template Types

\n\n

Quick Start

\n

1. Create a Migration Template

\n
    \n
  1. Navigate to a page in site edit mode (e.g., /routes/example-route)
    2. \n
  2. Build your ideal structure using the page editor:\n
    3. \n
  3. Enable Template Mode:\n
    4. \n
  4. Enable Mapping Mode:\n
    5. \n
  5. Configure mappings (see sections below)
    6. \n
  6. Click "Create template" to publish
    7. \n
\n

2. Configure Parent Pages

\n

Navigate to Admin → System Settings → Migration → Settings tab

\n
    \n
  1. Add a parent page for your migration
    2. \n
  2. Configure the settings (see Parent Page Configuration section)
    3. \n
  3. Select your migration template from the "Override Template" dropdown
    4. \n
  4. Save configuration
    5. \n
\n

3. Run Migration

\n
    \n
  1. Stay in Migration Settings
    2. \n
  2. Configure which pages to migrate
    3. \n
  3. Click Run migration
    4. \n
  4. Monitor progress in the Activity tab
    5. \n
\n

Parent Page Configuration

\n

Each migration site can have multiple Parent Pages - these are index/category pages that contain lists of child pages to migrate.

\n

Parent Page Settings

\n

For each parent page, configure:

\n

URL Pattern: The parent page URL

\n\n

Path Prefix: Prefix for child page paths

\n\n

Link Selector: CSS selector for child page links (optional)

\n\n

Max Child Pages: Limit number of children to migrate (optional)

\n\n

Migrate Parent Page Options

\n

Controls what happens to the parent page itself during migration:

\n

Not migrated (default)

\n\n

As-is

\n\n

With Links as Action Buttons Row

\n\n

Override Template (optional)

\n

By default, child pages use the site's default migration template. Use this field to specify a different template for children under this parent page.

\n

Use cases:

\n\n

How it works:

\n\n

Example configuration:

\n
Parent Page: /walks/routes\n├─ Migrate parent page: With Links as Action Buttons Row\n├─ Override Template: routes-template\n└─ Max child pages: 50\n\nResult:\n- /walks/routes → Index page with action buttons\n- /walks/routes/wealdway → Uses routes-template\n- /walks/routes/north-downs-way → Uses routes-template\n
\n

Template Creation Workflow

\n

UI Components

\n

Template Options Panel

\n

Located at top of page editor when "Template options" toggle is ON:

\n
[Shared fragment][User template][Migration template] [Remove] [Create template] [Mapping mode ☑]\n
\n\n

Template Library Panel

\n

Shows published templates for reuse:

\n
Template library\n├─ Dropdown (shows fragment paths)\n└─ [Replace] [Append]\n
\n\n

Paste Page Content

\n

The page editor includes a Paste page content badge next to save/revert buttons. Use this to:

\n\n

The parser automatically unwraps API response shapes, so you can paste output from /api/page-content?path=... directly.

\n

Data Mapping Configuration

\n

Row-Level Mappings

\n

When Mapping Mode is enabled, each row shows configuration based on its type.

\n

Location Row Mapping

\n
Source type: [Extract] [Static] [Metadata]\n☑ Extract from content\n☑ Hide location row\nDefault location: [____________]\n
\n

Extract from content: Automatically extract location from narrative text

\n\n

Hide location row: Make the row invisible

\n\n

Default location: Fallback location if extraction fails

\n\n

Map Row Mapping

\n
☑ Extract GPX from content\nGPX file path: [____________]\nUse location from row: [0]\n
\n

Extract GPX from content: Pull GPX file from source page

\n\n

GPX file path: Static GPX file path (alternative to extraction)

\n\n

Use location from row: Row index to get location coordinates from

\n\n

Metadata Row Mapping

\n
Source type: [Metadata ▼]\nField: [Title | Path | Migration note]\nPrefix: [Automatically migrated from]\nDate format: [yyyy-LL-dd HH:mm]\n
\n

Field options:

\n\n

Customization:

\n\n

Index Row Mapping

\n

Index rows use existing Index row settings:

\n\n

No additional mapping configuration required.

\n

Column-Level Mappings

\n

For rows with columns containing nested rows, column-level mapping configuration appears.

\n

Column Mapping UI

\n
Column 2 Mapping\n─────────────────────────────────────\nSource Type: [Extract | Static | Not mapped]\nContent Type: [Text | Image | Mixed]\n\nNested Rows Configuration\n─────────────────────────────────────\nContent source: [Extract from content ▼]\nPacking behavior: [One row per item ▼]\n☑ Group text with images\n\nDocumentation notes\n[Notes about what this mapping does...]\n
\n

Content Source Options

\n

Not mapped

\n\n

Extract from content

\n\n

Static content

\n\n

Content Type (what to extract)

\n

Remaining images

\n\n

Remaining text

\n\n

All content

\n\n

All images

\n\n

Pattern match

\n\n

Packing Behavior (when extracting)

\n

One row per item

\n\n

All in one row

\n\n

Collect with breaks

\n\n

Additional Options

\n

Group text with images (groupTextWithImage)

\n\n

Hide row if empty (hideIfEmpty)

\n\n

Documentation notes

\n\n

Dynamic Target Rows

\n

When Mapping mode is enabled, you can mark any nested row as a Dynamic target using the badge button in the nested row header (stacked-layers icon).

\n

What it does:

\n\n

How it works:

\n
    \n
  1. Content Source specifies what to extract and how to pack it
    2. \n
  2. Dynamic target button specifies which row structure to use
    3. \n
  3. When migration runs:\n
    4. \n
  4. Other nested rows (without button) are static and added after dynamic rows
    5. \n
\n

Example:

\n
Nested Rows Configuration:\n├─ Content source: remaining-images\n├─ Packing: one-per-item\n│\n├─ Nested Row 1 (Text) ← "Use as dynamic target" (active)\n│  └─ Column: image + caption placeholder\n│\n└─ Nested Row 2 (Shared Fragment) ← Static row\n    └─ Shared fragment reference\n
\n

Result:

\n\n

Column Processing Order

\n

Important: When a row has multiple columns with dynamic collection, order matters:

\n
    \n
  1. Columns with remaining-images processed first\n
    2. \n
  2. Columns with remaining-text processed second\n
    3. \n
\n

This ensures images grab caption text before remaining-text column consumes all text.

\n

Implementation: The system automatically sorts column mappings by priority:

\n
    \n
  1. Pattern-match images (specific content)
    2. \n
  2. Remaining-images with groupTextWithImage: true
    3. \n
  3. Other content types
    4. \n
  4. Remaining-images without text grouping
    5. \n
\n

Content Matchers & Patterns

\n

Text Patterns

\n

Used when Content Type is TEXT:

\n

ALL_TEXT_UNTIL_IMAGE

\n\n

ALL_TEXT_AFTER_HEADING

\n\n

REMAINING_TEXT

\n\n

PARAGRAPH

\n\n

STARTS_WITH_HEADING

\n\n

CUSTOM_REGEX

\n\n

Image Patterns

\n

Used when Content Type is IMAGE:

\n

FIRST_IMAGE

\n\n

REMAINING_IMAGES

\n\n

ALL_IMAGES

\n\n

FILENAME_PATTERN

\n\n

ALT_TEXT_PATTERN

\n\n

Pattern Usage Table

\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
PatternUse CaseMarks as Used?
FIRST_IMAGESingle hero image, featured imageYes
REMAINING_IMAGESGallery consuming all remainingYes
ALL_IMAGESShow all images in carousel, then detailNo
FILENAME_PATTERNSpecific image like route mapYes
ALT_TEXT_PATTERNImages with specific alt textYes
\n

Understanding "Used" vs "Unused" Content

\n

The transformation engine tracks which content has been consumed:

\n

Text Content:

\n\n

Image Content:

\n\n

Best Practices for Content Extraction

\n

1. Extract in Order of Specificity

\n
Step 1: Extract specific content (e.g., route map by filename)\nStep 2: Extract structured content (e.g., intro text until image)\nStep 3: Extract remaining content (e.g., remaining-text, remaining-images)\n
\n

2. Use REMAINING Patterns Last\nAlways use REMAINING_TEXT or REMAINING_IMAGES in final rows to ensure no content is lost.

\n

3. Test Custom Regex Incrementally\nStart simple and add complexity:

\n\n

4. Filename Patterns Are Case-Sensitive

\n\n

5. Extract Images Before Text\nWhen both use remaining-* patterns, ensure images process first to capture captions before text consumes them.

\n

Common Template Patterns

\n

Pattern: Two-Column Layout with Featured Image

\n
Row 1: Full-width intro text\nRow 2: Two columns\n  ├─ Column 1 (8 cols): Featured image\n  └─ Column 2 (4 cols): Remaining content in nested rows\n
\n

Mapping:

\n\n

Pattern: Route Page with Hidden Location

\n
Row 1: Location (hidden)\nRow 2: Map (references Row 1)\nRow 3: Two columns (narrative + sidebar)\n  ├─ Column 1 (8 cols): Main narrative text\n  └─ Column 2 (4 cols): Points of interest images\n
\n

Mapping:

\n\n

Pattern: Photo Gallery with Map

\n
Row 1: Two columns\n  ├─ Column 1 (6 cols): Map showing photo locations\n  └─ Column 2 (6 cols): Dynamic gallery (one row per photo)\n
\n

Mapping:

\n\n

Pattern: Index Page with Action Buttons

\n
Row 1: Intro text\nRow 2: Index row (maps + clustering)\nRow 3: Action buttons for child pages\n
\n

Parent Page:

\n\n

Advanced Topics

\n

Nested Row Mappings Data Model

\n
interface NestedRowMappingConfig {\n  contentSource: 'remaining-images' | 'remaining-text' | 'all-content' | 'all-images' | 'pattern-match';\n  packingBehavior: 'one-per-item' | 'all-in-one' | 'collect-with-breaks';\n  breakOn?: 'image' | 'heading' | 'paragraph';\n  stopOn?: 'image' | 'heading';\n  groupTextWithImage?: boolean;\n  filenamePattern?: string;\n  textPattern?: string;\n  imagePattern?: string;\n}\n
\n

Column-Level Mappings Data Model

\n
interface ColumnMappingConfig {\n  columnIndex: number;\n  sourceType?: 'extract' | 'static' | 'metadata';\n  contentType?: 'text' | 'image' | 'mixed';\n  extractPreset?: string;\n  extractPattern?: string;\n  nestedRowMapping?: NestedRowMappingConfig;\n  notes?: string;\n}\n
\n

Row-Level Mappings Data Model

\n
interface MigrationTemplateMapping {\n  targetRowIndex: number;\n  sourceType?: 'extract' | 'static' | 'metadata';\n\n  // Type-specific mappings\n  location?: MigrationTemplateLocationMapping;\n  map?: MigrationTemplateMapMapping;\n  index?: MigrationTemplateIndexMapping;\n\n  columnMappings?: ColumnMappingConfig[];\n  hideIfEmpty?: boolean;\n  notes?: string;\n}\n
\n

Bootstrap Grid System

\n

Remember the 12-column grid:

\n\n

Columns in a row should sum to 12 for best results.

\n

Image Styling

\n

Images support:

\n\n

Content Exclusions

\n

Before transformation, filter content using site config:

\n\n

Troubleshooting

\n

Template Issues

\n

Template not appearing in library

\n\n

Mapping mode not showing

\n\n

Content Extraction Issues

\n

Content not appearing

\n\n

Wrong image selected

\n\n

Images show placeholder text instead of captions

\n\n

Narrative column includes image caption headings

\n\n

Nested rows not populating

\n\n

Text split incorrectly

\n\n

Custom regex not matching

\n\n

Filename pattern not matching images

\n\n

No nested rows generated

\n\n

All content in one nested row when expecting multiple

\n\n

Location & Map Issues

\n

Location extraction not working

\n\n

Map not centering on location

\n\n

Blank maps showing

\n\n

Example Workflows

\n

Example 1: Migrating Route Pages

\n

Goal: Migrate old route pages with location extraction, map display, and narrative content.

\n

Template Structure:

\n
Row 0: Location (hidden, extracted from content)\nRow 1: Map (500px height, uses location from Row 0)\nRow 2: Two columns\n  ├─ Column 1 (8 cols): Main narrative text\n  └─ Column 2 (4 cols): Points of interest images with captions\nRow 3: Migration note (metadata)\n
\n

Mappings:

\n
    \n

  1. Row 0 (Location):

    \n\n
    2. \n

  2. Row 1 (Map):

    \n\n
    3. \n

  3. Row 2, Column 1:

    \n\n
    4. \n

  4. Row 2, Column 2:

    \n\n
    5. \n

  5. Row 3 (Metadata):

    \n\n
    6. \n
\n

Parent Page Config:

\n\n

Result: All route pages get consistent structure with extracted locations, centered maps, narrative text, and image galleries with preserved captions.

\n

Example 2: Photo Gallery Migration

\n

Goal: Complex gallery with map showing photo locations and dynamic nested rows.

\n

Template Structure:

\n
Row 0: Title and description\nRow 1: Two columns\n  ├─ Column 1 (6 cols):\n  │   ├─ Nested Row 0: Description text\n  │   ├─ Nested Row 1: Location (hidden)\n  │   └─ Nested Row 2: Map (references nested row 1)\n  └─ Column 2 (6 cols): Dynamic gallery rows (one per photo)\n
\n

Mappings:

\n
    \n

  1. Row 1, Column 1:

    \n\n
    2. \n

  2. Row 1, Column 2:

    \n\n
    3. \n
\n

Result: Gallery page with map on left, dynamically generated photo rows on right (one row per image).

\n

Implementation Status

\n

✅ Completed

\n\n

🚧 In Progress

\n\n

📋 Planned

\n\n

File Locations

\n

Frontend

\n

Models:

\n\n

Components:

\n\n

Services:

\n\n

Backend

\n

Migration Engine:

\n\n

Migrations:

\n\n

Migration Service:

\n\n

Routes Template Example

\n

The repository includes docs/routes-template-page-content.json as a reference implementation containing:

\n\n

To use:

\n
    \n
  1. Open /admin/page-content/fragments/templates/routes-template
    2. \n
  2. Click Paste page content
    3. \n
  3. Paste JSON from docs/routes-template-page-content.json
    4. \n
  4. Click Apply pasted content
    5. \n
  5. Save template
    6. \n
\n

This gives you a working baseline to experiment with mappings and configurations.

\n

Related Documentation

\n\n"}