25th October 2025 — Optimized HTML/Markdown Paste Pipeline #67

build 321 — #67 — commit 07644ee

Dramatically improved paste experience with client-side HTML detection, server-side processing enhancements, URL auto-fetch, and admin clipboard tools.

Where to Find It

• Markdown Editor: Paste HTML content, toolbar "Convert to Rows" button
• Admin → Migration Settings: Paste/Copy buttons for site configuration JSON
• Fragment Index: Recursive detection of nested row usage

Benefits

• Instant Plain Pastes: Client filters trivial HTML/browser wrappers before server call
• Cleaner Rows: Enhanced preprocessing removes artifacts, produces better Markdown
• Smart URL Handling: Paste view-source:// or http(s):// URLs to auto-fetch and convert pages
• Base URL Suggestions: Dialog prompts with migration config URLs for relative link resolution
• Nested Content Support: Choose nested/sibling placement with smart column-count defaults
• Admin Efficiency: Copy/paste site config JSON directly in migration settings
• Fragment Awareness: Recursive detection includes nested rows and normalized paths
• Reduced Duplication: DRY Puppeteer utilities shared across migration functions

Technical Details

Client-Side Detection (PasteDetectionService):

• Filter browser wrappers, trivial tags, images, links before server call
• Defer to server only after user confirms in prompt
• Toolbar "Convert to Rows" button with preview count (24 tests)
• Detect view-source:// and http(s):// URL pastes
• Auto-fetch HTML via /api/migration/html-from-url
• Auto-fill derived base URL from or URL path

Server Processing:

• Preprocess: entity decode, strip Chrome code-table wrappers
• Unwrap anchors in src/href attributes
• Normalize sources (data-src/srcset fallbacks)
• Resolve relative URLs against base URL
• Post-processing: remove tags/comments, normalize  
• Strip stray CSS/property declarations and font-family suffixes
• Collapse whitespace and blockquote artifacts
• Turndown integration via createTurndownService
• Final markdown cleanup for cleaner rows

New APIs:

• POST /api/migration/html-from-url: loads page with Puppeteer, extracts HTML, derives base URL
• POST /api/migration/html-paste-preview: returns markdown + rows for preview
• POST /api/migration/markdown-paste-preview: returns markdown + rows for insertion
• Safe fallback to fetch() when Puppeteer unavailable

UX Enhancements:

• Base URL prompt with suggestions from migration config
• Directory extraction via ensureTrailingSlash
• Paste dialog: "Split into rows" vs "Convert without splitting"
• Row placement radios: nested vs sibling (smart default by column count)
• Split images and adjacent short captions into rows
• Keep Content Together (Nested Rows) configuration toggle
• Unnest action for moving content to previous/next outer row

Admin Tools:

• Site configuration Paste/Copy buttons (JSON textarea)
• ClipboardService for reliable copy operation
• Immediate JSON apply with validation and Cancel
• Paste auto-expands the site config section
• Duplicate site action retained

Puppeteer DRY:

• New puppeteer-utils.ts centralizes launchBrowser() and deriveBaseUrl()
• Refactored migrate-static-site.ts, migrate-static-site-engine.ts, migration-routes.ts
• Shared launcher eliminates code duplication

Fragments:

• Fragment index includes nested row usages
• Normalize fragment path comparison (leading slash handling)
• Recursive usage detection
• Redirect /fragments to /admin/fragment-index

Testing:

• Comprehensive unit tests: turndown-service-factory.spec.ts
  • Tests: entity decoding, wrapper removal, anchor unwrapping
  • Relative image resolution, nbsp normalization, blockquote cleanup, CSS stripping
• Related route specs and e2e previews
• Frontend builds succeed, server tests pass
• No database schema changes