4th November 2025 — Migration to Serenity/JS with Screenplay Pattern
Migrate static site migration engine from Puppeteer to Serenity/JS with Screenplay pattern, providing environment toggle and improved reliability.
Where to Find It
- Admin → Migration: Static site migration functionality
- Environment Variable: USE_SERENITY_FOR_MIGRATION toggles between engines
- Local Development: Uses bundled Chrome/Chromedriver from server directory
Benefits
- Modern Architecture: Actor-based Screenplay pattern for cleaner, more maintainable code
- Environment Flexibility: Toggle between Puppeteer and Serenity/JS via environment variable
- Type-Safe Scraping: browser.execute() provides type-safe data extraction
- Better Debugging: Comprehensive debug logging in all catch blocks
- Docker Compatible: Environment-specific paths for Chrome/Chromedriver
- Improved Error Handling: Selector error collection and logging for failed CSS selectors
Technical Details
Backend Changes:
- serenity-utils.ts: Browser launching with environment-specific Chrome/Chromedriver paths
- serenity-migration-engine.ts: Actor-based navigation with browser.execute() for scraping
- serenity-migration-utils.ts: Actor creation and Cast implementation
- migration-routes.ts: Route handler toggles based on USE_SERENITY_FOR_MIGRATION
- migration-types.ts: Centralized type definitions (BaseHrefResult, HtmlFetchResult, etc.)
- environment-model.ts: Add USE_SERENITY_FOR_MIGRATION to Environment enum
Implementation:
- WebdriverIO configuration with Chrome and Chromedriver paths for local/Docker
- Screenplay Tasks for navigation (NavigateAndWait) and content scraping
- Navigate via Actor, scrape via type-safe browser.execute()
- All selectors passed dynamically from config (no hard-coded selectors)
- Remove unused PageElements and Questions (not suitable for dynamic selectors)
Code Quality:
- Export all inline types to migration-types.ts
- Convert all Array types to T[] syntax (migration and backup modules)
- Add comprehensive debug logging to all catch blocks
- Remove empty catch blocks throughout
Dev Environment:
- Update test-setenv.sh and app-start-scripts for local Chrome/Chromedriver
- Support Mac (Chrome for Testing.app) and Docker (Dockerfile paths)
- Bundled Chrome/Chromedriver in server directory