This is a write-up of the WebdriverIO → Playwright migration completed on the NGX Ramblers project under issue #114. Two specific things are worth sharing: why the move was an obvious win operationally, and why the real-time Serenity/JS domain-event publisher — which had worked cleanly under WDIO since commit 48aca026 (May 2025) — went silent under @serenity-js/playwright-test without any obvious error.
The fix is pragmatic and works. The interesting bit is what was discovered about how @serenity-js/playwright-test surfaces domain events, and a small suggestion that a first-class real-time option would close the gap we ran into.
Our Ramblers walks upload flow runs a Serenity/JS scenario that logs into Ramblers Walks Manager, selects walks by date and title, deletes existing ones, uploads a CSV, publishes, and so on. The whole thing takes one-to-two minutes and happens off to the side of the admin UI. Users want to see progress as it happens, not a blank screen followed by a summary.
The original design, as shown in the animated example above was introduced in 48aca026, was clean and idiomatic Serenity:
StageCrewMember called DomainEventPublishernotifyOf(event), filter for InteractionFinished / TaskFinished / TestSuiteFinished / SceneParametersDetectedThat architecture is exactly what Jan suggested in Serenity/JS discussion #2797 — treating domain events as a first-class stream from the test runner to any interested listener.
Under WebdriverIO the pipeline is synchronous and live end to end:
flowchart LR
UI@{ icon: "ngx:user", label: "Admin UI", pos: "b", h: 48 }
subgraph subprocess["Serenity/JS subprocess"]
ACTOR["Actor performs Click / Enter / Wait"]
STAGE["Serenity Stage"]
CREW["DomainEventPublisher"]
WSCLIENT["WebSocket client"]
end
subgraph backend["NGX Ramblers backend"]
NODE@{ icon: "logos:nodejs-icon", label: "Node spawn", pos: "b", h: 48 }
WS@{ icon: "ngx:ramblers", label: "WebSocket server", pos: "b", h: 48 }
end
ACTOR --> STAGE
STAGE --> CREW
CREW --> WSCLIENT
WSCLIENT --> WS
NODE --> subprocess
WS --> UI
style subprocess fill:#E8F5EE,stroke:#9BC8AB,stroke-width:2px,rx:12,ry:12,color:#404143
style backend fill:#E8F5EE,stroke:#9BC8AB,stroke-width:2px,rx:12,ry:12,color:#404143
Under WebdriverIO, every activity an actor performs flows synchronously through the Stage. InteractionFinished is announced the moment that interaction ends; the crew member receives it, serialises it, and hands it to the socket. The UI sees each step appear within a few hundred milliseconds of the action actually happening on the page.
This worked cleanly for about a year.
Two mounting costs made WDIO feel like a tax:
@puppeteer/browsers into the worker image at build time, matching versions by hand, and shipping CHROME_BIN / CHROMEDRIVER_PATH through the runtime env. Every Chrome bump meant image rebuilds and config.ghcr.io/serenity-js/playwright image with Node 24, OpenJDK, and Playwright browsers pre-installed on Ubuntu 24.04. If we used it, none of the browser plumbing was ours to maintain.The screenplay code was already framework-agnostic — tasks and questions are written against @serenity-js/web (PageElement, Click, Enter, Navigate). Only the ability layer and runner were WDIO-specific. The migration shape was therefore:
@serenity-js/webdriverio for @serenity-js/playwright + @serenity-js/playwright-testwdio.conf.ts with playwright.config.tsdescribe/it with the @serenity-js/playwright-test actorCalled fixtureModerate, not a full rewrite. And that's how it played out — until the first end-to-end test.
The migrated scenario worked. The walks uploaded. The report generated. But the audit view in the UI stayed empty until the browser closed at the end of the run, then every step appeared at once in a one-second burst.
Worse, all the step audits ended up with identical timestamps (same wall-clock millisecond) because they were being written to MongoDB concurrently with dateTimeNowAsValue() as their auditTime.
Under @serenity-js/playwright-test the same DomainEventPublisher saw events only via a buffered end-of-test flush:
flowchart TB
subgraph subprocess["Serenity/JS subprocess"]
ACTOR["Actor performs Click / Enter / Wait"]
PWORKER["Playwright test worker"]
REPORTER["SerenityReporterForPlaywrightTest"]
BUFFER["PlaywrightEventBuffer"]
end
subgraph batch["End-of-test flush"]
FLUSH["onTestEnd → flush all events"]
BATCH["Events announced in one burst"]
STAGE["Serenity Stage"]
CREW["DomainEventPublisher"]
end
BACKEND@{ icon: "ngx:ramblers", label: "Backend /progress", pos: "b", h: 48 }
ACTOR --> PWORKER
PWORKER --> REPORTER
REPORTER --> BUFFER
BUFFER --> FLUSH
FLUSH --> BATCH
BATCH --> STAGE
STAGE --> CREW
CREW --> BACKEND
style subprocess fill:#E8F5EE,stroke:#9BC8AB,stroke-width:2px,rx:12,ry:12,color:#404143
style batch fill:#E8F5EE,stroke:#9BC8AB,stroke-width:2px,rx:12,ry:12,color:#404143
The DomainEventPublisher looked healthy from the outside: it was attached to the Stage, it was receiving events, it was serialising them, it was POSTing them. The POSTs just all happened in one clump at the end of the run.
Reading @serenity-js/playwright-test's reporter source (SerenityReporterForPlaywrightTest.js) made it click. The real flow is:
onTestEnd(test, result) {
// ... retry / crash handling omitted ...
this.eventBuffer.appendSceneEvents(test, result);
if (pendingAfterAllHooks === 0) {
this.eventBuffer.appendSceneFinishedEvent(test, result);
const events = this.eventBuffer.flush(test, result);
this.serenity.announce(...events);
} else {
this.eventBuffer.deferAppendingSceneFinishedEvent(test, result);
}
}
PlaywrightEventBuffer accumulates an entire test's worth of domain events (TaskStarts, TaskFinished, InteractionStarts, InteractionFinished, ActivityRelatedArtifactGenerated, and so on) and then announces them to the Stage in a single batch at onTestEnd. Inside that batch, the events carry their original timestamps — so the order and durations are accurate once they arrive — but they don't arrive until the test is over.
This is architecturally sensible: Playwright's reporter API gives you TestStep info (step.title, step.duration, step.error) and Serenity/JS builds its domain-event stream from those. But the consequence for a real-time listener is the same: there is no point in the pipeline where a StageCrewMember can see InteractionFinished as the interaction finishes. The Stage only gets notified after the test has exited.
The corollary: under @serenity-js/playwright-test, StageCrewMember.notifyOf is not a real-time hook. It is the end-of-test flush point.
Because Playwright's own reporter API does fire onStepBegin/onStepEnd synchronously as steps execute, the fix is to hook Playwright directly and bypass Serenity's buffer for this one concern. RealtimeStepReporter below was implemented here, bootstrapped in the playwright config here:
export default class RealtimeStepReporter implements Reporter {
onStepEnd(_test: TestCase, _result: TestResult, step: TestStep): void {
if (!this.configured) return;
if (step.category !== "test.step") return;
const timestamp = new Date(step.startTime.getTime() + (step.duration || 0)).toISOString();
const payload = {
eventData: {
timestamp,
details: { name: step.title },
outcome: step.error
? { code: FAILED_OUTCOME_CODE, error: { message: step.error.message, stack: step.error.stack } }
: { code: SUCCESS_OUTCOME_CODE }
},
finished: false
};
this.pendingPosts.push(this.post(payload));
}
async onEnd(): Promise<void> {
if (this.pendingPosts.length > 0) await Promise.allSettled(this.pendingPosts);
}
}
Registered alongside the Serenity reporter in playwright.config.ts:
reporter: [
["@serenity-js/playwright-test", { crew: [/* photographer, BDD, artifact archiver */] }],
["./lib/serenity-js/reporters/realtime-step-reporter.ts"],
["list"]
]
Filtering by category === "test.step" keeps fixture/hook steps out of the audit view; Serenity's Activity.performAs wraps each interaction in a test.step(), so the step.title is the activity description users already recognise from the old WDIO output.
With a Playwright-native reporter added alongside the Serenity one, per-step events stream live while Serenity still handles the end-of-run artefacts:
flowchart LR
subgraph subprocess["Serenity/JS subprocess"]
ACTOR["Actor performs Click / Enter / Wait"]
PWORKER["Playwright test worker"]
RTR["RealtimeStepReporter"]
SRP["SerenityReporterForPlaywrightTest"]
end
subgraph backend["NGX Ramblers backend"]
PROGRESS@{ icon: "ngx:ramblers", label: "Progress endpoint", pos: "b", h: 48 }
WS@{ icon: "ngx:ramblers", label: "WebSocket push", pos: "b", h: 48 }
end
UI@{ icon: "ngx:user", label: "Admin UI", pos: "b", h: 48 }
S3@{ icon: "ngx:aws", label: "Serenity report on S3", pos: "b", h: 48 }
ACTOR --> PWORKER
PWORKER --> RTR
PWORKER --> SRP
RTR --> PROGRESS
PROGRESS --> WS
WS --> UI
SRP --> S3
style subprocess fill:#E8F5EE,stroke:#9BC8AB,stroke-width:2px,rx:12,ry:12,color:#404143
style backend fill:#E8F5EE,stroke:#9BC8AB,stroke-width:2px,rx:12,ry:12,color:#404143
Each step now arrives at the backend within a few hundred milliseconds of the action completing. The Serenity reporter is still attached to the same config, doing its usual end-of-run work — the BDD artefact, the Serenity-BDD report, the photographer, the artifact archiver. We haven't replaced Serenity; we've supplemented it with one extra Reporter for the one concern Serenity can't serve under Playwright.
Moving events to a faster pace also shook loose two backend issues that the old end-of-run batch had been hiding:
record counter was read-then-written outside any mutex, so concurrent sendAudit callbacks would all read record = N, each compute N + 1, and write duplicates. Fixed by serialising sendAudit per-session via a promise chain.updateRamblersUploadSession was { ...current, ...update } into a new object and putting that in the Map, so captured session refs never saw record increments. Fixed by Object.assign-ing in place.Both were latent before the migration; WDIO's per-step cadence wasn't concurrent enough to trip them.
Once Playwright was the runtime, the natural follow-on was to get every browser session off the website process entirely. The website image had historically carried Playwright/WDIO dependencies because Flickr scraping, legacy-site migration, and URL-fetching for venue discovery all ran in-process.
That's now gone:
The website image is Node 24 only — no browser binaries, no Playwright, no OpenJDK. The integration worker image is the ghcr.io/serenity-js/playwright base with the Serenity-BDD JAR pre-fetched.
flowchart TB
UI@{ icon: "ngx:user", label: "Admin UI", pos: "b", h: 48 }
subgraph website["Website image"]
BACKEND@{ icon: "logos:nodejs-icon", label: "Express + Mongoose", pos: "b", h: 48 }
HANDLERS["Migration, HTML fetch, Flickr, venue scraper"]
end
subgraph worker["Integration worker image"]
JOBS["Async jobs endpoint"]
BROWSER["Sync browser endpoint"]
SUBPROCESS["Serenity/JS subprocess"]
end
MONGO@{ icon: "ngx:mongodb", label: "MongoDB Atlas", pos: "b", h: 48 }
S3@{ icon: "ngx:aws", label: "S3 reports + content", pos: "b", h: 48 }
UI --> BACKEND
BACKEND --> MONGO
BACKEND --> HANDLERS
HANDLERS --> JOBS
HANDLERS --> BROWSER
JOBS --> SUBPROCESS
SUBPROCESS --> BACKEND
JOBS --> S3
style website fill:#E8F5EE,stroke:#9BC8AB,stroke-width:2px,rx:12,ry:12,color:#404143
style worker fill:#E8F5EE,stroke:#9BC8AB,stroke-width:2px,rx:12,ry:12,color:#404143
ngx-ramblers-integration-worker.@puppeteer/browsers, or any WDIO packages. Its Dockerfile stage is now just node:24.14.0 + mongosh, gh, flyctl. OpenJDK 21 and serenity-bdd-update came off too — the JDK-backed report generation happens only on the worker.DomainEventPublisher, process-test-step-event.ts, the worker's own WebSocket server, and a TEST_STEP_REPORTER event type in websocket.model.ts) was deleted outright once RealtimeStepReporter became the single real-time path. No duplicate delivery mechanisms.End state: one browser image (the worker), one real-time step delivery path (Playwright's onStepEnd → signed POST → UI WebSocket), one audit persistence path (sendAudit serialised per session). All committed under 58aa7fd1 on top of b0a6d384 on issue #114.
The migration itself was straightforward; the discovery in the middle was the interesting part. Knowing in advance that @serenity-js/playwright-test batches domain events at onTestEnd would have saved an afternoon of puzzling. The DomainEventPublisher kept working well enough to look unbroken: it was being wired up correctly, it was receiving events, it was POSTing them — just all at the wrong moment.
Two things that would have saved me time, and might be worth thinking about for Serenity/JS:
Document the buffering behaviour somewhere in the @serenity-js/playwright-test docs. I looked at the API overview and the handbook page and couldn't find it explicitly called out in either. The fact that StageCrewMember.notifyOf fires in a batch at onTestEnd rather than live is a meaningful architectural constraint for anyone building a live event listener. The discussion that inspired our original approach (#2797) is WDIO-context; a note about how that pattern translates under Playwright would close the loop.
Consider an eager-announce mode on the Playwright-test reporter. Something like:
["@serenity-js/playwright-test", {
crew: [/* ... */],
announceEventsEagerly: true // or: emitPerStep: true
}]
The reporter already has the data it needs in onStepEnd (Serenity converts these into InteractionFinished / TaskFinished domain events post-hoc). Announcing them to the Stage as each step ends, rather than collecting them into PlaywrightEventBuffer and flushing at onTestEnd, would let a StageCrewMember like the original DomainEventPublisher continue to serve the "live audit trail" use case without us needing to drop down to a Playwright-native Reporter.
There may well be reasons this would be harder than it looks — retry handling, after-all hook ordering, scene-finished deferral. But from a consumer's perspective it would restore the elegance of "domain events from Serenity, one mechanism, framework-agnostic".
If any of this is wrong — if there is a way to get real-time domain events out of @serenity-js/playwright-test that I missed — I'd love to know. It's entirely possible I've read past the right hook.
The official Serenity/JS Playwright image is genuinely a great piece of operational kit. Dropping Chrome, ChromeDriver, and a pile of WDIO runners from our worker build made the dev loop faster, the image smaller, and the failure modes fewer. The real-time streaming gap is a small thing against that, and easy enough to paper over with a Playwright-native reporter. But it's the sort of paper-over that would be nicer not to need.
Thanks to Jan and the Serenity/JS team for the image and the discussion thread that made the original design obvious in 2025. Happy to compare notes.