18-Apr-2026 — WDIO → Playwright: migrating Serenity/JS to the official image, and why real-time step streaming broke

This is a write-up of the WebdriverIO → Playwright migration completed on the NGX Ramblers project under issue #114. Two specific things are worth sharing: why the move was an obvious win operationally, and why the real-time Serenity/JS domain-event publisher — which had worked cleanly under WDIO since commit 48aca026 (May 2025) — went silent under @serenity-js/playwright-test without any obvious error.

The fix is pragmatic and works. The interesting bit is what was discovered about how @serenity-js/playwright-test surfaces domain events, and a small suggestion that a first-class real-time option would close the gap we ran into.

The feature: walks upload with live progress in the admin UI

Our Ramblers walks upload flow runs a Serenity/JS scenario that logs into Ramblers Walks Manager, selects walks by date and title, deletes existing ones, uploads a CSV, publishes, and so on. The whole thing takes one-to-two minutes and happens off to the side of the admin UI. Users want to see progress as it happens, not a blank screen followed by a summary.

The original design, as shown in the animated example above was introduced in 48aca026, was clean and idiomatic Serenity:

That architecture is exactly what Jan suggested in Serenity/JS discussion #2797 — treating domain events as a first-class stream from the test runner to any interested listener.

Under WebdriverIO the pipeline is synchronous and live end to end:

flowchart LR
    UI@{ icon: "ngx:user", label: "Admin UI", pos: "b", h: 48 }

    subgraph subprocess["Serenity/JS subprocess"]
        ACTOR["Actor performs Click / Enter / Wait"]
        STAGE["Serenity Stage"]
        CREW["DomainEventPublisher"]
        WSCLIENT["WebSocket client"]
    end

    subgraph backend["NGX Ramblers backend"]
        NODE@{ icon: "logos:nodejs-icon", label: "Node spawn", pos: "b", h: 48 }
        WS@{ icon: "ngx:ramblers", label: "WebSocket server", pos: "b", h: 48 }
    end

    ACTOR --> STAGE
    STAGE --> CREW
    CREW --> WSCLIENT
    WSCLIENT --> WS
    NODE --> subprocess
    WS --> UI

    style subprocess fill:#E8F5EE,stroke:#9BC8AB,stroke-width:2px,rx:12,ry:12,color:#404143
    style backend fill:#E8F5EE,stroke:#9BC8AB,stroke-width:2px,rx:12,ry:12,color:#404143

Under WebdriverIO, every activity an actor performs flows synchronously through the Stage. InteractionFinished is announced the moment that interaction ends; the crew member receives it, serialises it, and hands it to the socket. The UI sees each step appear within a few hundred milliseconds of the action actually happening on the page.

This worked cleanly for about a year.

Why migrate to Playwright

Two mounting costs made WDIO feel like a tax:

The screenplay code was already framework-agnostic — tasks and questions are written against @serenity-js/web (PageElement, Click, Enter, Navigate). Only the ability layer and runner were WDIO-specific. The migration shape was therefore:

Moderate, not a full rewrite. And that's how it played out — until the first end-to-end test.

The silent failure

The migrated scenario worked. The walks uploaded. The report generated. But the audit view in the UI stayed empty until the browser closed at the end of the run, then every step appeared at once in a one-second burst.

Worse, all the step audits ended up with identical timestamps (same wall-clock millisecond) because they were being written to MongoDB concurrently with dateTimeNowAsValue() as their auditTime.

Under @serenity-js/playwright-test the same DomainEventPublisher saw events only via a buffered end-of-test flush:

flowchart TB
    subgraph subprocess["Serenity/JS subprocess"]
        ACTOR["Actor performs Click / Enter / Wait"]
        PWORKER["Playwright test worker"]
        REPORTER["SerenityReporterForPlaywrightTest"]
        BUFFER["PlaywrightEventBuffer"]
    end

    subgraph batch["End-of-test flush"]
        FLUSH["onTestEnd → flush all events"]
        BATCH["Events announced in one burst"]
        STAGE["Serenity Stage"]
        CREW["DomainEventPublisher"]
    end

    BACKEND@{ icon: "ngx:ramblers", label: "Backend /progress", pos: "b", h: 48 }

    ACTOR --> PWORKER
    PWORKER --> REPORTER
    REPORTER --> BUFFER
    BUFFER --> FLUSH
    FLUSH --> BATCH
    BATCH --> STAGE
    STAGE --> CREW
    CREW --> BACKEND

    style subprocess fill:#E8F5EE,stroke:#9BC8AB,stroke-width:2px,rx:12,ry:12,color:#404143
    style batch fill:#E8F5EE,stroke:#9BC8AB,stroke-width:2px,rx:12,ry:12,color:#404143

The DomainEventPublisher looked healthy from the outside: it was attached to the Stage, it was receiving events, it was serialising them, it was POSTing them. The POSTs just all happened in one clump at the end of the run.

The diagnosis

Reading @serenity-js/playwright-test's reporter source (SerenityReporterForPlaywrightTest.js) made it click. The real flow is:

onTestEnd(test, result) {
    // ... retry / crash handling omitted ...
    this.eventBuffer.appendSceneEvents(test, result);
    if (pendingAfterAllHooks === 0) {
        this.eventBuffer.appendSceneFinishedEvent(test, result);
        const events = this.eventBuffer.flush(test, result);
        this.serenity.announce(...events);
    } else {
        this.eventBuffer.deferAppendingSceneFinishedEvent(test, result);
    }
}

PlaywrightEventBuffer accumulates an entire test's worth of domain events (TaskStarts, TaskFinished, InteractionStarts, InteractionFinished, ActivityRelatedArtifactGenerated, and so on) and then announces them to the Stage in a single batch at onTestEnd. Inside that batch, the events carry their original timestamps — so the order and durations are accurate once they arrive — but they don't arrive until the test is over.

This is architecturally sensible: Playwright's reporter API gives you TestStep info (step.title, step.duration, step.error) and Serenity/JS builds its domain-event stream from those. But the consequence for a real-time listener is the same: there is no point in the pipeline where a StageCrewMember can see InteractionFinished as the interaction finishes. The Stage only gets notified after the test has exited.

The corollary: under @serenity-js/playwright-test, StageCrewMember.notifyOf is not a real-time hook. It is the end-of-test flush point.

The fix: a Playwright-native reporter

Because Playwright's own reporter API does fire onStepBegin/onStepEnd synchronously as steps execute, the fix is to hook Playwright directly and bypass Serenity's buffer for this one concern. RealtimeStepReporter below was implemented here, bootstrapped in the playwright config here:

export default class RealtimeStepReporter implements Reporter {

  onStepEnd(_test: TestCase, _result: TestResult, step: TestStep): void {
    if (!this.configured) return;
    if (step.category !== "test.step") return;

    const timestamp = new Date(step.startTime.getTime() + (step.duration || 0)).toISOString();
    const payload = {
      eventData: {
        timestamp,
        details: { name: step.title },
        outcome: step.error
          ? { code: FAILED_OUTCOME_CODE, error: { message: step.error.message, stack: step.error.stack } }
          : { code: SUCCESS_OUTCOME_CODE }
      },
      finished: false
    };
    this.pendingPosts.push(this.post(payload));
  }

  async onEnd(): Promise<void> {
    if (this.pendingPosts.length > 0) await Promise.allSettled(this.pendingPosts);
  }
}

Registered alongside the Serenity reporter in playwright.config.ts:

reporter: [
  ["@serenity-js/playwright-test", { crew: [/* photographer, BDD, artifact archiver */] }],
  ["./lib/serenity-js/reporters/realtime-step-reporter.ts"],
  ["list"]
]

Filtering by category === "test.step" keeps fixture/hook steps out of the audit view; Serenity's Activity.performAs wraps each interaction in a test.step(), so the step.title is the activity description users already recognise from the old WDIO output.

With a Playwright-native reporter added alongside the Serenity one, per-step events stream live while Serenity still handles the end-of-run artefacts:

flowchart LR
    subgraph subprocess["Serenity/JS subprocess"]
        ACTOR["Actor performs Click / Enter / Wait"]
        PWORKER["Playwright test worker"]
        RTR["RealtimeStepReporter"]
        SRP["SerenityReporterForPlaywrightTest"]
    end

    subgraph backend["NGX Ramblers backend"]
        PROGRESS@{ icon: "ngx:ramblers", label: "Progress endpoint", pos: "b", h: 48 }
        WS@{ icon: "ngx:ramblers", label: "WebSocket push", pos: "b", h: 48 }
    end

    UI@{ icon: "ngx:user", label: "Admin UI", pos: "b", h: 48 }
    S3@{ icon: "ngx:aws", label: "Serenity report on S3", pos: "b", h: 48 }

    ACTOR --> PWORKER
    PWORKER --> RTR
    PWORKER --> SRP
    RTR --> PROGRESS
    PROGRESS --> WS
    WS --> UI
    SRP --> S3

    style subprocess fill:#E8F5EE,stroke:#9BC8AB,stroke-width:2px,rx:12,ry:12,color:#404143
    style backend fill:#E8F5EE,stroke:#9BC8AB,stroke-width:2px,rx:12,ry:12,color:#404143

Each step now arrives at the backend within a few hundred milliseconds of the action completing. The Serenity reporter is still attached to the same config, doing its usual end-of-run work — the BDD artefact, the Serenity-BDD report, the photographer, the artifact archiver. We haven't replaced Serenity; we've supplemented it with one extra Reporter for the one concern Serenity can't serve under Playwright.

Other backend bugs this uncovered

Moving events to a faster pace also shook loose two backend issues that the old end-of-run batch had been hiding:

Both were latent before the migration; WDIO's per-step cadence wasn't concurrent enough to trip them.

The architecture change that came with it

Once Playwright was the runtime, the natural follow-on was to get every browser session off the website process entirely. The website image had historically carried Playwright/WDIO dependencies because Flickr scraping, legacy-site migration, and URL-fetching for venue discovery all ran in-process.

That's now gone:

The website image is Node 24 only — no browser binaries, no Playwright, no OpenJDK. The integration worker image is the ghcr.io/serenity-js/playwright base with the Serenity-BDD JAR pre-fetched.

flowchart TB
    UI@{ icon: "ngx:user", label: "Admin UI", pos: "b", h: 48 }

    subgraph website["Website image"]
        BACKEND@{ icon: "logos:nodejs-icon", label: "Express + Mongoose", pos: "b", h: 48 }
        HANDLERS["Migration, HTML fetch, Flickr, venue scraper"]
    end

    subgraph worker["Integration worker image"]
        JOBS["Async jobs endpoint"]
        BROWSER["Sync browser endpoint"]
        SUBPROCESS["Serenity/JS subprocess"]
    end

    MONGO@{ icon: "ngx:mongodb", label: "MongoDB Atlas", pos: "b", h: 48 }
    S3@{ icon: "ngx:aws", label: "S3 reports + content", pos: "b", h: 48 }

    UI --> BACKEND
    BACKEND --> MONGO
    BACKEND --> HANDLERS
    HANDLERS --> JOBS
    HANDLERS --> BROWSER
    JOBS --> SUBPROCESS
    SUBPROCESS --> BACKEND
    JOBS --> S3

    style website fill:#E8F5EE,stroke:#9BC8AB,stroke-width:2px,rx:12,ry:12,color:#404143
    style worker fill:#E8F5EE,stroke:#9BC8AB,stroke-width:2px,rx:12,ry:12,color:#404143

Details

End state: one browser image (the worker), one real-time step delivery path (Playwright's onStepEnd → signed POST → UI WebSocket), one audit persistence path (sendAudit serialised per session). All committed under 58aa7fd1 on top of b0a6d384 on issue #114.

Observations — and a suggestion

The migration itself was straightforward; the discovery in the middle was the interesting part. Knowing in advance that @serenity-js/playwright-test batches domain events at onTestEnd would have saved an afternoon of puzzling. The DomainEventPublisher kept working well enough to look unbroken: it was being wired up correctly, it was receiving events, it was POSTing them — just all at the wrong moment.

Two things that would have saved me time, and might be worth thinking about for Serenity/JS:

  1. Document the buffering behaviour somewhere in the @serenity-js/playwright-test docs. I looked at the API overview and the handbook page and couldn't find it explicitly called out in either. The fact that StageCrewMember.notifyOf fires in a batch at onTestEnd rather than live is a meaningful architectural constraint for anyone building a live event listener. The discussion that inspired our original approach (#2797) is WDIO-context; a note about how that pattern translates under Playwright would close the loop.

  2. Consider an eager-announce mode on the Playwright-test reporter. Something like:

    ["@serenity-js/playwright-test", {
  crew: [/* ... */],
  announceEventsEagerly: true   // or: emitPerStep: true
}]

    The reporter already has the data it needs in onStepEnd (Serenity converts these into InteractionFinished / TaskFinished domain events post-hoc). Announcing them to the Stage as each step ends, rather than collecting them into PlaywrightEventBuffer and flushing at onTestEnd, would let a StageCrewMember like the original DomainEventPublisher continue to serve the "live audit trail" use case without us needing to drop down to a Playwright-native Reporter.

    There may well be reasons this would be harder than it looks — retry handling, after-all hook ordering, scene-finished deferral. But from a consumer's perspective it would restore the elegance of "domain events from Serenity, one mechanism, framework-agnostic".

If any of this is wrong — if there is a way to get real-time domain events out of @serenity-js/playwright-test that I missed — I'd love to know. It's entirely possible I've read past the right hook.

Closing thought

The official Serenity/JS Playwright image is genuinely a great piece of operational kit. Dropping Chrome, ChromeDriver, and a pile of WDIO runners from our worker build made the dev loop faster, the image smaller, and the failure modes fewer. The real-time streaming gap is a small thing against that, and easy enough to paper over with a Playwright-native reporter. But it's the sort of paper-over that would be nicer not to need.

Thanks to Jan and the Serenity/JS team for the image and the discussion thread that made the original design obvious in 2025. Happy to compare notes.