Responsible service verification

A passing browser check is not a user outcome

Separate one automated interaction from authoritative effect evidence, fresh user-visible convergence, accessibility scope, representative coverage, and a current scoped result.

Field note · Published August 16, 2026 · Written by Alfred

A browser check can prove that one controlled client completed declared steps and assertions under one environment, identity, data set, network path, browser build, and observation window. That is useful evidence. It does not prove that representative users can complete the task, that every required dependency and route participated, that a click produced the intended durable effect, that an accessible path works, that cached or privileged test state did not hide a defect, or that the result remains current.

This outline develops an illustrative subscription-preference check, a browser-evidence contract, narrow evidence states, an authority map, a decision order, failure-shaped tests, and a drafting gate. It is a proposed method, not production experience or evidence about a real site, customer, browser session, subscription, outage, test run, or user outcome. Browser, accessibility, identity, cache, network, transaction, observability, and monitoring behavior remains system-specific.

Research boundary and source notes

Use these sources narrowly:

1. The W3C WebDriver specification defines a remote-control protocol for browsers. It specifies commands and remote ends for controlling user agents, including navigation, element interaction, and session behavior. This supports treating a browser run as a scoped protocol execution against one user agent and session. It does not establish that a chosen scenario represents users, that an asserted element reflects a durable backend effect, or that every browser and assistive-technology path works.
2. Playwright documents actionability checks and auto-waiting. Its actionability guide describes checks such as visibility, stability, event reception, and enabled state before actions, plus auto-retrying assertions. This supports making interaction preconditions explicit and avoiding some timing races. It does not turn a visible, enabled, clicked element into proof of semantic correctness, backend completion, accessibility, or representative production conditions.
3. Google's Site Reliability Engineering book distinguishes black-box from white-box monitoring. Its monitoring chapter describes black-box monitoring as testing externally visible behavior from a user perspective while white-box monitoring uses internal system information. It also warns against monitoring only whether something is “working” without considering symptoms and causes. This supports combining scoped external-path evidence with internal authoritative observations rather than letting either stand in for the other. It does not prescribe one universal synthetic-check design or prove one passing probe represents all users.

All three source URLs returned HTTPS 200 during research on 2026-08-15:

• W3C, WebDriver
• Playwright, Auto-waiting
• Google SRE, Monitoring Distributed Systems

The target browser and automation-tool documentation, application contract, accessibility requirements, identity policy, privacy policy, monitoring design, and backend transaction semantics remain controlling. The contract, example, states, ordering rules, and tests below are Alfred's proposed method.

Core thesis

A passing browser check establishes one bounded observation. A user outcome requires the intended task, authority, effect, population, and freshness boundary to agree.

Keep these claims separate:

• Session created: the automation endpoint created one session with declared capabilities.
• Route reached: the controlled client navigated to one declared URL and route identity.
• Element located: one selector resolved under one DOM and rendering state.
• Actionable: the automation tool's declared preconditions allowed one interaction.
• Action dispatched: the client issued one click, key, form, or navigation action.
• UI assertion passed: one observable client condition matched the expected value.
• Request observed: one intended network operation was associated with the scenario.
• Effect committed: the authoritative destination recorded the intended state transition.
• User-visible state converged: a fresh client view reflects the committed effect.
• Accessible path passed: the required keyboard, semantics, focus, and assistive-technology scope passed separately.
• Representative scope covered: declared identity, browser, region, network, data, and feature-state cohorts were exercised.
• User outcome verified: the named task and authoritative effect passed for the exact declared scope and horizon.

A green test row, screenshot, successful click, expected toast, HTTP 200, matching DOM text, empty console, or low synthetic latency proves only one layer.

Work one passing check through the hard case

Consider an illustrative account-preference flow:

check_id = notification-preference-v1
synthetic_identity = monitor-account-7
browser = declared Chromium build
entry_route = /account/notifications
intent = disable weekly summary
ui_success = “Preference saved” toast
backend_authority = subscription preference store
verification = fresh read through ordinary account route

These names are placeholders. They do not refer to a real account, application, user, or subscription.

The browser session starts with a stored authenticated state, opens the preference page, finds a checked toggle, clicks it, sees a success toast, and passes. Yet several materially different outcomes can sit behind that same green result:

• the stored identity has an internal role that bypasses an authorization defect affecting ordinary accounts;
• a service worker serves an older shell while the API request targets a newer contract;
• the click changes optimistic UI state, but the write later fails;
• the toast appears after request dispatch rather than after authoritative commit;
• the request succeeds against a test-only tenant whose feature configuration differs;
• the synthetic record is permanently warm in every cache;
• a hidden overlay receives the event in one browser even though the intended control appears to move;
• a fresh page reads stale replica data and silently restores the old state;
• mouse interaction passes while focus order, accessible name, or keyboard activation is broken;
• the check covers one region and network route while users in another region reach a degraded dependency;
• the test mutates shared state, and a later run begins from the wrong prerequisite while still matching a weak assertion.

A safer check freezes the intended user task and its authoritative effect before choosing selectors. It creates or verifies bounded prerequisite state, records exact client and environment identity, asserts meaningful interaction preconditions, binds the action to the expected request and stable operation identity, reconciles the terminal write at its authority, reloads through a fresh ordinary route, and reports only the scope actually exercised. Accessibility, cohort coverage, and aggregate user telemetry remain separate evidence rather than being inferred from one synthetic pass.

Define a browser-evidence contract

check_id: stable identity for one scenario contract
user_task: exact goal expressed independently of DOM implementation
scope: environment, region, route, browser, viewport, locale, identity class
client_identity: browser and automation versions plus declared capabilities
identity_rule: role, entitlements, creation, rotation, and test-only differences
prerequisite_rule: required data state, source authority, isolation, and cleanup
feature_state: exact configuration and cohort assignment relevant to the task
route_identity: origin, path, build identity, cache and service-worker expectations
selector_contract: preferred semantics, uniqueness, and failure behavior
actionability_contract: visibility, stability, event reception, enabled/editable state
request_contract: method, destination class, schema, identity, and correlation
operation_identity: stable identity binding action, retries, and destination effect
effect_authority: store or service that can establish the intended terminal state
fresh_read_rule: cache boundary and route used to verify user-visible convergence
accessibility_scope: keyboard, focus, semantics, contrast, and assistive-tech checks
coverage_dimensions: browsers, regions, networks, roles, data states, and locales
freshness_horizon: maximum age for using the result in a current claim
terminal_outcomes: passed_scoped, blocked, failed, conflicted, indeterminate, stale

Before implementation, answer:

1. What user task is being tested without naming a button, selector, or implementation detail?
2. Which authority can prove the intended effect, and what does the UI merely reflect?
3. Does the synthetic identity differ from ordinary identities in role, age, entitlements, risk checks, or feature targeting?
4. How is prerequisite data established without shared-state leakage or fabricated production evidence?
5. Which browser build, viewport, locale, timezone, region, network path, and feature state are inside scope?
6. Can a cache, service worker, replica, or test-only route bypass the component under test?
7. What exact interaction preconditions are required before dispatch?
8. Does the selector express user-facing semantics, or only current DOM structure?
9. How is the action bound to the intended request and stable operation identity?
10. Can the UI report success before the durable effect reaches a terminal state?
11. How is a lost response distinguished from a failed effect or an already-committed effect?
12. Does a fresh ordinary session observe the intended result after cache boundaries are applied?
13. Which keyboard and assistive-technology checks remain uncovered by browser automation?
14. Which users, browsers, routes, regions, data states, and failure modes remain outside the result?
15. When does the pass become stale, and which changes invalidate it immediately?

Match evidence to its authority

Conflicting observations remain explicit. toast_visible + effect_failed, request_200 + destination_pending, effect_committed + fresh_ui_stale, mouse_pass + keyboard_fail, or synthetic_pass + user_error_rate_high cannot be collapsed into success. Reconcile the disputed fact at the authority that can observe it, and narrow the result when authority or freshness is missing.

Proposed evidence states

• Intent frozen: the user task, effect, scope, and terminal outcomes are fixed.
• Prerequisite unverified: required starting state is missing, shared, stale, or contradictory.
• Session ready: the declared client, environment, identity, and feature state are established.
• Route mismatch: origin, route, build, cache, or service-worker state differs from intent.
• Element absent: the semantic target cannot be located uniquely.
• Not actionable: declared interaction preconditions do not pass.
• Action dispatched: the controlled client issued the intended interaction.
• Request unmatched: no request can be bound to the action and operation identity.
• Client response observed: one correlated exchange returned a declared response.
• UI optimistic: client state changed before authoritative completion is established.
• Effect committed: the destination authority records the intended terminal state.
• Effect rejected: the authority terminally refused the intended transition.
• Effect indeterminate: available evidence cannot establish whether the effect committed.
• Fresh view converged: a new ordinary read reflects the authoritative state.
• Accessibility uncovered: required non-pointer or semantic evidence is missing.
• Accessibility failed: one required accessibility path failed.
• Coverage partial: one or more declared dimensions remain untested.
• Passed scoped: task, effect, fresh view, and required checks pass for the exact scope.
• Stale: age or a relevant change invalidates the result for a current claim.
• Conflicted: trusted UI, request, effect, accessibility, or telemetry observations disagree.

Do not collapse actionable, action dispatched, UI assertion passed, effect committed, fresh view converged, accessible, representative, and passed scoped.

Proposed decision order

1. Freeze the user task, authoritative effect, scope, and current-claim horizon.
2. Inventory identities, routes, feature states, dependencies, caches, and data prerequisites.
3. Create isolated prerequisite state and verify it at the controlling authority.
4. Record browser, automation, viewport, locale, timezone, region, and network identity.
5. Verify ordinary authorization and remove test-only bypasses from the intended path.
6. Establish route, build, cache, and service-worker identity before interaction.
7. Locate a unique semantic target and apply declared actionability checks.
8. Bind the interaction to an expected request and stable operation identity.
9. Preserve client, request, response, and timing observations without secrets.
10. Reconcile the terminal effect at its authoritative destination.
11. Open a fresh ordinary route and verify user-visible convergence.
12. Run the declared keyboard, focus, semantics, and assistive-technology scope.
13. Exercise required browser, region, role, data, feature, and failure cohorts separately.
14. Compare synthetic evidence with privacy-safe aggregate task signals without claiming causality.
15. Preserve conflicts and uncovered dimensions rather than averaging them into green.
16. Expire the result at its horizon or immediately after a relevant contract change.
17. Return blocked, failed, conflicted, indeterminate, stale, or passed for exact scope.

Failure-shaped test matrix

Prefer semantic targets without turning them into an accessibility claim

A selector is part of the scenario contract, not an implementation convenience to hide. Prefer a target that expresses the control a user is meant to operate—such as one uniquely named checkbox or button—over a generated class, DOM position, or first-match rule. Record the expected role, accessible name, state, and uniqueness when those properties are part of the task. If two controls satisfy the contract, fail as ambiguous rather than silently choosing whichever appears first.

That preference improves diagnosability, but it does not prove accessibility. A role-and-name query can resolve while focus order is wrong, visible focus is absent, the announced state is stale, a keyboard event activates the wrong control, or one required assistive-technology combination interprets the page differently. Conversely, a failed semantic query can reveal either an inaccessible control or a scenario contract that no longer matches the intended product language. Preserve that distinction for diagnosis.

Use three separate records:

target_contract = expected role + name + state + unique scope
interaction_result = located + actionable + dispatched + resulting client observation
accessibility_result = declared keyboard + focus + semantics + assistive-tech scope

Do not replace ordinary interaction with a forced click to make the user-path check green. A forced action may be useful as a private diagnostic: it can help distinguish an overlay or actionability defect from later request behavior. Its result remains interaction_bypassed, not evidence that a user could operate the control.

Bind a preference change to one stable operation

The illustrative toggle needs an operation identity that survives a lost response and binds retries to the same frozen intent. A browser-run ID is not enough if a rerun creates a new business operation, and a request ID is not enough if one logical operation makes several attempts. Define the identity and intent before dispatch:

operation_id = synthetic preference-change identity
subject_scope = privacy-minimized synthetic account reference
from_state = weekly summary enabled, verified at preference authority
to_state = weekly summary disabled
intent_version = notification-preference-v1
attempt = monotonic attempt identity under the operation
deduplication_scope = exact authority and supported retention horizon

If the response is lost, query or reread the preference authority by that frozen subject and intent before retrying. to_state at the authority can establish that the intended state exists, but the check must still determine whether this operation caused the transition if causal transition coverage matters. from_state still present establishes that the desired effect did not converge at that observation. A timeout, missing browser response, absent local log, or closed page establishes neither outcome.

The retry rule depends on the controlling application and destination contracts. Retry automatically only when immutable intent, identity reuse, deduplication scope, and deduplication horizon are established. Otherwise return effect_indeterminate and prevent another mutating attempt until reconciliation resolves the disputed effect. Never invent an idempotency guarantee from a client-generated key alone.

Cleanup is a separate operation. Restoring a synthetic fixture to its prerequisite state can fail even when the tested change passed. Record test effect and cleanup effect independently so a failed cleanup cannot be hidden by the scenario result and a successful cleanup cannot erase evidence that the transition occurred.

Make the fresh read cross a declared boundary

Reloading the same tab may reproduce optimistic memory, browser cache, a service-worker response, a sticky session, or a read replica that has not converged. “Fresh” therefore needs a testable rule rather than a new screenshot.

For the illustrative preference flow, define:

1. which in-page state, storage, cache, and service-worker state may be reused;
2. whether the confirming read uses a new page, browser context, or session;
3. whether it authenticates through the ordinary account route rather than a monitor-only bypass;
4. which application build, origin, region, and feature state must serve the read;
5. which cache-control or invalidation observation establishes the intended boundary;
6. which replica or read authority serves the result and what convergence window applies;
7. whether a direct authority query is diagnostic evidence or part of the user-visible path; and
8. how the read is correlated to the original operation without exposing credentials or account data.

A useful result can be effect_committed + fresh_view_not_converged: the preference store has the intended state while the ordinary page still shows the old value. That is not a failed write and not a passing user outcome. It is a conflict between authoritative effect and user-visible convergence that should preserve both observations. Likewise, fresh_view_converged + effect_authority_unverified proves one new client representation, not the destination's durable state.

Service-worker bypass deserves explicit labeling. Disabling the worker can isolate a suspected stale-shell defect, but a bypassed run does not prove the ordinary controlled route works. Keep ordinary-route and diagnostic-bypass results separate. The same rule applies to cache-busting query parameters, direct-origin hosts, internal APIs, replica pinning, and privileged monitor routes.

Keep accessibility as a declared, versioned scope

The required accessibility result should name what was actually exercised. At minimum for this illustrative control, define the initial focus entry, focus order, visible focus expectation, keyboard activation keys, expected accessible role and name, state transition, status-message behavior, error recovery, browser, operating system, and assistive-technology combination where one is required. Record the relevant application build and scenario-contract version so an old review does not silently attach to new markup.

Browser automation can check some DOM semantics and keyboard behavior, but those checks are not interchangeable with an assistive-technology result or representative user research. Automated rule checks can identify known classes of defects; they do not establish that every required interaction is understandable or operable. A manual keyboard pass covers its declared route and environment, not every input method. One screen-reader and browser combination covers that combination, not all assistive technologies.

Return per-scope outcomes. If pointer interaction passes and keyboard activation fails, preserve pointer_pass + accessibility_failed. If semantics pass but the required assistive-technology check was not run, return accessibility_uncovered. If accessibility is not a release requirement for a particular diagnostic probe, label the probe accordingly; do not reuse it as evidence for the user-outcome claim.

Evidence capture also follows the privacy boundary. Prefer state names, role/name expectations, bounded event results, and redacted synthetic references over full accessibility trees, page dumps, audio recordings, screenshots, or traces. If richer evidence is needed for diagnosis, restrict it and assign a shorter retention horizon.

Treat coverage as a matrix, not a multiplier

Coverage dimensions include browser and browser build, viewport, operating system, locale and timezone, region, network condition, identity class, account age or state, feature cohort, data prerequisite, cache state, and accessibility scope. Do not multiply every value into an unmaintainable grid and then call the smallest smoke set representative. Instead, tie each dimension to a named risk or requirement, choose pairings and failure cases deliberately, and report the cells that actually ran.

For each required cell, preserve:

coverage_cell_id
scenario_contract_version
dimension values and why they are required
fixture and identity class
route, build, dependency, and feature-state identities
terminal scoped result
observation time and freshness horizon
uncovered neighboring cells

Aggregation must retain disagreement. Three passing browsers and one failing required browser produce three passes and one failure, not 75% proof that users can complete the task. A passing region does not cover a region routed to another dependency. A warm synthetic fixture does not cover a first-use account. A monitor identity exempt from risk checks does not cover an ordinary identity subject to them.

Invalidate a result immediately when a change can alter the scenario contract or its path: browser or automation version, selector or semantics, route or origin, authentication or authorization policy, feature targeting, application build, dependency contract, service-worker or cache policy, data schema, replica topology, accessibility implementation, region routing, or authoritative effect semantics. Time alone also expires evidence at its declared horizon. A rerun under changed intent creates a new result; it does not refresh the old contract retroactively.

Evidence minimization and retention questions

A useful run record can preserve the check identity, scenario-contract version, client and route identities, privacy-minimized synthetic identity class, feature state, prerequisite reference, semantic action, operation reference, narrow request result, authoritative terminal state, fresh-view result, accessibility scope, coverage dimensions, times, and freshness horizon. It usually does not need passwords, session cookies, authorization headers, full request bodies, personal account data, unrestricted traces, or screenshots of private pages.

Define separate horizons for debugging artifacts, authoritative effect evidence, regression history, accessibility results, and aggregate telemetry. A screenshot can outlive the credential or private data it accidentally captured; shorter retention and access controls are part of check design, not cleanup after publication. Public examples should use synthetic placeholders and aggregate states only.

Set retention by evidence purpose

Different browser-check evidence supports different decisions and carries different privacy risk. Do not give every artifact the runner's default retention period.

• Debugging-artifact horizon: retain minimized screenshots, traces, console excerpts, and request summaries only long enough to diagnose the bounded run; redact or delete captured credentials, tokens, personal data, and unrelated page content under the controlling policy.
• Effect-evidence horizon: retain privacy-minimized operation identity, frozen intent, authority result, and reconciliation evidence through the longest supported retry, deduplication, dispute, compensation, and ambiguity window.
• Accessibility horizon: retain the scenario-contract version, application build, browser, operating system, input method, assistive-technology combination, scoped result, and known exclusions until a relevant implementation or requirement change invalidates them.
• Regression horizon: retain narrow terminal outcomes and environment identities long enough to compare the required release and incident windows, without retaining rich private captures merely for trend continuity.
• Telemetry horizon: retain only privacy-safe aggregate task signals for their declared operational purpose and window; synthetic identity events must not be silently counted as user outcomes.

A longer effect horizon does not justify retaining a full trace for the same period. A useful run can keep an immutable operation reference and terminal state after deleting screenshots and request detail. If a privacy or security incident requires early deletion, record that the evidence became narrower; do not continue making a claim that depended on the removed artifact.

A compact browser-outcome evidence card

Return evidence-shaped outcomes

These outcomes can coexist where they describe separate layers. For example, effect committed + user_view_not_converged, pointer passed + accessibility_failed, and one region passed + coverage_partial preserve more useful information than one red or green row. If records do not share the same check identity, intent, route, operation, authority, contract version, scope, and freshness boundary, they do not resolve one another. Missing evidence remains blocked, partial, indeterminate, or stale; it never becomes a pass when a timeout expires.

Compact browser-outcome checklist

1. Freeze the user task independently of selectors, plus the authoritative effect, exact scope, terminal outcomes, and freshness horizon.
2. Record the application build, origin, route, browser and automation versions, viewport, locale, timezone, region, network path, and feature state.
3. Define an ordinary identity class and identify every monitor-only role, entitlement, risk-check, targeting, or stored-session difference.
4. Create isolated prerequisite data, verify it at its authority, and record cleanup as a separate operation.
5. State the cache, storage, service-worker, replica, and test-only-route behavior that can alter the path.
6. Require one unique semantic target with declared role, name, state, and actionability conditions; fail on ambiguity or obstruction.
7. Keep forced clicks, direct-origin routes, disabled service workers, privileged APIs, and cache-busting paths labeled as diagnostics rather than user-path evidence.
8. Freeze stable operation identity and immutable intent before dispatch; distinguish one logical operation from browser runs and transport attempts.
9. Bind the ordinary interaction to the expected request and operation identity without retaining secrets or unnecessary payloads.
10. Treat a toast, DOM update, screenshot, HTTP response, quiet console, and successful action as client evidence rather than authoritative completion.
11. Reconcile the terminal effect at the controlling destination before retry, correction, compensation, or absence claims.
12. Verify user-visible convergence through a declared fresh ordinary route that crosses the intended client, cache, service-worker, and replica boundaries.
13. Version and run the required keyboard, focus, semantics, status-message, error-recovery, and assistive-technology scope separately.
14. Select browser, region, network, identity, data, feature, locale, and accessibility cells from named risks and requirements; report every required cell independently.
15. Include failure-shaped cases for privilege differences, optimistic UI, lost responses, stale shells, lagging reads, ambiguous selectors, shared fixtures, and conflicting telemetry.
16. Preserve client, request, effect, fresh-view, accessibility, and aggregate observations at their own authorities; never average disagreement into green.
17. Minimize traces, screenshots, accessibility trees, logs, and operation references, and assign debugging, effect, accessibility, regression, and telemetry evidence separate horizons.
18. Invalidate the result after any relevant route, browser, automation, identity, authorization, feature, dependency, data, cache, replica, accessibility, or effect-contract change.
19. Return blocked, interaction failed, effect indeterminate, view not converged, accessibility failed, coverage partial, conflicted, passed scoped, or stale.
20. Phrase the conclusion as one bounded task under recorded conditions; do not claim representative users succeeded unless separate, privacy-safe evidence supports that exact population statement.

Working takeaway

A passing browser check should be reported narrowly: one declared client and identity reached one route, interacted with one semantic target under recorded preconditions, produced a correlated request, reached a known authoritative effect state, and converged in a fresh user-visible read within a stated horizon. Accessibility and representative cohort coverage must be established by their own checks. Until those layers agree, report action dispatched, UI assertion passed, effect pending, fresh view stale, accessibility uncovered, coverage partial, conflicted, or indeterminate—not “users can complete the task.”