Snapshots and refs

A snapshot is the page as an accessibility tree in agent-browser's text format. Elements you can act on get a ref such as @e3, which works as a selector until the next snapshot.

val snapshot = page.snapshot(interactive = true)
println(snapshot.tree)
// @e1 [heading] "Example Domain" [level=1]
// @e2 [input type="email"] placeholder="Email"
// @e3 [checkbox] "Remember me" [checked]
// @e4 [button] "Later" [disabled]
// @e5 [button] "Submit"
// @e6 [link] "Learn more"
// @e7 [Iframe] "payment-frame"
//   @e8 [input] placeholder="Card number"
//   @e9 [button] "Pay"

page.locator("@e2").fill("test@example.com")
page.locator("@e5").click()

agent-browser: snapshot, with the same flags (-i, -u, -c, -d, -s, -C) and the same line format. From an LLM tool, use page.runCommand("snapshot -i") (see Commands).

Snapshot parametersCopy Anchor Link

page.snapshot(...) takes these named parameters:

• selector is passed to document.querySelector, so it must be CSS. It does not accept refs or KMPwright engines such as text=. An invalid selector throws Invalid snapshot selector: <selector>; one that matches nothing throws Snapshot selector did not match: <selector>.
• maxDepth counts levels of the underlying tree, not printed indentation. Transparent wrappers (see below) count as levels even though they do not print.
• snapshot() needs KmpwrightConfig.allowDomExtraction (the default) and is checked against ActionPolicy as ActionType.ReadContent.

The tree formatCopy Anchor Link

One element per line. Children are indented two spaces under their parent.

@e1 [heading] "Example Domain" [level=1]
[paragraph]
  [StaticText] "Some intro text."
[form]
  @e2 [input type="email"] placeholder="Email"
  @e3 [checkbox] "Remember me" [checked]
  @e4 [button] "Later" [disabled]
  @e5 [button] "Submit"
@e6 [link] "Learn more"
@e7 [Iframe] "payment-frame"
  @e8 [input] placeholder="Card number"
  @e9 [button] "Pay"

Each line is built from these parts, in this order; parts that do not apply are left out:

Input types printed as [input type="…"]: text, email, password, search, tel, url, number, date, datetime-local, month, week, time. Other inputs print their role ([checkbox], [radio], [button], [slider] and so on).

Escaping and truncationCopy Anchor Link

Names, values, placeholders and URLs come from the page, so they are printed as JSON string literals: quotes and backslashes are escaped, whitespace (including newlines) collapses to single spaces, and text longer than 200 characters is cut and ends with … inside the quotes. Roles and states must consist of letters, digits, _ and -; anything else prints as [generic] or is dropped, and refs must look like @e<number>.

The result is that a page cannot add lines, forge refs or fake states in the tree an LLM reads. A button named Pay\n@e99 [button] "Steal" prints as one line: [button] "Pay @e99 [button] \"Steal\"".

What is left outCopy Anchor Link

• The root of the page (RootWebArea), unnamed generic wrappers with at most one child, and empty text print nothing; their children print in their place at the same indentation.
• Adjacent text nodes merge into one [StaticText] line, and a text child that repeats its parent's name is dropped (a heading does not print its text twice).
• With interactive = true, every line without a ref is left out.
• With compact = true, structural lines that end up with no ref, no value and no children are left out.
• Hidden elements are skipped unless includeHidden = true.

IframesCopy Anchor Link

An <iframe> gets a ref and prints as [Iframe] with its title (or name) attribute as the name. The content of a same-origin iframe is inlined one level deep, indented under it, and its refs work like any other. Cross-origin iframe content is not included; use page.frameLocator for it.

Which elements get refsCopy Anchor Link

That is why snapshot -i lists headings as well as controls: they give the model landmarks to read and scope by.

SnapshotCopy Anchor Link

RefEntry describes the element a ref named when the snapshot was taken:

SnapshotNode is one node of nodes:

val snapshot = page.snapshot()
val submit = snapshot.refs.values.first { it.role == "button" && it.name == "Submit" }
page.locator(submit.ref).click()

Using refsCopy Anchor Link

A ref is a selector. Every API that takes a selector accepts it, in any of these forms:

page.locator("@e3").click()
page.locator("e3").click()
page.locator("ref=e3").click()      // also "ref=@e3"
page.click("@e3")                   // selector conveniences
expect(page.locator("@e5")).toBeEnabled()
page.runCommand("click @e3")        // runCommand targets

agent-browser: click @e3 — same. Playwright has no refs; page.locator("@e3") is a KMPwright addition.

A ref matches exactly one element, so it never causes a strict-mode violation.

Ref lifetimeCopy Anchor Link

How refs behave:

• Stable per element. The ref registry lives in KMPwright's page runtime and maps each element to its ref. The same element keeps the same ref in every snapshot of the same document, even when its text or position changes. New elements get new, higher numbers; numbers are not reused within a document.
• The latest snapshot wins. Each snapshot() replaces the set of resolvable refs with the refs it contains. A ref from an earlier snapshot keeps working only if the element appears again. A snapshot scoped with selector registers only the refs inside that subtree, and screenshot(annotate = true) takes a full snapshot of its own.
• Removed elements fail. A ref resolves only while its element is attached to the document. If the page removes it, or a framework replaces it with a new node (for example a list re-rendered with new keys), the ref stops matching. Mutating an element in place keeps its ref.
• Navigation resets everything. A new document (goto, a link click, reload, setContent) has a new runtime: no ref resolves until you snapshot again, and numbering starts over at @e1.
• Refs cannot be read or remapped by the page. The registry is held in a sealed runtime, not in page-visible globals, so a hostile page cannot point @e3 at a different element.

When a ref no longer resolves:

The rule for agents: re-snapshot after any action that changes the page, and lower TimeoutConfig.action so stale-ref errors come back quickly (see Build an agent).

DiffingCopy Anchor Link

Compare two snapshots line by line to see what an action changed:

val before = page.snapshot(interactive = true)
page.locator("@e5").click()
val after = page.snapshot(interactive = true)

val diff = page.diffSnapshot(before, after)
if (diff.changed) println(diff.unifiedDiff)
//  @e1 [heading] "Example Domain" [level=1]
// -@e5 [button] "Submit"
// +@e5 [button] "Submit" [disabled]
// +@e10 [heading] "Thanks!" [level=2]

Named baselinesCopy Anchor Link

page.diffSnapshot(name) takes a new full snapshot (default snapshot() parameters), diffs it against the one saved under name, and saves the new one under name:

page.diffSnapshot("cart")            // first call: records the baseline, changed = false
page.locator("@e7").click()
val diff = page.diffSnapshot("cart") // changes since the previous "cart" snapshot

• Every page.snapshot(...) call, with any options, also saves its tree under the reserved name "__baseline__". page.diffSnapshot("__baseline__") therefore diffs a fresh full snapshot against the most recent snapshot of any kind.
• Baselines are kept in memory for the life of the page, across navigations.
• diffSnapshot(name) takes a snapshot, so it replaces the current refs like any other snapshot.

agent-browser: diff snapshot.

Diffing URLsCopy Anchor Link

val diff = page.diffUrls("https://example.com/v1", "https://example.com/v2")

diffUrls fetches both URLs over HTTP (the same client as page.read, without the browser's cookies), reduces each HTML document to its text lines, and diffs those. It does not load the pages in the browser and does not produce accessibility trees. timeoutMs defaults to TimeoutConfig.default. It requires allowDomExtraction; KmpwrightConfig.allowedOriginRules, setAllowedDomains and the ActionPolicy (checked as Navigate) apply to both URLs, as they do for read. HttpRead is Supported on every backend.

agent-browser: diff url.

Annotated screenshotsCopy Anchor Link

val shot = page.screenshot(annotate = true)
shot.bytes                 // the image, with a numbered label on each ref
shot.annotations.forEach { println("${it.ref} ${it.role} ${it.name} ${it.box}") }

With annotate = true, KMPwright takes a full snapshot, draws a red label with the number N over each element that has ref @eN and a non-zero size, captures the screenshot, and removes the labels again. Labels are drawn into the page, so they appear in the image on every backend.

ScreenshotResult.annotations lists the labelled elements in snapshot order:

annotate applies to page screenshots; combine it with fullPage or clip and the other screenshot() parameters as usual. From runCommand, screenshot --annotate also prints a legend (see Commands).

agent-browser: screenshot --annotate.

Backend differencesCopy Anchor Link

Both backends print the same line format, assign refs with the same rules, and inline same-origin iframes one level deep. Expect the exact trees to differ: the same page can show [paragraph] and [StaticText] lines on the JVM that are absent on a WebView, and a native checkbox that is checked without aria-checked shows [checked] only on the JVM.

To check which engine a page uses at runtime, read page.capabilities().backend (BackendKind.JvmCdp, AndroidWebView or IOSWebView).