Alibaba Open-Sources Page Agent: An Agent That Lives Inside the Webpage, Reading Text Instead of Screenshots to Operate the UI
MIT-licensed, model-agnostic — plug in any OpenAI-compatible text model and go. For now it only operates a single page view.
- An Alibaba team has open-sourced Page Agent, an agent library that runs as plain JavaScript inside the webpage itself, reading the page's textual structure (the DOM) directly to understand and operate the UI — no screenshots involved
- The core technique is DOM dehydration: it compresses a page's thousands of nodes into a lean text map called FlatDomTree that keeps only the interactive elements, so an ordinary text model can pinpoint exactly which element to act on
- It's MIT-licensed and model-agnostic, connecting through any OpenAI-compatible endpoint — it only needs text, not a multimodal model. The code inherits its DOM handling and prompt logic from browser-use
- Because it runs inside the webpage, it automatically inherits the user's current login state, cookies, and permissions — no separate backend and no headless browser required
- The limits are just as clear: safety rules live in the prompt rather than as hard constraints, the core library can only operate a single page, and cross-tab work needs an extra Chrome extension
Alibaba open-sourced a browser agent that breaks the usual mold
An Alibaba team recently open-sourced Page Agent, an agent library that runs as plain JavaScript inside the webpage, understanding and operating the UI by reading the text-based DOM.
It lives inside the page like a real user, reading the page's text structure to click buttons and fill in forms — no headless browser, no screenshots, and no need for a model that can even see images.
Where the old approach's cost goes
To see what Page Agent actually saves, first look at how much baggage external tools carry just to operate one webpage. They never enter the page — they can only stand outside and direct it through a pane of glass.
This externally-driven approach still works well for cross-site scraping and end-to-end testing. What Page Agent is trying to fix is a different kind of hassle: when the webpage is your own product and you can edit its code, why go through all that trouble at all.
Compressing an entire webpage into a text list
A modern webpage can have thousands of nodes, and handing the raw HTML straight to a model is slow and expensive. Page Agent's approach is to "dehydrate" the page first, keeping only the handful of things that can actually be operated on.
Upon receiving an instruction, the agent scans the entire DOM (Document Object Model — the element tree the browser parses a webpage into), finds every interactive element — buttons, links, input fields — and tags each with a sequential index plus a role and a text label. All the redundant decorative markup gets stripped away, and the whole page is compressed into a lean text map called FlatDomTree. What the model reads is this list, not pixels.
It's like stripping all the body text out of a thick book and keeping only the chapter titles and page numbers from the table of contents. The model doesn't need to chew through the whole book — one glance at this table of contents tells it which page to flip to and which button to press.
How different what the model sees is, before and after dehydration
The original demo page lays this loop out directly: a "Dehydrated DOM" panel shows the list the model is reading, while an "Action trace" panel next to it updates step by step as the instruction executes — you can watch it click through the sequence.
What happens once the instruction comes in
From a single sentence of natural language to an actual click on the page, there's a fixed closed loop in between. The dirty work is handled by a component called PageController.
PageController exposes exactly these actions, operating on elements by index:
await this.pageController.updateTree()
await this.pageController.clickElement(index)
await this.pageController.inputText(index, text)
await this.pageController.scroll({ down: true, numPages: 1 })
The whole monorepo splits responsibilities across three small packages:
@page-agent/coreheadless agent core logicpage-agentthe full entry-point class with a UI panel@page-agent/page-controllerhandles DOM extraction and element indexing, with an optional SimulatorMask for visual feedbackThree guardrails in the developer's hands
Against other tools, who should use it
This comparison table is about scenario, not speed. The four approaches run in different places and read the page in different ways — each has its own turf.
| Approach | Runs where | How it reads the page | Integration cost | Best for |
|---|---|---|---|---|
| Page Agent | Inside the webpage (client-side JS) | Dehydrated text DOM | One script tag or npm | An operational copilot inside your own product |
| Selenium / Playwright / Puppeteer | External process | Reads the DOM via a driver (WebDriver/CDP) | Driver plus a runtime or service | Scripted end-to-end testing |
| browser-use | External process | DOM plus optional vision | Python plus a browser | An autonomous, multi-site agent |
| WebMCP | Server-side tool | Structured function calls | Requires the standard to see wide adoption | Agent-native tool calling |
It takes a different route: the webpage wraps its own functionality into structured "tool" functions and exposes them directly for an agent to call, relying on a standardized interface. Page Agent reads DOM text, WebMCP relies on a standard protocol — one works without touching the webpage's code, the other has to wait for its interface standard to gain broad acceptance.
The conclusion comes down to scope of use: Page Agent fits inside a product you control and can edit; if you need to scrape someone else's site, or work against a locked-down environment, external driving still wins.
What you can actually do with it
Because it lives right inside your application, it can actually finish an operation for the user, not just tell them beside it how to click. The original piece gives four concrete examples.
The lowest-cost path is a single script tag
If you just want to get a feel for it, one script tag loads Page Agent bundled with a free test model, ready to try right on the page.
<script src="https://cdn.jsdelivr.net/npm/page-agent@1.10.0/dist/iife/page-agent.demo.js" crossorigin="true"></script>
For production use, install the package and swap in your own endpoint:
import { PageAgent } from 'page-agent'
const agent = new PageAgent({
model: 'qwen3.5-plus',
baseURL: 'https://dashscope.aliyuncs.com/compatible-mode/v1',
apiKey: 'YOUR_API_KEY',
language: 'en-US',
})
await agent.execute('Click the login button')
The model and baseURL accept any OpenAI-compatible provider — switching models is basically just swapping the base URL and key.
new PageAgent gets bundled straight into your front-end code — in production, requests need to go through your own backend as a proxy; never expose the key on the client. The agent also supports popping up a confirmation before executing each critical action.What it can't do
This "live inside the webpage" approach comes with its own inherent trade-offs, and the official docs are upfront about the limits. These points need to be on the table before you use it.
Safety rules written into the prompt are only a suggestion
Rules like "never auto-submit a payment form" are placed in the system prompt. They're persuasive guidance, not a hard guarantee. For sensitive or destructive operations, server-side validation still has to stay in place — instructions in the prompt can't be your only line of defense.
The core library only handles a single page
The core library targets interaction within a single view; on its own it can't move between tabs or windows. For cross-page automation, you need the optional Chrome extension, which requires its own installation and authorization. There's also a Beta-stage MCP server that lets external agents like Claude Desktop or Copilot drive it in reverse.
Expand: what each of the three runtime locations solves
The core library runs inside the page and handles single-page operation; the Chrome extension adds cross-tab capability, at the cost of an install and permissions; the Beta MCP server turns Page Agent into a tool that external agents can call, connecting it back to external agents like Claude Desktop and Copilot. Each of the three layers covers a different scope, and the further out you go, the higher the cost of setup and authorization.
Back to the opening line: Page Agent and mainstream tools take two different roads — one embeds itself inside the webpage to read text, the other stands outside and remotely operates via screenshots and protocols. It extends where browser automation can actually land, from "an external script controlling someone else's webpage" to "a natural-language operating layer built right into the product."
The agent lives inside the webpage as plain JavaScript. It reads the live DOM as text and acts as the real user. No headless browser, no screenshots, no multi-modal model. , MarkTechPost, 2026-07-02
Getting AI to operate webpages: from "standing outside, remote-controlling by screenshot" to "living inside the webpage, reading text, clicking on its own"
Alibaba open-sourced Page Agent, turning the agent that operates a UI into a piece of code embedded right in the webpage. This page walks through, with diagrams, how it works, where it saves effort, and where it still can't be fully trusted.
↓ Read the whole thing in one pass · with one animated comparison diagram
First, what this actually does. Browser automation means having a program operate a webpage in place of a person — auto-clicking buttons, filling forms, running tests. This category of tool has existed for a while; the problem is the posture it works in.
✘ But they can't be dropped in lightweight as a built-in operating assistant inside your own product
Because they all live outside the browser, running as a separate program that remotely controls the page via screenshots or a debug protocol — you need to install a driver, and it often also needs a multimodal model that can "see" images to guess where the elements are. Heavy and expensive.
This time, Alibaba turned the agent into a piece of JavaScript embedded right in the webpage. It lives inside the page like a real user, reading the page's text structure to click and fill things in — so it can be dropped into your own product as an operating assistant. The most immediate change is what the model sees on every click:
Reading text is more accurate than looking at an image, and cheaper too — no need to keep a pricier vision-capable model around. So how does that indexed text list get squeezed out of one whole, messy webpage?
The signature move is called DOM dehydration (the DOM is the element tree the browser breaks a webpage into; dehydrating means pulling the handful of genuinely clickable things out of a page's thousands of parts, tagging each with an index, and compressing it into a lean text list). Same scenario, clearest when placed side by side with the old approach:
What people feel most viscerally about "how much this saves" is how much stuff they have to prep by hand. Take "adding an AI assistant that can operate the UI on the user's behalf to a product" and compare directly:
These are the project's publicly stated facts — license, version, integration method — not a performance benchmark; "reads only text, so it's more cost-efficient on inference" is a design inference, and the official docs give no third-party benchmark for it. Going to production still means swapping in your own model and moving the key to the backend — don't ship the demo key baked into the front end.
and fill forms for the user
automation tool drops in lightweight…
Selenium?
just wiring one up needs,
- × A separate process
- × A driver / protocol layer
- × Often a pricey vision model
- × A backend proxy on top
to start
a piece of code embedded in the webpage
No pricey model?
clicks and fills things in itself
the model can read all that?
tag each with an index
<button class>…
thousands of messy nodes
[3] button "Log in"
[4] button "Submit expense"
and it's in my own product
the core library only handles a single page, cross-tab needs a separate extension.
the moment money or sensitive operations are involved, backend validation still can't be skipped.
