Sightline: giving in-app agents page context without a screenshot
retained text
I've been working on Sightline, an MIT-licensed JS package for assistants embedded in web apps. It reads DOM labels, roles and control states locally and returns a compact text outline. For example, an agent can receive a named dialog, an empty Name field and a disabled Create button. That gives it useful context for helping someone through a form, though it still needs the product's rules to explain why the button is disabled. It can watch for page changes and highlight controls by reference. There's no model call in extraction, so it can replace a separate screenshot-description call. It doesn't replace vision for charts, images or layout problems. The default output cap is 6,000 characters. Typed field values are excluded, but other page text still needs appropriate handling. I'm interested in where this breaks down for agents in real apps. Repo and bookmarklet demo in the comments.
document fetched 2026-10-28 00:19 · canonical https://reddit.com/r/AI_Agents/comments/1xh3myt/sightline_giving_inapp_agents_page_context