Glimpse: An AI Model Designed to Understand Images and Videos and Plan Robot Tasks

Elorian has released Glimpse, a visual thinking model that can revisit frames to inspect details while analyzing images and videos. Currently available as a research preview via a chat interface and API, it focuses on several kinds of task:

• Tracking changes across video: Following the same object, distinguishing action sequences, and revisiting unclear parts of a clip.
• Understanding space and action consequences: Determining how objects connect, where physical support lies, and what might happen following a movement.
• Applying visual understanding to real-world workflows: Labeling robot training videos, breaking robot tasks into steps, describing fine-grained video details, as well as counting, solving mazes, and spot-the-difference puzzles.

When a Detail Is Unclear, Look Again

To determine whether a train is decelerating, merely recognizing that "this is a train" is not enough. The model must lock onto the edge of a specific window and compare how far it travels across equal time intervals: a shrinking distance is evidence that the train is slowing down.

An actual Glimpse run follows this approach. The train enters the frame around 2.2 seconds, with substantial window-edge displacement early on; by 4.5–4.9 seconds, the shift narrows sharply, dropping to just a few pixels near the end of the clip. After revisiting these timestamps to verify the displacement, it answers "decelerating." This requires linking visual cues across time—a single still photo cannot establish a velocity trend.

Another 6.13-second animation highlights the utility of "revisiting" frames even more clearly. Two bluebirds circle, land on a door handle, and then take off toward an illuminated arched window. The camera pans rapidly to follow them. Is this a continuous pan, or does a cut skip a portion of the movement? The answer determines whether the event is continuous and how the two birds arrive in sequence.

Glimpse initially inspects the clip at 8 frames per second (fps), then re-examines 4.3–5.2 seconds, a 0.9-second interval, at roughly 14 fps. The newly inspected wall positions form a continuous trajectory, supporting the conclusion that "the camera was panning continuously." It also distinguishes the two birds by their pink ribbon and light-green headpiece, confirming their sequential entry through the arched window. The extra inspection focuses on the short interval that determines the answer.

Revisit these 0.9 seconds: does the wall move continuously?4.30–5.20s · Bird video sampled frames

← Previous frameInspect intermediate framesNext frame →

Stepping forward or backward shows the wall surface and window frame aligning continuously; reference frames at both ends remain pinned. Here, 13 evenly spaced frames are extracted from the source video for reader observation, rather than reproducing the model's exact sampling trace.

Elorian refers to this capability as "visual thinking": visual representations actively participate in reasoning, allowing the model to revisit and refine its visual interpretation throughout the thought process. The "visual scratchpad" visible to users on the platform displays these operations, such as zooming in on regions, replaying video segments, and tracking visual features. The model can also invoke code execution, image transformations, and video tools to assist inspection. Cropped regions, sampled frames, and transformed views are fed back to the model, serving as visual grounding for downstream reasoning steps.

The model is also trained end-to-end to use these multimodal tools and coordinate multiple sub-Agents, enabling them to exchange text, intermediate visual artifacts, and internal representations to solve problems collaboratively. How these internal representations are stored and updated under the hood was not elaborated upon in the release materials; the external tool actions allow readers to witness how evidence is re-gathered, but they do not reveal the complete neural network mechanisms.

From Recognizing Objects to Understanding How They Evolve

Cross-frame tracking resolves temporal continuity, but physical judgment further requires identifying the spatial relationships between objects.

Three people are lifting a large tire. If the woman in the striped shirt suddenly lets go, who is most likely to get hit? The critical clue is that the lower rim of the tire remains on the grass, and the elevated side has not yet passed the vertical balance point. Upon losing support, the raised side will fall back down pivoting around the grounded edge, sweeping directly toward the woman leaning over on that side.

The same relationship judgment appears in a railroad switch operation video: flipping the lever from right to left shifts the linked switch rail, altering the route available to trains. Simply describing that "she moved her arm" is insufficient; one must connect the physical action to what it alters.

Counting imposes a similar bar for detail. To count "open pliers," the model must verify both that the object is a pair of pliers and that it is currently open. Closed pliers fulfill only the first condition; scissors, while sharing handles and a pivot, belong to an entirely different category. The official reference count is 4.

These examples demonstrate that recognizing "what is present" is merely the starting point. Answering concrete questions often requires preserving contact points, linkage relationships, physical states, and object identities; missing a single detail can send downstream reasoning down a flawed path.

Robots Need to Know When a Step Is Actually Complete

In robot training footage, "the robotic arm moved" and "the object has been properly placed" are two very different things.

Glimpse's annotation demo places bounding boxes around and labels objects such as a sponge, marker, and scissors, segmenting the pick-and-place actions for all three objects into 6 timestamped events. In this way, continuous video footage is transformed into actionable training data: which object was picked up, when it was picked up, and when it reached its target destination. Another official record further distinguishes the completion timestamps for "removing the marker from a container" and "placing it on the table and releasing grip" at 8.9 seconds and 14.085 seconds, respectively.

The same completion verification can guide real-time task execution. Give a robot the instruction: "Put the cream cheese in the bowl, then place the bowl on top of the cabinet." Glimpse handles high-level step decomposition, inspecting camera feeds while the arm operates to confirm whether step one has finished before triggering step two. Physical execution is handled by Physical Intelligence's π0.5 vision-language-action model.

Planning costs for this benchmark run came out to $0.035 and $0.249, respectively—Glimpse was approximately 86% cheaper. Note that this cost reflects only this specific task under the stated settings. While it illustrates a lower-cost path for high-level planning, having the model directly output low-level robotic actions remains an ongoing goal for Elorian. The team also plans to pursue native multimodal pretraining, unify understanding with generation, and achieve more accurate scene reconstruction—goals that remain to be realized in future work.

The Value of Video Captioning Lies in What Summaries Omit

A blanket statement like "someone is dancing" does nothing to help a creator pinpoint the exact moment a dancer lifts her skirt behind her head. Dense video captioning records these fine-grained details alongside timestamps, making footage searchable by specific actions, events, or character interactions.

In a 2-minute 46-second Havana street performance, the demonstration benchmarked 14 distinct checkpoints, including the barefoot dancer, skirt positioning, rightward camera pan, solo dance sequence, and the empty performance area at the end. In this run, Glimpse covered all 14 checkpoints, GPT-6 Sol covered 12, and Gemini 3.8 Flash covered 11; Glimpse averaged 13.7 across three runs.

Another 16-second clip of a packing dispute tests not only action recognition, but also the emotions implied by those actions: one person hurriedly packs a bag, the other extends an open palm in an attempt to plead with them, then buries their head in their hands, before the first person picks up the bag and leaves. Across 13 evaluation checkpoints, the three models covered 13, 12, and 11, respectively, in this run. Visuals substantiate inferences of tension, resistance, and frustration; without dialogue, however, one cannot verify what was said, nor whether the departure signifies a permanent breakup.

These demos use GPT-6 Sol as the comparative baseline label; the main official benchmark table additionally compares against GPT-6.1 Sol. Both videos employ custom event checklists and sampling configurations; the figures reflect detail coverage within these specific illustrative test cases.

When the Camera Pans Away and Returns, the Same Object Must Not Be Recounted

Counting in video adds a deduplication challenge: when the camera moves away and pans back, the same puppy must not be counted twice. The courtyard demo assigns a persistent color and ID to each dog, ultimately arriving at a count of 5.

A screen recording of construction vehicles reports "approximately 30 vehicles" alongside timestamped evidence, noting that different camera angles may capture the same equipment; this estimate does not provide an independently verified vehicle-by-vehicle ground truth.

Where Capabilities Caught Up, and What Remains Hard

With a compact team and several hundred GPUs, Elorian post-trained Glimpse on top of an open-weights foundation model. The foundation model's identity and exact parameter count have not been disclosed, and building upon an open-weights base does not mean Glimpse's weights are openly released.

According to the official benchmark table, Glimpse outperforms GPT-6 Astra on four benchmarks: robotic manipulation events, scientific chart reasoning, short video captioning, and open-vocabulary counting, while tying on embodied spatial reasoning. Below, all 12 benchmarks, 5 models, and input/output pricing tiers from the original table are preserved, showing both strengths and shortcomings side by side:

The table uses separate metrics for different benchmarks; their scores cannot be added into an overall score. Benchmarks primarily used each model's high reasoning tier and native agent environment, with certain exceptions for video settings, and HSS incorporates results published in prior literature. Glimpse still trails Astra on BabyVision fine-grained visual reasoning, DREAM-1K dense video captioning, TOMATO fine-grained temporal reasoning, and CountQA product counting; its scientific chart score also sits slightly below Opus 5.5.

The maze and spot-the-difference demos are striking: Glimpse solved 20/20 mazes; on spot-the-difference, its strict score was 47/53 because its returned bounding boxes for one image pair had the horizontal and vertical axes reversed. Swapping those axes and rescoring raised the result to 53/53. Applying this identical correction rule across all models yielded no score change for the other two. GPT-6 Sol scored 10/20 and 41/53 (with two false positives), while Gemini scored 3/18 (with two timeouts) and 29/53. Here, Glimpse had access to hosted tools to write and execute code, whereas the other two models had no tools. The results reflect each model's capabilities under its respective operational setup.

Physical intuition remains difficult. On 12 randomly sampled questions from HSS probing "where will it go, where will it land, and what will it collide with," Glimpse answered 5 correctly, Gemini also answered 5 correctly, and GPT-6 Sol answered 2; Glimpse had access to tools, while the other two did not. The 3 selected examples highlighted in the demo lean toward successful cases and should not be confused with the overall accuracy across all 12 problems.

The motion animations played after each question were generated by Seedance 2.5 from still frames to illustrate the reference answers. Evaluated models saw only the still frame and prompt, never the animation. Tracing a superficially plausible trajectory remains a far cry from reliably reasoning through real-world physical consequences.

Beyond Video Comprehension: Recreating Patterns and Rhythm

In a follow-up thread to the launch post, Andrew Dai showcased two creative outputs: a pelican wearing a red scarf riding a bicycle, and a lighting animation generated by referencing real-world footage of the Santa Monica Ferris wheel. He noted that the pelican task received no task-specific tuning, while the Ferris wheel task requested "an editable vector animation with a similar lighting style."

This application bridges visual comprehension and creative synthesis: the objective is to translate geometric structures and lighting dynamics from reference video into editable graphic assets. OpenRouter currently lists Glimpse's output modality as text, so the animation demos should not be taken as direct proof of native video generation capabilities. For creators, the more compelling experiment is whether the model can translate observed structure and rhythm into an editable representation.

How to Try It and How Pricing Works

The Elorian chat platform serves as the main playground; Andrew Dai noted in his launch thread that the chat interface is free to try and displays the visual scratchpad. Developers can access the official API or use OpenRouter under elorian/glimpse; the latter lists a context window of 262,144 tokens, accepting text, image, and video inputs while returning text.

The most direct way to test it is to upload a pet clip featuring camera movement and temporary occlusions, prompting: "How many distinct animals appear in total? Maintain a persistent ID for each, and list the timestamps and visual characteristics that confirm each re-emergence." The counts, identity mappings, and timestamps in the output can then be verified against your own video frame by frame.

Official API pricing stands at $0.60 per million input tokens and $3 per million output tokens. According to the official comparison chart, both input and output unit rates are 94% lower than Astra, 85% lower than Opus 5.5, 70% lower than GPT-6.1 Sol, and 20% lower than Gemini 3.8 Flash's introductory preview pricing.

Unit pricing differences do not translate linearly into end-to-end task costs. Revisiting regions and inspecting more frames take time and additional tokens, and Elorian has highlighted improving token efficiency as a primary upcoming objective. In practice, task expense, latency, and accuracy must be weighed together. The single robotics benchmark provides one cost data point; more complex video analysis and physical reasoning will require validation against your own specific workloads.