List the elements on screen

Open a website and use parse() to list the UI elements on screen.

This test opens Airbnb in Chrome and calls parse() to detect every UI element on screen.

Watch it run

The test

parse.test.mjs

import { describe, it } from "vitest";
import { TestDriver } from "testdriverai/vitest/hooks";

describe("Parse Test", () => {
  it("should open Airbnb and parse the screen", async (context) => {
    const testdriver = TestDriver(context);
    await testdriver.provision.chrome({ url: "https://www.airbnb.com" });

    // The SDK automatically outputs elements as a formatted table
    const result = await testdriver.parse();
    console.log(`Found ${result.elements?.length || 0} elements`);
  });
});

How it works

• parse() analyzes the screen and resolves to an object. Its elements array lists each detected element.
• The result also has annotatedImageUrl, a screenshot with the elements marked.
• It also has imageWidth and imageHeight.
• Use parsing to explore an unfamiliar screen before you write find() descriptions.

Run it

The test is in the TestDriver SDK repository. Clone it, install dependencies, and run the file:

git clone https://github.com/testdriverai/testdriverai
cd testdriverai
npm install
npx vitest run examples/parse.test.mjs

The test reads your API key from TD_API_KEY. Create a key at console.testdriver.ai/team/api-keys. Then set it in your environment or a .env file.

Related

• parse()
• Locate elements