Auto-healing is a powerful feature in TestDriver that ensures your tests remain resilient even when the application under test changes. Use --heal to enable auto-healing in your test runs. This feature allows TestDriver to automatically update tests when minor changes occur, such as text updates or small UI adjustments, without requiring manual intervention.

​
Why use auto-healing?

In modern software development, applications frequently undergo changes, whether it’s UI updates, text modifications, or layout adjustments. These changes can lead to test failures, requiring developers to manually update test scripts to reflect the new state of the application. This means that if a test fails due to a change in the application, TestDriver will attempt to recover by finding the updated element or text and retrying the action. If successful, it will update the test script accordingly. This feature is particularly useful in continuous integration (CI) environments, where tests need to adapt to frequent changes in the application.

​
How to enable auto-healing

To enable auto-healing in TestDriver, simply add the --heal and --write flags when running your tests. The --heal flag instructs TestDriver to attempt to automatically update any failing tests due to minor changes in the application, while --write ensures the updated test files are saved. 
npx testdriverai@latest run my-tests.yaml --heal --write

​
How auto-healing works

When TestDriver steps fail, the AI will progressively fall back to attempt to complete the test. When running with --heal and --write, TestDriver will update the test files locally with any changes made during auto-healing. You can then use GitHub Actions to commit these changes and create a pull request for review.

​
Benefits of auto-healing

  • Reduced Maintenance: Eliminates the need for manual updates to tests when minor UI changes occur.
  • Increased Test Resilience: Ensures tests adapt to changes without breaking.
  • Continuous Integration: Keeps your test suite aligned with the latest application changes.
  • Developer Collaboration: Automatically opens a PR, allowing developers to review and approve updates.

​
How auto-healing works

  1. Test Execution: TestDriver runs your test suite as usual.
  2. Failure Detection: If a test fails due to a change in the application (for example, text or UI updates), the AI identifies the failure point.
  3. Recovery Attempt: The AI uses its adaptive capabilities to locate the updated element or text and retries the action.
  4. Test Update: If the recovery is successful, TestDriver updates the test script with the new element or text.
  5. Commit and PR: GitHub Actions can detect the changes and automatically commit them and create a pull request for review.

​
Example: Button text changes from “Submit” to “Send”

1

Initial Test

- command: hover-text
  text: Submit
  description: Submit button in the form
  action: click
2

Application Update

  • The button text is changed from “Submit” to “Send” in the application.
3

Test Execution

  • TestDriver runs the test and fails to find the “Submit” button.
4

Auto-Healing

  • The AI detects the failure and searches for a similar element.
  • It identifies the “Send” button as the updated element and retries the action.
5

Test Update

  • The test script is updated to reflect the new button text:
- command: hover-text
  text: Send
  description: Submit button in the form
  action: click
6

Commit and PR Creation

  • GitHub Actions detects the changes and commits the updated test to a new branch and opens a PR:
    • Branch Name: auto-heal-update-submit-to-send
    • PR Title: Auto-Healed Test: Updated "Submit" to "Send"

​
Example GitHub Action for Auto-Healing

Here’s how you can configure a GitHub Action to enable auto-healing and automatically create PRs for any test updates: 
name: TestDriver Auto-Healing

on:
  push:
    branches:
      - main
  pull_request:
  workflow_dispatch:

jobs:
  test:
    name: "Run Tests with Auto-Healing"
    steps:
      - name: Check out repository
        uses: actions/checkout@v4

      - name: Run TestDriver with Auto-Healing
        run: npx testdriverai@latest run testdriver/onboarding.yaml --heal --write --headless
        env:
          TD_API_KEY: ${{ secrets.TD_API_KEY }}

      - name: Commit auto-healed changes
        run: |
          git config --local user.email "action@github.com"
          git config --local user.name "GitHub Action"
          git add testdriver/
          git diff --staged --quiet || git commit -m "Auto-healed test updates"

      - name: Create Pull Request
        uses: peter-evans/create-pull-request@v5
        with:
          token: ${{ secrets.GITHUB_TOKEN }}
          commit-message: "Auto-healed test updates"
          title: "Auto-Healed Test Updates"
          branch: auto-heal-updates
          base: main
          body: |
            This PR contains auto-healed test updates generated by TestDriver.

            Please review the changes to ensure they are correct before merging.

​
Limitations of Auto-Healing

  • Major UI Overhauls: Auto-healing is best suited for minor changes (for example, text updates, small layout adjustments). Significant UI changes may still require manual intervention.
  • Ambiguous Changes: If multiple elements match the updated criteria, the AI may require additional context to make the correct decision.
  • Step Deletion: If UI or wizard pages are removed, the AI may not be able to recover. In such cases, manual updates are necessary.