Driver UI Tests Guide

Guidelines for writing UI tests using IDE Starter and UI Driver frameworks.

Common Imports

// Driver core
import com.intellij.driver.client.Driver
import com.intellij.driver.sdk.waitForProjectOpen
import com.intellij.driver.sdk.advancedSettings

// Test utilities
import com.intellij.driver.tests.utils.waitForIndicators
import com.intellij.driver.tests.utils.Plugin
import com.intellij.driver.tests.utils.PluginInstaller
import com.intellij.driver.tests.utils.Plugins

// IDE Starter framework
import com.intellij.ide.starter.driver.runIdeTest
import com.intellij.ide.starter.ide.IDETestContext
import com.intellij.ide.starter.models.IDEStartResult
import com.intellij.ide.starter.models.VMOptions
import com.intellij.ide.starter.runner.IDERunContext
import com.intellij.ide.starter.runner.Starter
import com.intellij.ide.starter.utils.catchAll

// Extended test infrastructure
import com.intellij.ide.starter.extended.allure.AllureHelperExtended.step
import com.intellij.ide.starter.extended.allure.Subsystems
import com.intellij.ide.starter.extended.engine.newTestContainerExtended
import com.intellij.ide.starter.extended.engine.TestContainerExtended
import com.intellij.ide.starter.extended.license.StagingLicenseGenerator.licenseProductCode
import com.intellij.ide.starter.extended.loadMetadataFromServer
import com.intellij.ide.starter.extended.setupTestMetadataSchemeWithGroupsFromCode

// Test framework
import com.intellij.testFramework.TestApplicationManager

Test Structure

• Tests use JUnit 5 with an IDE Starter framework and UI Driver framework (community/platform/remote-driver)
• Test case projects are represented by the com.intellij.ide.starter.models.TestCase see src/com/intellij/ide/starter/project
• Tests run against specific IDE (community/tools/intellij.tools.ide.starter/src/com/intellij/ide/starter/ide/IdeProductProvider.kt)

Test project examples

See tests/intellij.ide.starter.extended/src/com/intellij/ide/starter/extended/data/cases

Page Object Pattern or UiComponent

Page objects extend UiComponent with ComponentData constructor:

class MyPageObject(data: ComponentData) : UiComponent(data) {
    val myButton = x { byAccessibleName("Button Name") }
    val myPanel = x { byClass("PanelClassName") }

    fun clickMyButton() {
        myButton.click()
    }
}

// Extension function on Finder to create the page object
fun Finder.myPageObject(): MyPageObject = x(
    xQuery { byAccessibleName("Root Element Name") }, MyPageObject::class.java
)

// Use specific ui component to specify the context of the search
fun AnotherPageObject.myPageObject(): MyPageObject = x(
  xQuery { byAccessibleName("Root Element Name") }, MyPageObject::class.java
)

Element Selectors

UI test examples

See directory tests/remote-driver-tests

Common UI Components

See directory community/platform/remote-driver/test-sdk/src/com/intellij/driver/sdk/ui

Scoping Element Searches

When multiple elements match a selector, scope to a parent element:

// BAD - will fail if multiple InstallButtons exist
ui.x { byClass("InstallButton") }.click()

// GOOD - scope to a parent element first
val detailPane = ui.x { byClass("PluginDetailsPageComponent") }
detailPane.x { byClass("InstallButton") }.click()

Keyboard Interactions

element.keyboard { typeText("search text") }
ui.keyboard { key(KeyEvent.VK_ENTER) }
ui.keyboard { hotKey(KeyEvent.VK_META, KeyEvent.VK_COMMA) }  // Cmd+,

Writing Tests

Required Annotations

Every UI test must have the following annotations at the class level:

For tests linked to TestOps test cases, also add:

• @AllureId("test_case_id") - Links the test to the TestOps test case ID

The annotation values should match the corresponding TestOps custom fields (Subsystem, Feature, Component).

Available annotations: See tests/intellij.ide.starter.extended.allure/src/com/intellij/ide/starter/extended/allure/Annotations.kt

Example with TestOps test case:

@Subsystems.Java
@Features.Completion
@Components.Editor
class MyTestFromTestOps {

    @Test
    @AllureId("318541")  // Required when test case exists in TestOps
    fun `my test from testops`(testInfo: TestInfo) {
        // ...
    }
}

Example for new test (not yet in TestOps):

@Subsystems.UI
@Features.PluginManager
@Components.Miscellaneous
class MyNewTest {

    @Test
    fun `my new test`(testInfo: TestInfo) {
        // ...
    }
}

Basic Test Structure

@Subsystems.Java
@Features.Completion
@Components.Editor
class MyTest {
    val testCase = TestCase(IdeProductProvider.IU, myProject)

    @Test
    @AllureId("123456")  // Required if test case exists in TestOps
    fun `my test name`(testInfo: TestInfo) {
        val context = Starter.newContext(testName = "TestName", testCase = testCase)

        context.applyVMOptionsPatch {
            addSystemProperty("ide.ui.non.modal.settings.window", "true")
        }

        context.runIdeTest(testName = testInfo.displayName) {
            waitForIndicators(5.minutes)  // Wait for indexing to complete

            step("Step description") {
                // Test actions here
            }
        }
    }
}

Waiting for Project Import and Indexing

Always wait for indicators at the start of your test:

waitForIndicators()

This ensures the project is fully imported and indexed before interacting with the IDE.

Opening Files

Use openFile instead of UI-based file navigation:

// GOOD - Direct and reliable
openFile(relativePath = "src/Main.java")

// AVOID - UI-based approach is slower and more fragile
invokeAction("GotoFile", now = false)
ui.keyboard { typeText("Main.java") }
ui.keyboard { key(KeyEvent.VK_ENTER) }

invokeAction: now Parameter

The now parameter controls whether the action completes before continuing:

// now = true: Waits for action to complete (use when keyboard input follows)
invokeAction("ToggleBookmarkWithMnemonic", now = true)
ui.keyboard { key(KeyEvent.VK_1) }  // This input goes to the bookmark dialog

// now = false: Returns immediately (use when waiting for UI to appear)
invokeAction("ShowSettings", now = false)
ui.x { byClass("SettingsDialog") }.shouldBe { present() }

Rule: Use now = true when the next step is keyboard input to prevent input going to the wrong component. Rule: Use now = false when you expect to the UI dialog to appear.

Custom Wait Conditions

Use waitFor to wait for specific conditions:

waitFor("description of what we're waiting for", 30.seconds) {
    ui.x { byClass("MyComponent") }.present()
}

waitFor("text to appear", 10.seconds) {
    ui.x { byClass("Tree") }.hasText("expected text")
}

Running Tests from Terminal

Driver tests require a fully built IDE. There are several ways to run them:

Option 1: Using tests.cmd (Recommended)

The tests.cmd script builds the IDE from sources and runs tests. Recommended for dev server mode.

Example:

./tests.cmd \
  --module intellij.driver.tests \
  --test com.intellij.driver.tests.idea.java.FindAndGoToTest

Key parameters:

• --test - fully qualified test class name (or pattern)
• --module intellij.driver.tests - required for driver tests

Example with specific test:

./tests.cmd \
  --module intellij.driver.tests \
  --test com.intellij.driver.tests.idea.ultimate.httpclient.BuiltInHttpClientBrotliCompressionUiTest

Debugging Test Failures

Output Locations

After test failure, check:

• UI hierarchy: out/ide-tests/tests/{IDE-version}/{TestName}/{test-method}/log/ui-hierarchy/ui.html
• IDE log: out/ide-tests/tests/{IDE-version}/{TestName}/{test-method}/log/idea.log
• Screenshots: out/ide-tests/tests/{IDE-version}/{TestName}/{test-method}/log/screenshots/
• Exceptions: out/ide-tests/tests/{IDE-version}/{TestName}/{test-method}/error/

Common Issues

1. Element Not Found: Check UI hierarchy HTML for the correct accessible name or class
2. Multiple Elements Match: Scope search to parent element

Critical Rules

• Never use Thread.sleep() or delay() - Driver framework automatically waits for UI elements
• Wrap test logic in step("description") { } for better logs
• Verify assertions actually fail – Comment out the action being tested and confirm the test fails. If it still passes, your assertion is too weak.
• Use common UI components – create new if necessary
• Always check UI hierarchy to understand the UI state when a test fails