argent-device-interact

Each tool's own description carries its parameters, defaults, return shape and examples. This skill covers choosing between tools and the gotchas those descriptions don't state.

1. Before You Start

• Get a target id from list-devices; if none is ready, boot-device one. Full setup flow: argent-ios-simulator-setup / argent-ios-device-setup / argent-android-emulator-setup. The same tool names drive iOS, Android and Chromium; each takes the udid from list-devices.
• Physical iPhone (kind: "device"): read argent-ios-device-interact first. Only the tools it lists exist there, and describe sees only the app launch-app registered.
• Read the matching reference before driving these targets: references/chromium.md for a chromium target, references/harmonyos.md for a harmony target, references/foldables.md for a device list-devices marks foldable: true (or before calling fold).
• Load a deferred tool's schema with ToolSearch before its first call (e.g. gesture-pinch, gesture-rotate, gesture-custom); without it, numbers go out as strings and fail validation.
• If you delegate device tasks to sub-agents, make sure they have MCP permissions.

2. Choosing the Right Tool

3. Finding Tap Targets

After an action moves you to a different screen, or whenever you don't know an element's coordinates, discover before tapping. Interaction results already carry the element tree (--- Elements after action (describe) ---): use it, and call a discovery tool only when no fresh tree covers the current screen.

If describe returns an empty tree, follow its hint; with no hint, the screen is blank, loading or unlabeled: check the attached screenshot and retry once content has loaded.

For a React Native testID or component name that describe does not show, use debugger-component-tree; take tap points from describe.

React Native: LogBox toasts sit over the bottom of the screen and catch taps meant for what is under them. Argent hides them when a debugger-* tool connects and after debugger-reload-metro; after restart-app, call a debugger-* tool again. After any other reload they come back: close each by tapping the right end of its describe frame (the X), not its middle.

System and permission dialogs: on a simulator or emulator, describe returns a visible dialog's buttons with tap coordinates; keep using it for in-app modals too. Fall back to screenshot only if it does not expose the controls: then tap one obvious button (Allow, OK, Don't Allow, Not Now, Continue) at a time, check the returned screenshot, and go back to describe once the dialog is gone.

Prefer the dialog over the Settings tool. When the app triggers its own permission prompt, answering it is the real user path. Use settings-permissions only when you can't make the change through the app: set a permission before the app asks, re-enable one the user denied (iOS won't re-prompt), or reset it so the prompt reappears. See argent-settings-permissions.

4. Waiting: await-ui-element and await-screen-idle

Never poll screenshot/describe in a loop to wait for something. await-ui-element blocks server-side on the same tree describe reads. It has no bare-timer mode; for a plain pause, use your own harness sleep. Its description covers conditions, selector matching and defaults. What it does not tell you:

• A hidden check that succeeds immediately may be a false pass: its note then says the selector never matched anything. Treat that as a failed check and fix the selector; it does not mean "the element went away".
• The synthetic ROOT container describe prints is never matched, so a role like AXGroup/html won't trivially "match the screen".
• To disambiguate a loose selector, pin role to a text role like StaticText; that skips a same-named button.

await-screen-idle is for live diagnosis after a launch or navigation, before a raw tap on an element that may still be moving. Continue only when settled: true, and pair it with an await-ui-element for the destination: stillness does not identify a screen, and the tool can return during a presentation-layer animation. Do not record it in an e2e test; tests use await: { idle: true }, which also compares pixels.

5. Action Sequencing with run-sequence

Use run-sequence when every step is known in advance; if a step depends on what the previous one shows, use separate calls. Its description lists the allowed tools and their arguments. Typical batches:

• "scroll to bottom", "scroll until X": 3-5 swipes
• "clear and retype a field": triple-tap to select all, then type the new value
• typing then submitting: a keyboard text step, then a keyboard enter step (one call cannot carry both)
• "submit form": fill every field, then tap submit
• "go back to X": a known tap path

Gate a step that depends on a screen transition with an await-ui-element step rather than a fixed delayMs. If its condition is not met before the timeout, the sequence stops there and the later steps do not run, so a tap can't fire against a screen that never appeared. Any failing step also stops the sequence, with partial results.

6. Text Entry and Secrets

• Tap the field first. Typing or pasting with no focused field is a silent no-op.
• paste is not a faster keyboard. keyboard types the way a user types and stays the default for every text entry. Use paste only where a real user would paste: a 2FA / OTP code from another app, a long link or token, or testing how the app handles pasted input. That it can carry text keyboard can't type on a platform (multi-line text, non-ASCII on Android) is not a reason on its own.
• Credentials: type them as a {{secret:<NAME>}} placeholder in text (keyboard, paste, run-sequence keyboard steps, e2e type steps), never as plaintext, and never ask the user for the value. Do not describe or screenshot a non-secure field you just filled with a secret: submit or navigate away first, then verify the resulting screen.

7. Screenshots

Interaction tools already attach a screenshot. Call screenshot for the initial state, for a baseline before you change visible UI, to check state after a delay, or when the attached capture shows a transitional or loading frame.

Keep the default downscaled capture for UI context and state checks; do not use scale: 1.0 as a readability or tapping aid. Full resolution is only for saving baseline/current PNGs for comparison (scale: 1.0 with includeImageInContext: false); for visual regression checks use the argent-screenshot-diff skill.

If screenshot times out, restart the device transport with stop-simulator-server.

8. Android notes

• First-launch permission prompts: reinstall-app on Android installs with -g, so runtime permissions are pre-granted on first launch.