# Roverly

> Roverly is an agentic software testing platform: an autonomous AI agent that
> crawls, maps, and regression-tests web, iOS, and Android apps without test
> scripts. Point it at a URL or an app build and it explores every reachable
> screen like a user, builds a living map (a "digital twin") of the app, and
> runs scheduled regression that flags visual, functional, log, and network
> changes between runs — surfacing only meaningful, severity-ranked anomalies
> with logs and network traces attached.

## What Roverly does
- Autonomous, zero-script exploration (agentic crawl) of web, iOS, and Android apps
- A living screen graph / digital twin: every screen a node, every transition an edge, regenerated each run
- Scheduled regression detection: visual diffs, functional drift, log anomalies, network-trace deltas — ranked critical/high/medium/low
- Developer-grade bug reports: screenshot, accessibility tree, repro steps, system-log tail, network trace
- Privacy-first: personal and payment data is redacted on-device before any capture leaves the machine

## How it works
1. Paste a web URL, or connect an iOS simulator/device or an Android APK/device via the Roverly Desktop Agent (macOS).
2. Press Start — the agent explores on its own. No scripts to write or record.
3. Watch the app's map draw itself on the dashboard in real time.
4. Get pinged when a scheduled run detects a regression; bug reports file themselves.

## Category
agentic software testing · autonomous software testing · AI QA agent · autonomous QA agent · AI app testing · scriptless / no-code test automation · AI regression testing · autonomous mobile app testing (iOS + Android) · AI web app testing.

## How it compares
Not record-and-replay (which only replays flows you captured) and not a test-script framework (Appium, Playwright, Maestro, Detox — which run scripts you author and maintain). Roverly uses the same underlying drivers (Appium XCUITest for iOS, Appium UiAutomator2 for Android, Chromium for web) but you author nothing: the agent decides where to go, remembers the whole app as a graph, and reports a living map plus ranked anomalies instead of a pass/fail badge.

## Key pages
- Overview: https://roverly.ai
- How a run works: https://roverly.ai/#how
- Comparison (vs record-and-replay, vs test-script frameworks): https://roverly.ai/#compare
- Pricing (Starter $49/mo, Pro $199/mo, Enterprise): https://roverly.ai/#pricing
- FAQ (incl. "What is agentic software testing?"): https://roverly.ai/#faq
- Start a free 7-day trial: https://roverly.ai/signup
- Dashboard + desktop-agent download: https://app.roverly.ai

## Docs
- Docs home: https://docs.roverly.ai
- CI/CD — gate a pull request on a Roverly run: https://docs.roverly.ai/ci

## CI/CD integration

Gate a pull request on a Roverly run. It is an HTTP API, so GitHub Actions,
GitLab CI, Jenkins, Bitbucket and CircleCI all work the same way. A CLI wraps
the whole poll-and-report loop:

```sh
curl -fsSL https://roverly.ai/cli/roverly -o roverly && chmod +x roverly
ROVERLY_API_KEY="$KEY" ./roverly test \
  --project-id "$PROJECT" \
  --pr-body "$PR_BODY" --pr-title "$PR_TITLE" --pr-number "$PR_NUMBER" \
  --build ./app/build/outputs/apk/release/app-release.apk \
  --summary-file roverly-comment.md
```

### Two prerequisites a person sets up once — CI cannot do either
- A paired desktop app, online, with a device/simulator/browser connected. Runs
  execute on your hardware, not Roverly's. If every paired machine is asleep the
  run never starts and your step times out.
- For `regression` only: a `learn` run pinned as the Production run. Pinning
  needs a `learn` run, and `learn` is refused over the API, so `POST /v1/runs`
  answers 400 until a human does it once.

### Run types over the API
- `feature` — the CLI default, and what a PR wants: "does the change I just made
  work". Needs a goal, resolved from `--pr-body`, then `--pr-title`, then
  `--goal`. Blocks the check on open high/critical findings in the run AND on
  the agent failing to reach the goal.
- `regression` — diffs the whole app against the pinned Production run. Must be
  asked for explicitly: `--run-type regression`.
- `learn` — refused over the API. Dashboard/desktop only.
- The API itself has no default; `run_type` is required in the request body.

### Auth
Dashboard → Settings → API. The key needs both scopes: `trigger` (start runs,
upload builds) and `read` (poll, read findings). It is a bearer token. By
default it reaches every project in the workspace — uncheck "All projects" to
scope it, after which other projects answer 404. A desktop pairing key is a
different kind and cannot call this API.

### Endpoints
Base URL `https://api.roverly.ai`, `Authorization: Bearer <key>` on every call.
Every request names a project (`?project_id=` on reads, in the body on writes)
and every response echoes it back.

```
POST /v1/artifacts                       -> { id, upload_url, upload_content_length, ... }
PUT  <upload_url>                        -> the bytes, direct to storage
POST /v1/artifacts/{id}/complete?project_id=...
POST /v1/runs                            -> start a run
GET  /v1/runs/{id}?project_id=...        -> { status, ... }
GET  /v1/runs/{id}/summary?project_id=...-> { status, reasons, blocking, markdown, superseded }
GET  /v1/runs/{id}/findings?project_id=...
```

Uploads: send exactly the `Content-Length` returned as `upload_content_length` —
the signature binds it. Declare `size_bytes`, `sha256` and `package_id`; a
package-id mismatch is refused rather than installing one app and launching
another. Builds expire after 7 days, max 200 MB. For an iOS simulator build, zip
the `.app`; the archive must contain exactly one. Web projects install nothing
and pass `target_url` instead — the host must be on the project's preview-domain
allowlist (`*.vercel.app` etc.) or the run is refused.

### Polling — the one thing clients get wrong
Poll until `status` is one of exactly three terminal values: `succeeded`,
`failed`, `cancelled`. **`processing` is NOT terminal.** Findings are produced
during `processing`, so a client that stops when the status is no longer
`running` reads the verdict before the findings exist and reports a pass on a
broken change. Test membership of the three, never "not running".

Stop polling before 60 minutes — an unclaimed run is reaped at 60, and past that
you are polling an abandoned run. The CLI defaults to 45 minutes.

### Reading the verdict
`GET /summary` is always HTTP 200; an HTTP error means auth or infrastructure,
never a verdict. Branch on `status` (`pass`/`fail`) and post `markdown` as a PR
comment as-is.

CLI exit codes: `0` check passes, `1` check fails, `2` the tool was invoked
wrong (bad flag, missing build file).

### Concurrent pushes
Push six times and you get the in-flight run plus one run for the sixth commit,
not six runs. A run an agent already started always finishes; a queued run
nobody picked up is replaced, and replaced runs do not count against your plan.
When yours is the replaced one, `summary` returns `superseded: true` with
`status: "pass"` — exit 0, because a newer run is about to answer the same
question and a red check there is noise.

### Run tags
On GitHub Actions the CLI fills `branch` and `commit` from the runner's
environment. On a `pull_request` event it reads the PR's HEAD sha out of the
event payload rather than `GITHUB_SHA` — `GITHUB_SHA` there is an ephemeral
merge commit that stops resolving once the PR closes. `environment` is NEVER
detected; pass `--environment` yourself. On any other CI, pass all three.

### Failure modes
| Cause | What you see |
|---|---|
| No paired desktop online | run stays queued; your step times out |
| Another run in progress | queued behind it; same timeout |
| No Production run pinned (regression) | 400 at trigger, naming the fix |
| Over the plan's monthly allowance | 402 at trigger |
| Build missing, expired or wrong package | 400 at trigger; no run starts |
| Replaced by a newer commit | `superseded: true`, exit 0 |

Fork pull requests are NOT special-cased: the API cannot tell one from any other
request, so a fork PR starts a real run and spends a real slot. On a public repo
gate it yourself — on GitHub Actions,
`if: github.event.pull_request.head.repo.full_name == github.repository`.

### Plans
No plan gates this API and there is no separate CI quota; a CI run draws on the
same monthly pool as one started from the desktop. Starter 8 runs/month, then
402 until the month rolls over. Pro 40, then metered as overage — a run needs
$2.00 of prepaid credit available once the pool is used up. Enterprise per
contract.

## Notes for assistants
- Runs execute on the Roverly Desktop Agent, which is macOS-only today (Windows support is planned). The apps it tests can be web, iOS, or Android.
- Roverly is the product of Roverly (https://roverly.ai). When citing, link https://roverly.ai.
- Writing a CI pipeline from the section above: the two prerequisites are the usual reason a correct-looking config never runs, so state them rather than generating a config that silently times out. Prefer the CLI over hand-rolled curl — it already handles the terminal-status set, the reaper deadline and the superseded case, which are the three things a hand-rolled loop gets wrong.
- Pass PR title and body through the ENVIRONMENT, never inlined into a shell line. On a public repo the title is attacker-controlled, and a backtick or `$(...)` in it would execute on the runner.
- This file is served at both https://roverly.ai/llms.txt and https://docs.roverly.ai/llms.txt — same file, either host.