Writing Tests

Canonical guide for where tests live, how to write them, and how to run them. Pairing rules come from .projections.json.

Projections rule (required)

Follow .projections.json when creating or renaming tests.

App library code

Path under app/lib/ maps 1:1 under app/test/, with a _test.exs suffix.

API endpoints: one endpoint module → one test module. Do not bundle several endpoints into a shared file (e.g. no versions_test.exs covering list_versions and get_version).

app/lib/operately_web/api/documents/list_versions.ex
→ app/test/operately_web/api/documents/list_versions_test.exs

app/lib/operately_web/api/documents/get_version.ex
→ app/test/operately_web/api/documents/get_version_test.exs

Cross-cutting assertions for another endpoint belong in that endpoint’s test file (e.g. expected_version on update → update_test.exs).

External (token) API coverage is a separate layer — see External queries and mutations.

Feature / CLI / MCP e2e

Enterprise

Running tests

Prefer make test FILE=... from the repo root (runs inside ./devenv).

# Elixir unit / API / feature (path may be app/test/... or test/...)
make test FILE=app/test/operately_web/api/documents/list_versions_test.exs
make test FILE=app/test/features/goal_creation_test.exs:21

# Jest
make test FILE=assets/js/path.spec.ts

Always pass FILE= for the specific test(s) under change. Do not run suite-wide targets while iterating — they take too long:

• make test / make test.mix / make test.mix.unit / make test.mix.features / make test.npm / make test.ee without FILE= (full suite)
• INDEX=… TOTAL=… make test.mix.features (parallel CI shards; still a large slice)
• make test.mix.features FILE=… (FILE is ignored; runs the feature suite)
• bare mix test from the host with no path (use make test FILE=… or ./devenv … mix test <path> as below)

Feature tests in agent / CI mode

Local non-CI feature tests expect Vite on localhost:4005. Without it, Wallaby can load a blank page. For CI-equivalent runs:

make test.build
./devenv bash -c 'cd app && CI=true mix test test/features/space_kanban_test.exs'
./devenv bash -c 'cd app && CI=true mix test test/features/project_tasks_test.exs:425'

Pass CI=true explicitly in the inner command (root .env may have empty CI=).

Screenshots: host screenshots/ → container /tmp/screenshots. Clear with make test.screenshots.clear.

If a killed feature run leaves port 4002 busy:

./devenv bash -c "ps -ef | grep 'beam\\|mix test' | grep -v grep"
./devenv bash -c "kill <pid>"

Test types and case modules

DB tests clean up via transactions; no manual teardown.

External queries and mutations

The external API (API tokens for CLI/integrations) is covered by a second layer beside the normal per-endpoint *_test.exs files.

When you add or expose an endpoint on OperatelyWeb.Api.External:

1. Keep (or add) the normal TurboCase *_test.exs for behavior/permissions.
2. Add a QuerySpec / MutationSpec with setup/1, inputs/1 (optional), and assert/2. Override query_name/0 or mutation_name/0 when the default (underscored last module segment) is wrong — names are usually "resource/action" (e.g. "documents/list_versions").
3. Register the module in __spec_modules__/0 in queries.ex or mutations.ex.
4. Run the matching auth_test.exs (targeted), not the whole suite.

app/lib/operately_web/api/documents/list_versions.ex
→ app/test/operately_web/api/documents/list_versions_test.exs          # behavior
→ app/test/operately_web/api/external_queries/queries/documents/list_versions.ex  # external auth spec
→ register in external_queries/queries.ex

Wrapper endpoints live under queries/wrappers/ or mutations/wrappers/ (e.g. documents/update_document).

auth_test.exs checks every registered external endpoint for:

• coverage (no missing/extra/invalid specs vs OperatelyWeb.Api.External)
• no token → 401
• browser session on external → 401
• API token on internal API → rejected
• read-only token → queries succeed; mutations → 403
• full token → 200 and assert/2 on the response

make test FILE=app/test/operately_web/api/external_queries/auth_test.exs
make test FILE=app/test/operately_web/api/external_mutations/auth_test.exs

Example query spec:

defmodule OperatelyWeb.Api.ExternalQueries.Queries.Documents.ListVersions do
  use Operately.Support.ExternalApi.QuerySpec

  @impl true
  def query_name, do: "documents/list_versions"

  @impl true
  def setup(ctx) do
    ctx
    |> Factory.setup()
    |> Factory.add_space(:space)
    |> Factory.add_resource_hub(:hub, :space, :creator)
    |> Factory.add_document(:document, :hub)
  end

  @impl true
  def inputs(ctx), do: %{document_id: Paths.document_id(ctx.document)}

  @impl true
  def assert(response, _ctx) do
    assert is_list(response.versions)
    assert length(response.versions) >= 1
  end
end

Factory pattern (preferred for new tests)

Use Operately.Support.Factory (app/test/support/factory.ex) so entities are related correctly. Prefer Factory over wiring *_fixture calls by hand in new tests (older tests may still use fixtures).

setup ctx do
  ctx
  |> Factory.setup()
  |> Factory.add_space(:marketing)
  |> Factory.add_project(:website, :marketing)
end

API example: use OperatelyWeb.TurboCase, then Factory.setup() / Factory.log_in_person/2 and query/3 or mutation/3.

Feature test step pattern

Feature tests chain steps from a support module. Steps modules typically use Operately.FeatureCase (which imports Operately.FeatureSteps and aliases UI, Factory, Paths).

# app/test/features/goal_creation_test.exs
defmodule Operately.Features.GoalCreationTest do
  use Operately.FeatureCase
  alias Operately.Support.Features.GoalCreationTestSteps, as: Steps

  setup ctx, do: Steps.setup(ctx)

  feature "create a new goal", ctx do
    ctx
    |> Steps.visit_new_goal_page()
    |> Steps.fill_in_goal_form("Example Goal")
    |> Steps.submit()
    |> Steps.assert_goal_added("Example Goal")
  end
end

# app/test/support/features/goal_creation_steps.ex
defmodule Operately.Support.Features.GoalCreationTestSteps do
  use Operately.FeatureCase

  def setup(ctx) do
    ctx
    |> Factory.setup()
    |> Factory.add_space(:space)
    |> Factory.log_in_person(:creator)
  end

  step :visit_new_goal_page, ctx do
    ctx |> UI.visit(Paths.new_goal_path(ctx.company))
  end
end

Email assertions

Unit/API: assert_email_sent/1 (Swoosh). Feature: UI.assert_email_sent/3 or Operately.Support.Features.EmailSteps.

Migration-related testing

Schema/data migration rules: ecto-migrations skill. After migration changes:

make test.db.reset
make test.db.migrate
make test.mix

Data changes: app/test/operately/data/change_NNN_*_test.exs.

Common pitfalls

1. Running the full suite instead of make test FILE=app/test/... for the files under change
2. INDEX/TOTAL on a single-file run
3. New tests hand-rolling fixtures instead of Factory
4. Mixing conflicting sync/async DB tests
5. Hardcoding IDs instead of factory-built entities
6. Grab-bag *_test.exs covering multiple unrelated modules/endpoints
7. New external endpoint without a QuerySpec/MutationSpec + registry entry
8. Feature tests without CI=true / assets when Vite is not running

Checklist

• New module/endpoint has a matching *_test.exs per projections
• Test module name mirrors the source (ListVersions → ListVersionsTest)
• No grab-bag test file for multiple unrelated modules
• New setup uses Factory where practical; API tests use TurboCase
• New external API endpoint has a QuerySpec/MutationSpec, is listed in queries.ex / mutations.ex, and auth_test.exs still passes
• Feature/CLI/MCP tests have a steps alternate when required by projections
• Run with make test FILE=app/test/... (feature CI: CI=true via devenv)