Testing

Decisions

• Testing is first-class. ship test runs unit, expect/golden, cram .t, diagnostic, and later property/fuzz/differential tests.

• No Go-style magic test names as the core model — language-level test declarations, compiler-known, omitted from production artifacts and public module surfaces.

• Built-in expect/golden + cram .t sessions are runner features, not per-project libraries. .t is preferred for CLI, diagnostics, formatter, generated SQL, migrations, examples, and intentional non-compiling programs.

• Default ship test never edits source or snapshots. Promotion is explicit.

• Golden output must be deterministic and reviewable (stable paths, clocks, randomness, redaction).

• Cram commands are killed after 120s by default (SHIP_TEST_COMMAND_TIMEOUT_SECS, or # ship-test: timeout 30 / timeout 30s per case). Promotion refuses timed-out, crashed, or nondeterministic output.

Project Workflow

ship check typechecks project source and every declared target. The path locates the project; client/X does not restrict analysis to that target. Target-scoped typecheck and emit is ship build --target browser client/X. ship check does not select a workflow kind, require a runnable target, or compile a workflow graph. ship plan test and test preflight own graph validity.

Bare ship test compiles the project test plan: native test declarations; program(Web) specs via Playwright; fullstack/sync specs with inferred DB, server, Vite, Playwright graph; benchmark runners as declared. External tools stay the engines; Ship stages, preflights, reaps.

Playwright specs are discovered once, recursively, under tests/ before the typed Test graph is planned:

• *.server.spec.js is server-only Playwright. It starts a selected non-Static server, waits for its origin, and runs without a Web client or Vite.

• *.fullstack.spec.js and the conventional sync.spec.js require a bound server and Web client, so the graph owns both server and Vite readiness.

• every other *.spec.js is a browser spec for a Web client.

• perf.*.spec.js in any lane is benchmark metadata and is selected by --benchmark; an explicit typed-lane --spec may select it directly.

  Bare project testing skips Playwright targets with no specs in their typed lane. A bound Web target selects browser and full-stack specs, never server-only specs; the bound server is not also scheduled as a standalone server lane. server(Static) never receives server-only Playwright. Targeted ship test --browser, ship test --fullstack, and ship test --server require at least one matching spec. --spec is validated against the selected lane, so a suffix from another lane is a diagnostic rather than an execution-time override. Staging receives exact planned spec and support-file paths and does not rediscover suffix semantics.

Database-Backed Tests

When the compiled test plan requires a database, ship test owns its complete test lifecycle. Provisioning, applying checked-in migrations, dependent test runners, and teardown are nodes in the same workflow graph; ship plan test shows them before anything runs.

The target must be isolated and test-owned. Ship never resets or drops an ordinary development or production database merely because its URL is present in the environment. Destructive cleanup is allowed only for a target Ship provisioned for the run or one whose test configuration explicitly grants Ship ownership. Teardown runs after success, failure, or interruption.

Users do not run ship db test prepare or reset as a prerequisite. Low-level ship db test operations may expose the same lifecycle for inspection, debugging, and cleanup, but bare ship test remains the reproducible entry point.

Per-test versus per-suite lifecycle is selected explicitly through Test.Service.per_test / per_suite; database providers choose rollback, fresh-namespace, or shared-baseline behavior without database-specific compiler rules.

Expect-capture and snapshot identity/promotion details remain open: ../questions/192-test-declaration-and-expect-syntax.md. Language-level expect in test bodies is the v1 assertion.

Test Data

Reusable fixture data is ordinary typed Ship code, not a compiler-known fixture declaration or raw SQL/data file by default. Fixture functions call the same generated database operations as application code, through a transaction the test opens, and execute inside the test's selected per_test / per_suite isolation scope.

let basic_org = fn(tx) {
  let org = AppDb.Orgs.insert(tx, { name = "Acme" });
  let user = AppDb.Users.insert(tx, { email = "teej@example.com" });
  AppDb.Memberships.insert(tx, {
    org_id = org.id;
    user_id = user.id;
    role = :owner;
  });
  { org; user }
};

Test-only module layout, deterministic generated values, and whether demo/dev seeds reuse fixture builders remain open in ../questions/209-test-fixtures-and-seeds.md.

Language-Level Tests

test "slugify lowercases and dashes words" {
  let actual = slugify("Hello Ship");
  #type actual: String;
  expect actual == "hello-ship";
  expect actual != "Hello Ship", "slugify should change the input";
}

suite groups tests and may nest arbitrarily. Individual executable leaves remain test; Ship does not add describe / it aliases.

suite "users" {
  providing AppDb = Test.Service.per_test((_test) => AppDb.test_database());

  test "lists users" {
    // Receives a fresh, schema-initialized SQLite provider for this attempt.
  }

  suite "as admin" {
    test "deletes a user" {
      // Inherits the AppDb provider factory.
    }
  }
}

Suite bodies are declarative: leading providing statements followed by nested suite and test declarations. They do not run arbitrary code while the test tree is collected. A test body may also begin with providing statements before ordinary executable statements. Providers initialize in source order from outermost suite to leaf, may depend only on earlier or outer providers, and drop in reverse order. A nested declaration overrides an inherited provider for the same service; two declarations for one service in the same scope conflict. Filtering selects leaves before any provider is acquired.

Test.Service.per_test and per_suite are ordinary standard-library functions. They take pure factory functions whose Test.Case / Test.Suite arguments are ordinary values and whose result is a named or anonymous service-provider module. The factory constructs a provider; its ordinary init acquires the resource and outcome-aware drop releases it. per_test calls the factory for every selected attempt (including retries); per_suite calls it once for each selected, possibly parameterized suite instance.

providing BrowserHost = Test.Service.per_suite((suite) =>
  Playwright.browser(profile = suite.resource_name("browser"))
);

The manifest gives every leaf a structured identity from its module, suite path, parameter key (when parameterized suites land), and test key. Display names are not identity. This keeps nested tests independently filterable even when different trees render similar prose.

Bodies are Unit-returning statement blocks, sealed (exn = []). expect takes a Bool and an optional String message. Failed expects record and continue; ship test reports every failure. Discovery emits a manifest (stable IDs, spans, tags, services, targets, snapshots). Compiled only in the test profile. ship test --no-run typechecks without executing; --list lists. ship test path.shp, ship test docs/book, or ship test docs runs language tests without a main. Compiler-repository docs coverage is owned by dedicated Cargo tests, not a hidden mode of product ship test.

#display(expr) and #inspect(expr) evaluate expr once, emit its Display or Inspect document, and return the original value. Content tests place the document below the source line; ordinary programs write it to the presentation and diagnostic streams respectively. The hidden Observations test in Formatting executes this. Rendered book pages never show directive syntax: observations appear as their subject with the value beneath it, and #type x: T; reads as x: T;.

#type <expr>: <Type>; makes an exact static assertion inside a test body. The checker infers <expr> without an expected type, normalizes both sides, and compares their typed structure (including alpha-equivalent type variables). The annotation cannot resolve an ambiguous expression and coercible-but-distinct types still fail. A mismatch is a compile-time diagnostic, so both ordinary ship test and ship test --no-run stop before executing the test.

The expression is typechecked but never evaluated. Passing #type assertions are erased during lowering and do not appear as runtime expect events or test report entries. The form is illegal outside a direct test body, including ordinary functions, suites, and nested function bodies. This compiler-only statement does not provide runtime reflection or settle the future design of types as values. Executable examples: book/002-Inference.shp.

Literate chapters may declare YAML frontmatter in leading //! lines:

//! ---
//! fails: true
//! ---

fails: true keeps a failing chapter out of CI: diagnostics still print, and ship test exits 0 when the file actually fails. A chapter that declares fails: true and then passes is a failure. Parse errors stay in cram .t sessions; frontmatter is for typecheck and expect failures in files that still parse.

Pending tests

test(Pending) "Name" { ... } records decided behavior that does not work yet, in place of a prose-only specification. Like program(Web), the parameter is a capitalized name; Pending is the only test parameter. The body only has to balance its braces: a body that does not parse is kept as source plus the first parse error rather than reported as a file diagnostic, so a pending example never breaks its chapter. Checking and lowering skip pending bodies.

Pending is strict. After the ordinary run, ship test promotes each parseable pending test in scope to an ordinary test in a copy of the project, then checks, lowers, and runs it. Every pending test is named with its reason (does not parse yet, does not typecheck yet, does not lower yet, fails) and counted apart from passes and failures: 257 passed, 0 failed, 2 pending. A pending test that passes fails the run with a diagnostic on (Pending); delete it. Candidates are checked together; an error outside every candidate reruns them one at a time, and scopes with no parseable pending test pay nothing. A scope whose ordinary tests do not compile, and a fails: true chapter, report pending tests without running them. --filter selects pending tests like ordinary ones. The site renders each as a test(Pending) card with its reason. The behavior is pinned by crates/ship-cli/tests/golden/run-tests-pending.

The public language site (apps/ship-site) renders standalone and grouped book chapters, the lexicon, and the old root Markdown docs to static HTML. ship test docs still runs the chapter tests; the site app's own test declarations cover layout and links.

A chapter can point at a committed cram transcript without copying it:

//! ::golden examples/003-utility-functions
//! ::golden crates/ship-types/tests/golden/check-block-value failures

The path is the repository-relative case directory that holds the run.t. The chapter parser owns the directive: a malformed citation is a chapter diagnostic, and the content report carries each citation next to the chapter's project references. At generation the host reads each cited run.t with ship_snapshot::parse_transcript, the harness's own reader, and passes typed commands to the site; the site never parses the file or re-runs it.

Each $ command becomes a card in the chapter-test style. The prose paragraph above it is the title, the status reads recorded exit N, and a nonzero command shows one diagnostics block per rendered error or warning. The transcript path appears once above the cards. failures keeps only nonzero commands. A missing transcript, or a citation with nothing to show, fails generation. DEV does not watch run.t files, so restart it after promoting a cited transcript.

A chapter can also render a declaration in place:

//! ::show label
//! ::show typed_label: signature
//! ::show label: infer
//! ::show Vector.map: infer

The path is any value or type name the chapter's tests could write. The typechecker resolves it in the chapter module's test scope with the same lookup an expression uses, so opens, nested modules, and Stdlib paths all work. The mode chooses what renders:

• source (the default): the declaration as written.

• signature: the declaration without its body. For a function, that is everything before the body.

• infer: the inferred type as a contract declaration, formatted exactly as #type prints it.

• full: the declaration as written, followed by its inferred type.

  Types always render their source. An unknown name is a check diagnostic at the directive. So is a request for source when the value has none, such as a compiler builtin; use infer for those. The content report carries each resolved show, and the site renders it as a highlighted code block.

  Static-site workflows cache successful content-test reports per .shp file. The key includes the exact Core/compiler identity, canonical path, and source bytes. A DEV content generation executes only changed chapter files and merges their canonical reports; repeated production/--once generation reuses unchanged reports. A compiler change or clean cache reruns the full set.

Expect And Snapshots

test "invoice snapshot" {
  Expect.snapshot(Fixtures.paid_invoice(), name = "paid-invoice");
}

Captures may include stdout/stderr, inspected values, diagnostics, generated SQL, formatted source. Snapshots respect redaction/opacity metadata.

Cram .t Tests

test/parser/bad-route.t          # file session
test/parser/bad-route.t/         # fixture dir
  run.t
  bad-route.shp

Sandbox + normalize + compare transcript. Compile failures inside .t are expected behavior (parse/typecheck/CLI/migration/formatter negative cases). The harness writes [exit N] only when N is nonzero. That failure does not pass unless the line directly above the command declares it:

  # ship-exit: 1
  $ ship check bad.shp
  error: unknown name `nope`
  [exit 1]

Exit 0 is the default, so # ship-exit: 0 is optional. A mismatch fails the case and is not promoted. The declaration is the only way to change an expected status: edit or remove it, then promote.

# ship-show: names the case files a book ::golden card displays above that command's output. It joins # ship-exit in the lines directly above the command, takes space-separated case-relative paths, and fails the case when a path leaves the case or names no file:

  # ship-show: src/main.shp src/Tasks.shp
  # ship-exit: 1
  $ ship check .

Commands without it show only their output.

Each $ command is killed if it exceeds 120 seconds (override with SHIP_TEST_COMMAND_TIMEOUT_SECS or # ship-test: timeout 30). The process group is signalled so child ship run loops cannot outlive the harness.

Promotion Workflow

just promote is the normal exhaustive entry point for compiler goldens in this repo. Product ship test compiles the project Test graph; it does not discover this compiler repository or run crate goldens.

Language-level expect capture and source promotion remain ../questions/192-test-declaration-and-expect-syntax.md. Until that lands, product ship test has no promotion flag.

just promote runs exact Cargo .snap tests first with SHIP_UPDATE_SNAPSHOTS=1, then cram run.t suites with --auto-promote. Default verification runs both phases with updates disabled.

just promote

# Focused exact-snapshot iteration remains available:
SHIP_UPDATE_SNAPSHOTS=1 just test-crate ship-token --test snapshots

ship-goldens discovers ordinary snapshots test targets by convention; additional target names can be listed in [package.metadata.ship-snapshot].exact-tests. Promotion refuses timed-out, crashed, or nondeterministic cram output. Promotion is not approval. Never hand-edit generated run.t command output. Read every source, .snap, and run.t diff; accept only intentional changes. Snapshots target desired Ship behavior, not temporary gaps.

Compiler watcher

just compiler-watch-status   # start `just compiler-watch` only when none runs
just test-crate ship-types   # wrappers wait for and request what they need

just compiler-watch (ship-dev watch) is the only process that builds the workspace while it runs. It owns .tmp/ship-compiler/cargo-target, publishes generations under .tmp/ship-compiler/toolchain, and builds itself once at startup, so restart it after editing crates/ship-dev.

One generation

1. A change under crates/, ship/, or the Cargo and toolchain configuration starts a generation after a 100 ms quiet window. An agent edit, editor save, or git operation lands as one burst a few milliseconds wide; separate edits arrive seconds apart.

2. The worker hashes the Core and compiler identities (about 20 ms). When the toolchain inputs recorded for the last published generation are unchanged, it republishes that generation by hard link without Cargo, and ship-dev keeps app processes running. The inputs are the workspace files in Cargo's dep-info for the toolchain binaries plus the manifests, lockfile, and toolchain configuration (.ship-toolchain-inputs).

3. When the Core identity changed, Cargo builds ship-core-builder, which assembles Core while a second Cargo stage links ship and the runtime workers; otherwise only the compiler stage runs. The ship/Core pair publishes together, then the source worker.

4. A newer edit that changes the identities interrupts the running Cargo stage with SIGINT and starts over. One that leaves them unchanged lets the stage finish, because the next generation needs the same artifacts.

5. Test images exist only on request. The first scripts/ship-test call for a generation builds the workspace tests and copy-on-write clones exactly the binaries cargo nextest list reports into an immutable image; later tests run it without Cargo.

   A test-image failure never retracts a published compiler, a compiler failure leaves the last good pair in place, and the watcher never runs or promotes tests. The published ship is a launcher that execs the main driver or the smaller fmt/lex/parse worker; each worker keeps its own last good generation.

What an edit costs

Edit to published compiler, just compiler-bench --samples 3 (one warmup), 16 cores, October 2026:

Edit	p50	Critical path
Test, doc, or other non-toolchain file	0.13 s	hard-link republish; test image 0.76 s
ship-cli, ship-js, ship-native-server private function	0.9–1.1 s	compiler stage
ship-js new public function	1.7 s	compiler stage
ship/stdlib source	1.5 s	Core assembly
ship-types or ship-lower private function	2.4–2.6 s	Core builder 1.0 s, Core assembly 1.3 s
ship-lower new public function	4.3 s	dependents re-check
ship-parse private or public function	3.8–4.3 s	every dependent rebuilds
ship-types new item reachable outside the crate	5.0 s	ship-types re-codegens most of itself

Three things set the remaining cost:

• Core assembly takes about 1.3 s, 0.9 s of it type-checking the stdlib, and gates every edit inside the ship-stdlib-generator closure. That closure includes ship-interpreter, ship-gc, and ship-fmt through ship-lower and ship-types. Faster checking or a narrower closure shortens all of them.

• A new item reachable from outside ship-types re-codegens most of the crate (about 3.8 s); a private item or one in a private module costs about 1 s.

• Rustc compiles an inherent method in the codegen unit of its Self type's module, wherever the method is written. Large crates keep checker logic in module-level functions (fn check_x(infer: &mut Infer, ...), called as module::check_x(self, ...)) so an edit re-optimizes only that module's unit.

Settings and their evidence

Each setting below was measured with real edits on this workspace; change one only with new measurements.

• opt-level = 1, dependencies at 3. Opt-level 0 rebuilt 15–25% slower (larger code to link) and made Core assembly five times slower (6.3 s against 1.3 s). Opt-level 2 rebuilt about 6% slower and assembled about 10% faster: a wash for Core edits and a loss for the rest.

• debug = false. Line tables cost 8–11% (about 100 ms) on every rebuild. Panics still report their source location. [profile.debugging] carries full debug info at opt-level 0, and just flamegraph-ship builds a release profile with symbols; just profile-command on the published compiler shows functions without inlined frames.

• lto = "off". Crate-local ThinLTO re-optimizes the whole crate after any edit (a ship-types leaf edit took 10 s instead of 4 s) for about 7% faster compiler runtime.

• Ten jobs on sixteen cores, bundled LLD. Rebuilds follow the crate dependency chain: all sixteen cores at nice 19, mold, and split-debuginfo = "unpacked" each measured within noise.

• SIGINT cancellation. 51 interrupts at random points 150 ms–7 s into toolchain builds all rebuilt, linked, and ran cleanly. Cargo and rustc treat SIGINT as an interactive cancel; builds are never sent SIGTERM. Three edits a second apart publish as fast after the last edit as one edit does.

• Separate Cargo stages. The compiler stage already overlaps Core assembly, so one streaming Cargo invocation would save under 100 ms off the critical path.

• Whole-workspace test images. A package-scoped build changes Cargo feature unification and recompiled shared dependencies for 14–43 s; the full image takes about 3 s after a compiler edit.

• Nextest. ship-cli's driver changes the working directory and environment, which needs process-per-test isolation, and nextest ran the ship-types suite in 1.2 s against libtest's 2.6 s.

• One integration test binary per crate with several integration test files (autotests = false plus tests/integration.rs declaring each file as a module) halved the image (4.1 GB to 2.4 GB) and its link work. Add a new file there as mod name;; ship-cli's layout test fails when a tests/*.rs file is neither a declared target nor a module of one.

Working beside the watcher

Keep one watcher; check existing terminals before starting another. just compiler-watch-status reports the generation without invoking Cargo. Run scripts/ship-compiler wait only when the very next command needs that exact generation; test and golden wrappers request it themselves. Never run direct cargo build, cargo run, or cargo test beside the watcher, and stop it before recipes that start Cargo themselves, such as just gc-bench and just flamegraph-ship. Do not preface ship build, ship run, or ship test with ship check. Rust doctests are not part of the suite; examples that must compile belong in the owning crate's tests. Generated .tmp, build, dist, target, coverage, and dependency directories are not watched.

Benchmarking the watcher

just compiler-bench --samples 5 --edit item --path crates/ship-types/src/apply.rs
just compiler-bench --samples 5 --edit pub-item --path crates/ship-lower/src/lib.rs \
  --baseline latest --max-regression-percent 10

--edit item (the default) appends a private function and pub-item a new public one; comment changes bytes without changing any item and mtime only touches the file, so neither supports a build-time claim. Add --wait-for-test-image when the claim includes image preparation and --cargo-timings for Cargo's unit timeline of the Core builder and compiler stages under .tmp/ship-compiler/cargo-target/cargo-timings/. The benchmark holds an exclusive lock, excludes one warmup, and writes reports under .tmp/ship-compiler/benchmarks/ (latest.json updates atomically); test wrappers wait for it. Do not edit or run broad suites while it measures. Report sample count and spread, not only p50, with the generation and identities from the report. just profile-command and just flamegraph-ship profile the compiler's own runtime, not Cargo scheduling.

Running tests

Run the smallest proving command and let its wrapper wait for the generation it needs. Focused golden filters use the same image as promote; prefer the full case directory name:

# Verify without writes:
just goldens run-error-channels
just goldens check-behavior-modules

# Promote cram output:
just promote run-error-channels

just test-crate waits for the watcher's current generation, requests its exact test image if absent, then runs the requested Nextest package/test arguments against that image. Custom harness = false suites implement Nextest's one-test protocol, so whole suites run normally there. Filtered verification and promotion pass their intent through the image-backed golden runner, which schedules the same prebuilt exact and cram harness binaries through Nextest. This keeps crates that are themselves dependencies of Core assembly from adding a generator build-dependency cycle.

Filters match the case directory name relative to tests/golden/. If any case name equals the filter (or ends with /{filter}), only those exact cases run. Otherwise the filter is a substring (fmt still selects fmt-stdin). Exact names win over prefix collisions: check-types-error does not select check-types-errors.

The first-party ship-snapshot crate owns both repository snapshot mechanisms. Library-rendered text that does not exercise a CLI or workflow uses an ordinary Cargo test:

#[test]
fn parse_basic() {
    let rendered = render_fixture("tests/fixtures/parse-basic.shp");
    ship_snapshot::assert_snapshot!("parse-basic", rendered);
}

For command-shaped library output, keep status and both streams explicit:

ship_snapshot::assert_output_snapshot!(
    "parse-error",
    status: output.status,
    stdout: &output.stdout,
    stderr: &output.stderr,
);

This writes stdout, then stderr, then [exit N]\n only when N is nonzero. It does not insert, trim, normalize, or redact stream bytes or newlines.

The logical name resolves to raw UTF-8 at tests/snapshots/parse-basic.snap in the calling crate. Comparison is exact, including newline presence. Missing and mismatched snapshots fail with the stable relative path and a focused diff. Tests never write unless SHIP_UPDATE_SNAPSHOTS=1 is set; promotion atomically replaces each changed file and prints every updated path:

SHIP_UPDATE_SNAPSHOTS=1 just test-crate ship-token --test snapshots

Keep source inputs under tests/fixtures/. The same crate's cram harness owns run.t; use it only when command, process, CLI, or multi-step workflow behavior is part of the contract.

Still run broader promotion before treating a large compiler change as finished.

Snapshot Intent

Write fixtures for the behavior Ship wants. Commit today's actual diagnostic when unimplemented, but do not claim "not supported yet" is the behavior under test.

Authored run.t prose (lines before a $ command) is the claim. Generated command output is the evidence. You may edit the claim; never hand-rewrite the evidence — promote it.

Three kinds of [exit 1] goldens:

Kind	What it is	What to do
Negative	Lasting user-facing error (type mismatch, unknown name, illegal shape). The diagnostic text is the contract.	Promote wording fixes. Keep the failure.
Stub	Planned feature not landed. The diagnostic says so and cites Linear / a question (not implemented yet, not checked yet).	Leave until the feature ships, then delete its # ship-exit line and promote.
Hole	The fixture meant to prove X, but the snapshot is a compiler/runtime bug (wrong type, crash, runtime error occurred here, follow-on nonsense after the real error).	Fix the compiler, then promote. Do not treat the hole output as desired Ship behavior.

Mark holes and stubs in the authored preamble so they grep:

HOLE: `Feed.Event` vs `Event` — not the locality check this fixture intends.
STUB: non-local `return` from a `with` body waits on SHIP-31.

rg -n '^(HOLE|STUB):' crates/*/tests/golden

Useful hole signatures in run.t (not all are holes — read the preamble): runtime error occurred here, ship defect on a path that should be a typed error, not implemented yet without a Linear/question citation, type mismatches that contradict the fixture's claim (expected Event, found Feed.Event).

Follow-on expected named fields (), found 'a' after a real unknown name is diagnostic noise. Do not treat those extra errors as the behavior under test.

When a feature is intentionally unimplemented, write a stub (not implemented yet + citation), not a hole that snapshots the wrong diagnostic. Language behavior that fits a book example is a test(Pending) in its chapter instead; stubs are for what a test body cannot show.

Formatter And Repo Checks

ship fmt: one style with two deterministic layout budgets. A structurally breakable compound value stays compact through 60 characters; nested compounds make that decision recursively. The complete source line has a 120-character smart target, or the nearest ship.project's format_width. Exceeding either budget uses the form's hanging layout. Atomic fragments (comments, raw SQL/browser blocks, strings) may exceed both because the formatter cannot split them safely.

A pipeline with one |> follows those budgets. Two or more |> operators always expand one step per line.

Record-like arrow bodies always begin below =>, even when their compact form fits 60 characters. Any other multiline arrow body also moves below => as a unit. Default just ship-fmt uses the published compiler and walks from ., formatting every walkable .shp / .shc file. Files whose first non-blank line is // fmt: ignore are skipped, including deliberate parse-error fixtures. Unparseable files without that comment fail the run.

just fmt          # cargo fmt --all + ship fmt
just fmt-check
just check

CLI Surface

ship test
ship test parser
ship test --filter "signup" --retries 2
ship test --no-run | --list
# later: language-level promote (question 192), --json, --target interp | client/… | server/…

Services And Determinism

Same capability model as production:

suite "signup" {
  providing Email = Test.Service.per_test((_test) =>
    module: Service.Provider(Email) #{
      let service Provider: Email = RecordingEmail;
    }
  );

  test "sends welcome email" {
    signup(Fixtures.basic_org().admin_email);
    Expect.snapshot(RecordingEmail.sent(), name = "welcome-email");
  }
}

Tests install deterministic providers such as a fixed Clock, seeded Random, or captured Console explicitly through the same provider system. Higher-authority services (Postgres, FS, network, email, queues, secrets, browser, …) stay explicit too. Runner normalizes paths, line endings, seeds, timestamps, non-semantic map order, and redacted secrets.

Performance Benches

End-to-end Ship language/runtime workloads live under benches/<name>/ (a normal Ship project with bin/). just bench discovers and runs every such project, prints /usr/bin/time -p wall/user/sys, and writes the same timing lines under .tmp/benches/. Pass args through to each workload (just bench 3 6).

Private Rust API or crate-pipeline microbenchmarks stay as [[bench]] / crate binaries in the owning crate (cargo bench -p …, just gc-bench), with the compiler watcher stopped before direct Cargo. App fullstack perf stays with the app (e.g. scrum just perf). Do not invent placeholder benches to fill the tree.

Later Modes

Property/fuzz/bench/coverage/differential share the same runner and result format when ready — do not block the first implementation. Fuzz failures promote minimized corpus files with the same review mindset.

Core Toolchain / Provider Gates

Beside ordinary goldens; widen just check-toolchain / CI as the pipeline hardens:

Gate	Intent
PathOwner	One owner per callable path; reject collisions
Provider ABI	Validate native/JS claims beyond ID membership
Differential	Shared vectors across native / JS / comptime pure subset
Whole-stdlib check	RuntimeSource pairs typecheck without permanent allowlists
Toolchain identity	Automatic build output matches current compiler + normalized Core source
Installation failure	Missing/mismatched external artifact fails before app checking
Profiles	Decode/remap/install cost regressions

Distinct from app incremental caches (12-future-tooling.md). Provider ABI open in ../questions/237-provider-semantic-abi.md.