On this page · 11 sections

Runtime And Implementation

How Ship runs programs. Rust owns the implementation permanently — compiler, runtime, interpreter, tooling, and any future JIT/AOT. No self-hosting; Rust is never an emitted Ship target.

Decisions

Handwritten Rust parser (lexer + recursive descent + Pratt). Prefer no third-party deps; small focused crates.
Convention-first layout (src/, bin/, client/<App>/init.shp, server/<Name>/init.shp). Optional manifest deferred — ../questions/230-build-manifest-name-and-shape.md.
ship check --shape shares .shc analysis with full check; shape stops before body inference.
ship run = Rust-owned lowered-IR runtime/interpreter (native semantic reference). JavaScript emission exists only for Browser/Android programs. Servers always run native code. Node is external tooling, never a Ship target.
Supported compiler and development hosts are Linux (primary) and macOS only. Host tooling is Unix-only and rejects unsupported hosts at compile time.
Boundary goldens + easy promotion; no internal Rust-struct snapshot churn. Product ship test runs the project Test graph. Compiler run.t / .snap suites use just promote — 07-testing.md. Language-level expect promotion remains question 192.

ship lex path/main.shp
ship parse path/main.shp
ship check --shape
ship test
just promote

Pipeline

.shp
  -> source database + spans
  -> lex / parse
  -> check
  -> lower
  -> lowered program + runtime metadata
     -> native-rust runtime/interpreter
     -> js-browser/android

Unsupported lowered forms → target diagnostics, not silent fallback. Incremental builds: 12-future-tooling.md.

A module value is a record of operations. A functor value is a function: module record(s) in, module record out (OCaml lambda/translmod.ml). module Scores = MakeLookup(ScoreStore) lowers to a call, not a cloned copy of the functor body. Result types are hash-consed Apply paths; identity is never an interned "UserCache.Entry" global. Flattening operations into dotted globals is a lookup optimization, not the identity model.

Source-Owned Builtins And Stdlib Artifact

#builtin(...) on .shc value headers is the only source of provider path, signature, ID, runtime availability, and comptime policy. No parallel Rust catalog; no contract line-rewriting generator.

Shape	Authored in	Backend registry
`#builtin` provider	`.shc` directive	Native/JS claim by `id`
Source-backed wrapper	Ordinary `.shp` body	None (lowered Ship IR)
Compiler-private op	`BuiltinId` in the native registry	Not path-installed; wired by canonical modules
`#builtin.magic(value)`	Explicit `.shp` expression	Erased; same runtime value

PathOwner: every callable path has exactly one owner — source-backed operation, #builtin provider, type-only declaration, or service operation. Backend optimizations must not install an alternate binding. Primitive and structural behavior implementations are checked canonical Ship modules, just like application behavior modules. Target-owned leaf operations use their caller-oriented .shc path and provider ID. Do not invent an Intrinsics / __ship.private.* / __intrinsic.* module or keep public-shaped primitive_* twins. Prefer an honest .shp body when the operation is scaling, compare, or a thin constructor over remaining host ops; keep #builtin on the public module when Ship would lie (representation bridges, host grammars, :overflow arithmetic). Services are not collapsed into #builtin.

#builtin.magic is the one explicit representation cast. Checking assigns its result the expected type (or a fresh unconstrained type); lowering erases the cast and keeps the operand. It is appropriate only when a low-level abstraction owns representation identity. It never performs semantic conversion and must not become a workaround for missing checked behavior or providers.

Provider ABI (arity, labels, defaults, exn/eff channels, runtime set, comptime policy) validates native/JS claims beyond ID membership — ../questions/237-provider-semantic-abi.md.

Source eff and provider operational facts stay distinct:

Axis	Owner	Meaning
Source `eff` / `pure`	`.shc` Call channel	Provide-discharged service/control/command requirements
`comptime = Pure \	Effectful`	`#builtin` → `ProviderAbi`	Whether the provider may run under pure comptime policy
Backend claims	Separate claim descriptors	Allocation, blocking, re-entry — never inferred from `pure`

Free host builtins that perform I/O or host calls but invent no service requirement seal with pure (empty source eff) and comptime = Effectful (services). Do not overload pure to mean "no host I/O".

Core compiles into one external toolchain artifact (checked modules, logical sources, runtime/comptime and contextual-code IR, provider facts). The sectioned format lives in ship-stdlib; the generator/checker/lowerer stage produces lib/ship/core-<toolchain-id>.bin and its immutable manifest beside the compiler toolchain. toolchain-id is a cheap deterministic digest of the generator's Cargo-derived normal/build workspace dependency closure, workspace resolution/toolchain inputs, and normalized ship/stdlib source. Computing it never parses or checks Ship. The watcher performs one semantic parse/check/ generate/lower/encode assembly per identity, then shares those exact bytes with debug, release, and demand-driven test images. The identity is carried inside the artifact and manifest, so a worker accepts only its generation-selected match.

The compiler watcher owns assembly and gates compiler publication on matching successful Core. ship-cli has no custom build script or generator dependency. Installed toolchains use <root>/bin/ship + <root>/lib/ship; the launcher selects an immutable manifest through SHIP_CORE_MANIFEST.

Product check/build/run opens that external file lazily and never walks or checks ship/stdlib. Missing, corrupt, or mismatched files are toolchain installation errors before application loading. There is no embedded fallback, checkout freshness probe, manual artifact promotion, or cross-version ABI.

Bootstrap is explicit and narrow: ship-core-identity computes source identity and validates manifests without compiler dependencies; ship-stdlib owns the byte format/decoder and compiles with no artifact bytes; the checker/lowerer- backed ship-stdlib-generator is built only for the watcher-owned Core stage. Release installation copies the launcher, generation workers, generation Core marker, immutable manifest, and exact artifact into <root>/bin and <root>/lib/ship.

Syntax.parse and Syntax.format_item are Native-only #builtins (syntax_parse, syntax_format_item). They materialize generated Syntax.* values; walkers are ordinary generated Ship, not builtins. The public site generator is a native bin/. See 01-syntax.md.

Native Value Representation

Tagged 64-bit Value: 63-bit Int, Bool, Unit, hole sentinel as immediates; else heap pointer. This is the native runtime representation, not a cross-target width promise. Keep the one-word native representation sacred.

Current ship-gc uses boxed Rust payload nodes (semantic experiments). Serious runtime target:

Value word → immediate | heap pointer
heap object → compact header + dense payload
LayoutDesc (hot, ≤16B) → size / pointer slots / no-scan / drop / pin
TypeDesc (cold) → names / fields / wire / redaction / schemas

Hot GC uses LayoutDesc, not reflection. Nominal product identity is stamped at construction — do not identify stdlib values by field shape. Detail: ../questions/173-value-representation-and-gc-layout.md.

GC And Barriers

Decided: precise, single-domain, generational tracing collector (OCaml/OxCaml-shaped). Rejected as serious native targets: conservative GC, reference counting, Go's non-generational baseline as the model.

Bump nursery → promote to major heap; exact scan descriptors; explicit roots; remembered sets + write barriers for old→young.
Every allocation may collect → every live Value in a mutable root slot.
Frames: flat contiguous Value slots. Mutations go through barrier-aware APIs.
Remembered entries are slot-indexed (stale slots ok; live old→young must be remembered).

Hot-layout budgets: Value = 8B; headers prefer 8B; LayoutDesc ≤16B; remembered entries ≤16B; dense Value payloads; no-scan for strings/bytes. The interpreter keeps String/Bytes (and their slices) as views over a shared no-scan ByteBacking; of_slice copies to drop the parent. JavaScript uses host strings / Uint8Array and copies at the String/Bytes edge. String semantics: 23-string-and-bytes.md.

Still short of the target nursery (boxed ObjNode, most payloads still Trace, symbols in hot payloads, etc.) — track in implementation work, not here.

Numeric Representation

Int: the target's ordinary efficient integer. Native uses a tagged 63-bit signed immediate; overflow past that representation is a runtime defect. JavaScript uses Number, including its normal large-integer rounding behavior. Ship does not add safe-range checks or representation machinery to force extreme values to agree across targets.
Float: IEEE f64; boxed on native and a JavaScript Number on JS.
Portability: ordinary application-sized integer operations should behave alike, but exact width and extreme-range parity are not promises of Int. Code that genuinely needs a fixed-width or wider integer can use a dedicated type once that product need exists.
UInt64: currently a typechecker alias of Int, not an exact-width promise. A future exact-width type would be separate from ordinary Int.

Comptime Execution

Same interpreter under an explicit policy — never a parallel constant evaluator. Clients: deriver generate, ensure of comptime-evaluable values, #insert, future #run. Policy decides denied effects, fuel, loaded modules.

Evaluability-by-trying may degrade to runtime; #insert requires comptime Code(T, 'scope) — no runtime insertion fallback. A blocked evaluation names its ComptimeBlockReason: a masked runtime local, a service call, an impure builtin, fuel, or a name with no compile-time value. Only the masked local means the input is runtime data. Modules check in dependency order; each newly checked module lowers into the comptime session, and a module's own types (constructors with their validation schemas) enter before its values check. Mutual recursion groups check in path order (back-refs stay dynamic). Entry is the dependency sink.

Compiler Synthesis

Before check: enum-first TypeDesc facts + approved module derive generators emit behavior modules (never source strings) that stay in memory through check, lower, and emit. Only ship check --generated formats them and writes .ship/generated/ for review. The deriver session boots on the first derive request and Core's comptime program decodes on the first evaluation, so projects that never need either pay for neither. The public synthesis policy is specified by the executable Behavior Synthesis chapter. database blocks get table helpers + codecs. api declarations also emit protocol-neutral route facts for checked clients, links, forms, tests, and response adapters; frontend integrations consume those facts instead of reconstructing route strings.

Contextual quotation (settled):

#code(expr) → opaque hygienic Code(T, 'scope) (no execute)
Transformers run under pure comptime policy
#insert splices; checker validates complete result + obligations
Erased before native/JS runtime IR

Code = expression IR; Meta.* = declaration/descriptor surface. Neither formats-and-reparses. Open: user derive plugins, module quotation, #run, runtime reflection — questions 215, 172.

Native Servers And HTTP

Every server(...) profile executes native Ship. server(Native) compiles to one checked manifest (routes, derived database handlers, Sync facts, and the checked handler IR program) served by ship-native-server; it never contains generated Rust source. server(Static) uses the Rust static adapter. There is no JavaScript server artifact or runtime profile.

HTTP is split by ownership:

ship-http: dependency-free sans-I/O HTTP/1.0 and HTTP/1.1 syntax, framing, serialization, pipelining, informational responses, trailers, and upgrade boundaries.
ship-net: dependency-free epoll/kqueue readiness, timers, wakeups, cancellation, and bounded handoff.
ship-http-server: bounded admission, worker scheduling, ordered responses, streaming backpressure, phase deadlines, and graceful drain.
ship-http-client: origin pooling, redirects, structured phase errors, streamed responses, DNS, and direct Rustls TLS.

Protocol code owns no JSON, routing, Datastar, SSE event semantics, or static files. Native server adapters consume checked HttpRouteSemantics and own those application representations. Header values remain bytes through the transport; no lossy UTF-8 conversion occurs.

Browser Targets

Convention, not source annotations:

client/Storefront/init.shp   # browser
server/Api/init.shp          # native server

Peers of src//bin/. Shared code in src/; database only in src/. Browser-only capabilities → diagnostics on native, not polyfills. Database effects in browser programs are check-time errors; browser bundles carry only throwing stubs for unreachable src database operations.

JS output is Ship-owned emit — not a public JS API, not TS source-of-truth, not generic npm FFI. Packaging tools (Vite, etc.) are machinery only.

Browser builds emit generated Vite packages. Servers opt browser programs into their artifact with bind Client.<App>; and an optional route = "/app/". Typed workflow adapters install and run those packages for dev, run, and test (25-cli-workflows.md).

Meaningful Ship-span source maps, generated-frame policy, and composed browser stack traces remain open in ../questions/159-js-artifacts-debug-and-bundling.md.

Platform And Agent Policy

Native execution = hand-written Rust over lowered facts, not generated Rust. Browser-only capabilities do not need native equivalents.

The compiler, CLI, development supervisor, runner adapters, TUI, and test harness support Linux and macOS hosts only. They share Unix signals, permissions, executable lookup, and process groups so supervised commands and their descendants stop consistently on both hosts. Emitted targets such as browser and Android are separate from this host support matrix. Unsupported hosts fail compilation; do not accumulate fallback process behavior or platform-specific compatibility branches.

While this stands, do not:

recreate generated-Rust / separate VM backends
reintroduce Rust-first builtin catalogs or contract rewriters
install a second owner for a public path
add third-party Rust dependencies without concrete repeated pain and a narrow ownership boundary
emit Rust, LLVM, Cranelift, conservative GC, RC, or a general bytecode VM as the default native path

Do implement Rust source, diagnostics, parse, CLI, and test-harness work when compiler work is requested.