ADR-0002: Snix Integration via Service Boundaries

Status: SUPERSEDED — wholesale by ADR-0006 §3 (2026-07-07): §Tier 3 was superseded by ADR-0005; the remaining tiers existed as the optional passthrough-snix executor's spec, and that executor is removed. Retained as historical record only.
Date: 2026-06-05
Deciders: nrd
Source: Eos Build Scheduler Specification | Eos Network Protocol
Supersedes: ADR-0002 (2026-06-03 draft, "Decoupling Snix Backend from Eos Core and Scheduler")
Related: ADR-0003, ADR-0004, ADR-0005

Document Classification: Architecture Decision Record Audience: Architects, Core Developers

Context¶

Snix is a microservice architecture. Its store services (BlobService, DirectoryService, PathInfoService), build service (BuildService), and evaluator (snix-eval) are designed as independently deployable components connected by gRPC contracts. The protobuf definitions are MIT-licensed specifically to encourage third-party clients.

Our current integration inverts this architecture. We embed snix's store services, build services, and evaluator as linked Rust crates, creating a monolith where snix designed composable network services. This manifests as five concrete coupling violations:

eos-daemon/src/main.rs directly calls snix_store::utils::construct_services() to create in-process store service instances.
eos-daemon/src/scheduler.rs is parameterized on the concrete SnixEngine type, reaching into its public fields (engine.blob_service.clone(), engine.directory_service.clone()) to construct SnixIngestService and CastoreBridge.
eos/src/bridge.rs (CastoreBridge) directly holds Arc<dyn BlobService>, Arc<dyn DirectoryService>, Arc<dyn PathInfoService> — snix-specific trait objects that couple the orchestrator to snix's storage layer.
eos/src/fetch.rs imports nix_compat::nixbase32 for hash decoding.
eos-snix embeds all six snix crates in-process, including store and build services that have dedicated gRPC interfaces.

The prior draft of this ADR (2026-06-03) correctly identified the scheduler → SnixEngine coupling but misdiagnosed the root cause. It proposed a Worker trait abstraction that decouples the scheduler from a specific engine, but if the LocalWorker still embeds the full snix stack in-process, the result is indirection over the same monolith. The core question is not "how does the scheduler talk to the engine" but rather: why are we embedding snix runtime code at all, when gRPC clients would suffice for stores and builders?

Snix's gRPC Service Catalog¶

Snix defines three gRPC service domains:

Service	Proto Package	Remotely Accessible	Protocol
`BlobService`	`snix.castore.v1`	✅	Stat, streaming Read/Put
`DirectoryService`	`snix.castore.v1`	✅	Streaming Get/Put
`PathInfoService`	`snix.store.v1`	✅	Get, Put, List, CalculateNAR
`BuildService`	`snix.build.v1`	✅	Unary DoBuild
Evaluator	—	❌	In-process library only (by design)

The evaluator is intentionally not a network service. It is a language interpreter that calls out to stores and builders. However, with atom encapsulation, Eos can wrap evaluation as a schedulable network service — something the Nix ecosystem has never achieved because flakes lack true encapsulation.

Architectural Forces¶

Snix is a service, not a library. Its store and build components are designed for remote access. Embedding them as linked crates bypasses the service boundaries snix itself maintains.
Eos must schedule across multiple snix instances. The daemon is a scheduler, not a frontend to a single snix installation. Scheduling requires network interfaces.
Atom encapsulation enables remote evaluation. Unlike flakes (which copy entire repos), atoms are self-contained units. An eval worker can receive an atom-id, fetch the encapsulated content, and evaluate it against a remote store. No ambient repo context needed.
Single Responsibility. The scheduler coordinates task distribution, deduplicates concurrent requests, tracks leases, and manages worker health. It must not execute builds, manage store paths, or run evaluations.

Decision¶

We integrate with snix primarily through its gRPC service interfaces rather than linking its runtime code. The integration is organized into four tiers, each with a distinct rationale and coupling budget.

(Note: Following ADR-0005, Tiers 1, 2, and 4 below are not superseded — they hold, re-scoped as the spec for the optional passthrough-snix executor, and ADR-0005's wire-first GPL-seam decision widens rather than reverses the seam they establish here. Tier 3 is superseded wholesale; see the note at its heading.)

Tier 1: Store Access via gRPC¶

The three store services (BlobService, DirectoryService, PathInfoService) SHALL be accessed as remote gRPC services. The eos daemon connects to one or more snix store daemons via gRPC URIs (e.g., grpc+http://[::1]:8301).

Current state: eos-daemon/src/main.rs calls snix_store::utils::construct_services() to create in-process store instances.

Target state: eos-daemon takes store URIs as configuration and connects via gRPC clients. The snix store daemon runs as an independent process.

Rationale: Snix's store services already have full gRPC servers with multiple backend implementations (memory, redb, Bigtable, S3, gRPC proxy). The gRPC clients implement the same Rust traits (BlobService, DirectoryService, PathInfoService), so the eval worker's SnixStoreIO works identically with remote stores.

Coupling eliminated: snix-store and snix-castore runtime dependencies removed from eos-daemon and eos.

Tier 2: Build Dispatch via gRPC with Cap'n Proto Shim¶

Build execution SHALL be dispatched to snix builder instances via the snix.build.v1.BuildService gRPC protocol, wrapped in a thin Cap'n Proto shim that adds scheduling-layer concerns.

The snix build protocol is unary (DoBuild(BuildRequest) → Build) and lacks progress streaming, job cancellation, and lease tracking. These are Eos scheduling concerns. A thin build worker shim bridges the protocols:

┌──────────────┐  Cap'n Proto   ┌───────────────────┐  gRPC   ┌───────────────┐
│ Eos Scheduler │ ────────────→ │ Build Worker Shim  │ ──────→ │ snix Builder  │
│              │ ← progress,   │ (cancel, progress, │         │ (OCI/bwrap)   │
│              │   cancel       │  lease, dedup)     │         │               │
└──────────────┘                └───────────────────┘         └───────────────┘

The shim:

Implements the Eos BuildWorker Cap'n Proto interface
Translates BuildRequest to snix's gRPC format
Wraps the unary gRPC call with a CancellationToken for abort semantics
Reports state transitions (queued → building → done) as ProgressStream updates
Heartbeats lease renewals back to the scheduler

Coupling eliminated: snix-build runtime dependency removed from eos-daemon. The shim is a standalone binary that depends on snix-build for protobuf types only.

Tier 3: Evaluation via Remote Eval Workers¶

(Note: Superseded wholesale by ADR-0005 §Supersede ADR-0002 §Tier 3 — the MVP has no evaluation stage to schedule workers for; this entire tier, including its Cap'n Proto EvalWorker interface and worker pool, is dead scope. Tiers 1, 2, and 4 below are unaffected and survive scoped to the optional passthrough-snix executor.)

Evaluation SHALL be schedulable across a pool of remote eval workers, enabled by atom encapsulation. This is the key architectural differentiator from the Nix ecosystem.

An eval worker:

Receives an EvalRequest over Cap'n Proto (atom-id, label, expression entry point, store URIs, pre-resolved inputs)
Connects to a snix store daemon via gRPC (using the provided store URIs)
Runs snix-eval + snix-glue in-process on a dedicated OS thread (required by the !Send constraint)
Writes produced derivations back to the store via gRPC
Returns derivation digests to the scheduler

The existing SandboxedEvalConfig already takes store service addresses as strings. The eval worker connects to snix store daemons via gRPC URIs, which support multiple backend implementations. Pointing eval workers at remote stores requires zero code changes to the evaluation path itself.

Pure Evaluation and Sandboxing¶

Eval workers run snix in pure evaluation mode. Pure eval confines the evaluator to the atom's encapsulation boundary — it cannot import code or data external to the atom being evaluated. Content-addressed fetches (where a hash is pre-declared) are permitted because they are safe by construction. This language-level confinement eliminates the need for OS-level process sandboxing (Bubblewrap, Birdcage) during evaluation, significantly reducing operational complexity.

Cap'n Proto Interfaces¶

interface EvalWorker {
  evaluate @0 (request :EvalRequest) -> (result :EvalResult);
  cancel @1 (jobId :Data) -> ();
  status @2 (jobId :Data) -> (state :EvalStatus);
}

interface BuildWorker {
  build @0 (request :WorkerBuildRequest) -> (result :WorkerBuildResult);
  cancel @1 (jobId :Data) -> ();
  status @2 (jobId :Data) -> (state :BuildStatus);
  attachProgress @3 (jobId :Data, callback :ProgressStream) -> ();
}

Eval workers and build workers are separate services. They have distinct resource profiles (eval is memory-bound and thread-per-eval; builds are CPU/disk-bound and semaphore-gated), distinct security profiles (eval needs atom content access; builds need sandbox capabilities), and distinct scheduling policies (no work-stealing for eval; work-stealing for builds). Operators who want both on a single machine simply run both binaries.

Coupling budget: eos-snix retains snix-eval, snix-glue, and nix-compat as in-process dependencies. These are required — there is no gRPC interface for evaluation, and this is by design. All store and build access from within the eval worker goes through gRPC.

Backend-Agnostic Shim Topology¶

The gRPC transport between worker shims and their downstream backends (snix store daemons, snix builders) is specific to the snix integration. It is NOT an architectural requirement imposed by Eos on all backends.

The only universal contract is the Cap'n Proto EvalWorker and BuildWorker interfaces that face the scheduler. How a shim communicates with its backend is an implementation detail of that shim. Alternative backends have different transport characteristics:

Legacy Nix: The Nix daemon protocol is designed for local Unix domain socket communication and is prohibitively inefficient over the network. A Nix build worker shim SHOULD be co-located on the same host as its Nix daemon, communicating via /nix/var/nix/daemon-socket/socket or direct process invocation. A Nix eval worker shim would similarly invoke nix-instantiate (or link libnixexpr) as a local process.
Guix: A Guix shim would translate ATerm derivations to Guix's S-expression format and dispatch builds to a co-located Guix daemon.
Future backends: Any system that can accept a build specification and produce content-addressed outputs can be wrapped in a shim. The shim's internal transport (gRPC, Unix socket, pipe, FFI) is unconstrained.

Operators deploying non-snix backends should expect shim and backend to run as a single logical unit on the same host. The snix case — where gRPC enables optional separation of shim and builder onto different machines — is an affordance of snix's service-oriented architecture, not a design requirement of Eos.

Tier 4: Type Dependencies (Compile-Time Only)¶

nix-compat provides protocol-level data structures (Derivation, StorePath, NixHash, nixbase32). These are compile-time type definitions, not runtime service dependencies. They are analogous to protobuf-generated types — necessary for serialization but not coupling to a runtime.

nix-compat is an acceptable dependency in eos-snix and in the build worker shim (for Derivation → BuildRequest conversion). It SHALL NOT appear in eos-core, eos, or eos-daemon.

Architecture Diagram¶

                    ┌─────────────────────┐
                    │    Eos Scheduler     │
                    │   (eos-daemon)       │
                    │  Cap'n Proto RPC     │
                    └──────────┬──────────┘
                               │
              ┌────────────────┼────────────────┐
              │                │                │
              ▼                ▼                ▼
    ┌─────────────────┐ ┌──────────────┐ ┌──────────────────┐
    │  Eval Worker    │ │ Eval Worker  │ │ Build Worker     │
    │  (eos-snix)     │ │ (eos-snix)   │ │ Shim             │
    │  snix-eval +    │ │              │ │ Cap'n Proto →    │
    │  snix-glue      │ │              │ │ snix gRPC        │
    └────────┬────────┘ └──────┬───────┘ └────────┬─────────┘
             │                 │                  │
             │    gRPC         │     gRPC         │  gRPC
             ▼                 ▼                  ▼
    ┌─────────────────────────────────────────────────────────┐
    │                  snix Store Daemon(s)                   │
    │  BlobService  │  DirectoryService  │  PathInfoService   │
    └─────────────────────────────────────────────────────────┘
                               │
                          ┌────┴────┐
                          │ snix    │
                          │ Builder │
                          │ (OCI/   │
                          │  bwrap) │
                          └─────────┘

Dependency Delta¶

Eos Crate	Current snix deps	Target snix deps	Change
`eos-core`	0	0	—
`eos-proto`	0	0	—
`eos-snix` (eval worker)	6	3 (snix-eval, snix-glue, nix-compat)	−3
`eos` (orchestrator)	3	0	−3
`eos-daemon` (scheduler)	3 + eos-snix	0	−3 − eos-snix
build-worker-shim (new)	—	1 (nix-compat, for protobuf types)	new crate

Deployment Model¶

The default deployment uses process composition (e.g., process-compose) to start all required services for local development with a single command. The repository provides a canonical composition file.

Eos does NOT manage the lifecycle of snix store daemons, builders, or eval workers. Lifecycle management is delegated to the operator's chosen orchestrator (process-compose for development, systemd for production, k8s for clusters).

Scheduler Integration¶

(Note: Superseded in part by ADR-0005 — with no evaluation stage, one worker pool (build) remains; the eval worker pool and its ifdSystems metadata below have no substrate analogue.)

The scheduler manages two worker pools with separate registries:

Eval worker pool: Dynamic registration via Cap'n Proto handshake. Rendezvous hashing for routing. Deduplication via compute_eval_cache_key() (BLAKE3 hash of normalized request). No work-stealing (eval durations have low variance).
- Metadata: ifdSystems: List<Text> (systems the worker's dedicated IFD builders handle; used solely to route eval requests to capable eval workers — never for IFD build placement; empty if the worker has no IFD builders), speedFactor: UInt32 (relative priority).
Build worker pool: Dynamic registration via Cap'n Proto handshake. Rendezvous hashing with cache affinity. Deduplication via plan_digest(). Work-stealing enabled for load balancing across heterogeneous machines.
- Metadata: systems: List<Text> (hard predicate — derivation system ∈ worker systems), supportedFeatures: List<Text> (hard predicate — derivation required features ⊆ worker features), speedFactor: UInt32.
maxConcurrency is operational config (admission control), not a scheduling predicate. It limits concurrent job slots per worker but does not influence job routing.

Both pools use the same lease-based health monitoring defined in the scheduler spec. Workers send periodic heartbeats to the scheduler via registration.heartbeat() on the Registration capability returned at registration time (worker→scheduler keepalive model). The scheduler tracks last_heartbeat per worker and marks workers unhealthy if the deadline is exceeded.

Transition Path and Implementation Timeline¶

The transition from the monolithic bootstrap state to the target distributed service boundary architecture proceeds in three distinct phases:

Phase 1: Local In-Process Orchestration (Bootstrap): The scheduler evaluates and executes builds locally by linking the in-process SnixEngine (via eos-snix). Builders are invoked directly or via host-level Bubblewrap sandboxes. This establishes correct build and evaluation logic without network distribution overhead.
Phase 2: Distributed Service Boundary Migration: Introduce the Cap'n Proto worker interfaces (EvalWorker and BuildWorker) and the standalone Build Worker Shim. The scheduler is refactored to delegate jobs via Cap'n Proto RPC client handles instead of running them in-process. The local SnixEngine is removed from eos-daemon.
Phase 3: Remote Store/Worker Deployments: Migrate store access to remote gRPC endpoints (BlobService, DirectoryService, PathInfoService). All workers (eval and build) connect to these shared stores, enabling stateless daemon failovers and dynamic scaling of worker pools.

Simplicity and Volatility Boundaries (Hickey/Lowy Audits)¶

The service-boundary architecture decouples Eos from the snix runtime by separating concerns along physical and logical interfaces:

Spatial Simplicity (Hickey Audit):
- Uncomplecting Protocols: Decouple the wire protocol used for job scheduling and cluster orchestration (Cap'n Proto) from the wire protocol used for low-level build execution (snix gRPC). Eos scheduling concerns (capability handshakes, cancellation, progress streams) are decoupled from the stateless unary build invocation.
- Core Scheduler Decoupling: The scheduler does not hold or import snix types, store traits, or evaluation runtimes. It operates solely over abstract Cap'n Proto interface handles. Concerns of resource allocation and execution engine details are fully uncomplected.
Temporal Volatility (Lowy Audit):
- Volatility Axis (Sandboxing Environment): Build isolation technologies (Bubblewrap namespaces, OCI runtimes, or microvms) evolve rapidly and represent a high volatility axis of change. By confining these execution environments to the remote worker shim/builder daemon boundary, we ensure that changes or updates to sandboxing technologies do not propagate into or require changes in the Eos scheduler.
- Temporal Decoupling via Leases: Traditional execution models tie the lifecycle of a build task to the lifetime of the underlying network connection. By using lease-based job state management and keepalive heartbeats, we decouple transient connection drops from job failures. A dropped connection triggers lease expiry, allowing the job to be cleanly reclaimed and re-queued or routed elsewhere.

Consequences¶

Positive¶

True decoupling. The scheduler has zero snix dependencies. It speaks Cap'n Proto to abstract worker interfaces.
Multi-instance scheduling. Eos can dispatch eval and build jobs across any number of snix instances on disparate machines.
Eval parallelism. Large batches of atom evaluations are distributed across an eval worker pool, eliminating the single-evaluator bottleneck that plagues Nix deployments.
Independent scaling. Eval worker count and build worker count are independent knobs. A cluster can run 32 eval workers and 8 builders, or vice versa.
Stateless scheduler. The scheduler holds only ephemeral in-flight state. The artifact store (snix blob service) is the durable source of truth. External orchestrators can swap scheduler instances freely.
No eval sandboxing overhead. Pure eval provides language-level confinement. No Bubblewrap, Birdcage, or namespace management needed for eval workers.
Clean testing. The scheduler is testable with mock Cap'n Proto workers. No snix installation needed.
Global shared artifact store. All workers and builders in the cluster use the same network store. Artifacts accumulated anywhere are instantly available cluster-wide, eliminating redundant builds.

Negative¶

Operational complexity. Local development requires starting multiple processes (store daemon, eval worker, builder, scheduler). Mitigated by process-compose.
Latency. gRPC store access adds network round-trips compared to in-process calls. Negligible for builds (which dwarf store latency) but measurable for eval-heavy workloads with many store lookups. Mitigated by deploying eval workers co-located with store daemons.
Two wire formats. Eos uses Cap'n Proto; snix uses gRPC/protobuf. The build worker shim bridges them. This is intentional — Eos's protocol carries scheduling semantics (capabilities, progress streaming, cancellation) that snix's protocol does not.

Risks Accepted¶

gRPC compatibility. We depend on snix's gRPC interface stability. The MIT-licensed protos suggest this is intended as a stable contract, but snix has not formally committed to API versioning. Mitigation: pin to a specific snix commit (already done via eos-patches branch).
Eval worker complexity. The eval worker still links snix-eval, snix-glue, and nix-compat. This is the tightest coupling remaining. Mitigation: the eval worker is a separate binary with a focused responsibility; changes to snix-eval affect only this binary.

Deferred Decisions¶

ADR-0003 (proposed): Import-from-Derivation Topology¶

IFD is an internal concern of the snix evaluator — the Eos scheduler is not aware of IFD builds. Eval workers handle IFD internally via SnixStoreIO's BuildService gRPC handle. Operators configure IFD builder topology:

An ADR-0003 is no longer needed for IFD topology. The decision has been made: IFD builds are internal to the IFD-enabled evaluator, not scheduled as separate eos cluster builds. The scheduler has no direct IFD visibility; it only needs to know which evaluators have IFD builders configured (via ifdSystems worker metadata). See ADR-0003 for the deployment model decision that now occupies this slot.

(Note: Superseded by ADR-0005 — with no evaluator, IFD has no substrate analogue; this entire topology question is dead scope under the primary executor and survives only within the optional passthrough-snix executor's own internals.)

Future ADR (proposed): Eos Caching and High Availability¶

Two interrelated concerns deferred from this ADR:

Eval/build output caching. Atom metatags (cryptographically signed by the atom owner) can declare derivation digests, providing a distributed, decentralized cache. For third-party evaluations, an Eos-level cache backed by the snix blob service provides the fallback. The interface design (latency requirements, cache invalidation, consistency model) warrants dedicated analysis.
High availability. The stateless scheduler design enables external orchestrators to swap instances, but the transition semantics (in-flight job recovery, worker re-registration, lease handoff) need specification.

(Note: The ADR-0004 slot was ultimately utilized for ADR-0004: Learning-Augmented Build Scheduling, which defines the scheduler's heuristic placement and cache affinity scoring.)

Eos Software Architecture Document — the comprehensive architecture blueprint that this ADR supports
Eos Scheduler Specification
Eos Network Protocol Specification
Ion–Eos Contract
ADR-0003: Composable Deployment Modes — supersedes the monolithic binary alternative deferred by this ADR
ADR-0004: Learning-Augmented Build Scheduling — defines the scheduling algorithm running in the daemon

Alternatives Considered¶

Alternative 1: Embedded Snix with Worker Trait Abstraction (Prior ADR-0002 Draft)¶

Decouple the scheduler from SnixEngine via a Worker trait, but keep snix services embedded in-process within a LocalWorker.

Why reconsidered: This addresses the scheduler → engine coupling but not the fundamental issue. The LocalWorker still embeds the full snix stack (6 crates). The orchestrator (eos/src/bridge.rs) still holds snix service trait objects. The daemon still cannot schedule across multiple snix instances because "local" and "remote" are architecturally different paths. The Worker trait is retained in the revised design but applied to Cap'n Proto clients, not in-process engines.

Alternative 2: Direct snix gRPC with No Eos Protocol Layer¶

Have the scheduler speak snix's gRPC protocols directly to stores and builders, without the Cap'n Proto shim.

Why rejected: Snix's gRPC build protocol is unary and lacks progress streaming, job cancellation, lease tracking, and deduplication. These are essential scheduling concerns. Additionally, Eos needs sovereign authentication on all daemon-to-worker communication, which is handled at the Cap'n Proto capability level. Adopting snix's gRPC as the scheduler's wire format would require extending snix's protocol with Eos-specific features — a poor separation of concerns.

Alternative 3: Monolithic <code>ion</code> Binary with Embedded Services¶

Compile all services (scheduler, eval worker, builder, store) into a single ion binary via feature flags, eliminating the need for process composition.

Why deferred: Attractive for developer ergonomics but premature. It would create two integration paths (monolithic and microservice) before either is mature, doubling the testing and maintenance surface. It also violates layer discipline (L3 embedding L2 internals). May be revisited once the gRPC-first architecture is proven, as an optional convenience feature.

Alternative 4: Custom Store Protocol (Cap'n Proto for Stores)¶

Define Eos-native store interfaces in Cap'n Proto rather than using snix's gRPC store protocols.

Why rejected: Snix's store protocols are mature, production-tested (Replit runs tvix-store at scale: 6TB → 1.2TB via content-addressed dedup), and MIT-licensed for exactly this purpose. Reimplementing them in Cap'n Proto would duplicate effort without clear benefit. If future store requirements diverge (e.g., Cyphr/Coz-native storage), a new store backend can implement snix's existing gRPC interfaces rather than requiring a protocol change.