Capabilities, operators and environments
Pipelines and flows say what kind of work a step needs. The environment decides which tool does it. That split is what lets the same pipeline run in a free sandbox and against real providers without changing a line.
flowchart LR
step["Step needs<br/>storage.put"] --> env{"Environment<br/>binding"}
env -->|"local-dev"| local["storage/put<br/>local files"]
env -->|"your profile"| s3["s3/put<br/>your bucket"]
local --> gate["Gate checks<br/>the operator"]
s3 --> gate
Capabilities and operators
Section titled “Capabilities and operators”An operator is the code that talks to one tool or provider. Each operator
declares, in its pack’s pack.toml:
- the capabilities it provides, such as
train.run,storage.putorcost.gate; - its risk:
read_only,external_write,paid_computeorproduction_mutation; - where it can execute (
in_process,remote_job,mcp_tool), and whether it has a mock path.
Several operators can provide the same capability. train.run, for example,
is provided by train/stub, train/local, train/modal, train/kaggle,
train/runpod and train/vast. The operators reference
lists every operator that ships with cairn.
Operators ship as their own Python packages. cairn finds an operator package
through its cairn.operators entry point once it is installed, and not before.
Environments
Section titled “Environments”An environment is a profile: a TOML file that binds capabilities to operators and picks the backends a run uses. Two environments are offered to people choosing where a pipeline runs:
| Environment | What it is |
|---|---|
sandbox | No credentials, no remote compute. In-process runtime and the built-in data and ML engines. |
prod | The production profile a deployment points at. Real providers, the audit trail switched on. |
Other profiles in cairn/profiles/ are base layers that these two extend
(extends = "local-dev.toml"), or private to cairn’s own testing.
A binding maps a capability to the operator that provides it. These are the
bindings of local-dev, the base layer sandbox extends:
[profile.bindings]"memory.search" = "memory/search" # operator-memory-local"memory.write" = "memory/write" # operator-memory-local"storage.put" = "storage/put" # operator-artifact-local"storage.get" = "storage/get" # operator-artifact-local"rag.search" = "rag/search" # operator-rag-local"rag.fetch" = "rag/fetch" # operator-rag-local"mcp.call" = "mcp/call" # operator-mcpA profile overrides only the bindings it needs. To keep artifacts in S3, for
example, a profile binds "storage.put" = "s3/put". An environment can also
set an operator’s deployment settings, such as cost thresholds; see
Cost and budgets.
The sandbox
Section titled “The sandbox”The sandbox environment has nothing to spend with: no keys, no remote compute, no durable runtime. A run started in sandbox mode (the console’s Playground) goes further: an operator runs for real only if it declares itself free of side effects (a read or an analysis). Every other operator that has a mock path takes it, so reads give real numbers while writes and spend are stubbed wherever the operator offers a stub.
The gate checks the operator
Section titled “The gate checks the operator”Every operator call passes one enforcement function before it runs. It asks
for approval when the operator itself requires it, or when a policy that
matches the call says require_hitl or deny. A flow leg is checked once,
on the highest risk among its stages: when an approval is required, the leg
pauses as awaiting approval before the engine is touched.
cairn pipeline explain <ref> prints, for each leg, the operator or engine
that will run it, its risk, and which gate (if any) applies and why.
- Gates and approvals: every way a pipeline waits.
- Connect a provider: give a production environment its credentials.
- Operators: the shipped operators and their capabilities.