Skip to main content

☰ 📦 Datalayer Runtimes

KubernetesREST API

Datalayer Runtimes service manages access to ⚪ Runtimes based on the defined 🎱 Runtime Pools.

🌳 Runtime Environments that define the image, the CPU/GPU and memory ranges, the capabilities, and the Runtime Contents they bring (selected by uid: git repositories pinned to a revision, subpaths of the shared filesystem, S3 buckets).

🎱 Runtime Pools that keep a number of Pods of one Environment warm, so a launch does not wait for a cold start.

⚪ Runtimes (active or paused) ready to serve users.

📎 Contents attachments — the Home Folder, a Volume, a local folder over the bridge relay — prepared with the Contents service at launch and handed to the Operator (in the cluster) or mounted by the sandbox itself (at an external provider).

Defining the first three is an administrator's job, described in Platform; what this service does with them is below.

Deploy Datalayer Runtimes​

plane up datalayer-runtimes
plane ls

Check the availability of the Datalayer Runtimes Pods.

kubectl get pods -n datalayer-api -l app=runtimes

Check the logs of the Datalayer Runtimes Pods.

kubectl logs -n datalayer-api -l app=runtimes -f

Check the availability of the Datalayer Runtimes Certificate.

kubectl describe certificate ${DATALAYER_RUNTIMES_URL#https://}-datalayer-runtimes-cert-secret -n datalayer-api

Check the availability of the Datalayer Runtimes Endpoints.

open ${DATALAYER_RUNTIMES_URL}/api/runtimes/version
open ${DATALAYER_RUNTIMES_URL}/api/runtimes/v1/ping

The Kubernetes objects: datalayer.io/v1​

Three custom resources hold what a runtime is made of. They live in the datalayer-runtimes namespace, ship with the Operator's chart (operator.crds=true), and the Operator is what watches them. Runtimes reads none of them from the cluster: it asks the Operator (GET /api/operator/v1/environments), so an object the Operator has not seen is one this API cannot answer for either.

KindWhat it holdsReference
RuntimeContentSomething an Environment can bring into a runtime: a Git repository, a subpath of the shared filesystem (nfs), or an S3 bucket. Selected by its spec.uid, never by nameRuntime Contents
RuntimeEnvironmentA platform environment: its image, resources, capabilities, and the contents it brings. Launched by nameRuntime Environments
RuntimePoolA set of prewarmed Pods of one environment, so a launch does not pay for a cold startRuntime Pools

The reference pages carry each kind's fields, examples and how to apply them. What this service adds on top of them is below: which contents a sandbox provider can deliver (Environment Contents), and what an account attaches to its own sandbox rather than what the Environment brings (Contents attachments). What the Operator does with them at launch is Runtime Contents and Environments on the Operator page.

Two things the kinds are not:

  • A RuntimeContent is the Environment's content, the same for everyone who launches it. A Home Folder, a Volume, a bucket or a folder of someone's own machine attached to their sandbox is a Contents source instead — see Contents attachments and the Contents page.
  • A RuntimeEnvironment is only the platform kind. A User Environment, which an account builds and versions itself, is not a custom resource; it lives in the registry and in Solr — see Environments.

The platform's own manifests live in the Services repository under plane/etc/specs/runtime-contents, …/runtime-environments and …/runtime-pools, applied with kubectl apply -f. An Operator upgrade that changes how they are read needs them applied again; see After the upgrade.

kubectl get rte,rtp,rtc -n datalayer-runtimes

A sandbox's home​

Every sandbox runs as datalayer (uid 1000, gid 100), and its home is /home/datalayer. That home is also the working directory: a kernel starts there, pwd in a cell answers /home/datalayer, and the file browser is rooted at the same place, so what a cell writes with a relative path is what the browser shows.

PathWhat is there
/home/datalayerThe home, and the working directory. Nothing is mounted over it
/home/datalayer/<handle>A Home Folder the caller reaches — their own, and one per organization and team (Contents attachments)
/home/datalayer/<path>What the Environment brings, at the path it declares: a Git checkout, a dataset, a bucket (Environment Contents)
/opt/datalayerReserved: the contract's doctor, its constraints and its wheelhouse. Read-only to the user

The working directory is the sandbox contract's, not each image's choice: an environment built for Datalayer, E2B, Daytona or Modal starts in the same place, and the contract's own check compares a built sandbox's pwd against it. In the cluster the Pod settles it — the Operator sets the Jupyter container's workingDir from the Environment's spec.home, so an image built elsewhere, or built before the contract, still starts in the home. Jupyter roots itself at the server process's directory and a pooled kernel keeps that directory until a client asks for a path, so this one setting is also what the file browser lists. The platform images declare the same WORKDIR, so a build and a Pod agree.

A pool pod keeps the spec it was born with for as long as it waits to be assigned (Runtime Pools), so a change like this reaches warm pods only as they are used and replaced: after changing it, expect the next launch or two to come from a pod created before the change. What an administrator declares for the home is The home.

Environment Contents​

An Environment selects Runtime Contents by uid. Whether a selected content can be delivered depends on where the sandbox runs. In the cluster everything is a mount. At an external provider there is no cluster: a git content is a materialized checkout, a bucket is reachable from code through a Python filesystem object (fsspec) with the bucket's credentials — not mounted — and the platform shared filesystem cannot be reached at all.

Content typeDatalayerDaytona, E2B, Modal
gitmount of the pinned checkoutmaterialized checkout
nfsmount of the shared filesystem subpathrefused (nothing can mount the platform filesystem)
s3mount (Node Mount Gateway, Mountpoint for S3)python access from code; not mounted

Any other provider has no delivery rules and delivers nothing.

Alongside the mounted contents, an Environment may carry a build manifest of immutable files — contents_build, each entry a source URL, a destination path and a sha256. These are not mounted at launch; they are baked into the artifact at build time, through the same engine that bakes an Environment's uploaded files (code_sandboxes build_commands, via files_step): one verified fetch per entry, a checksum that fails the build, and the manifest at /etc/datalayer/environment-contents.json — on the Datalayer image, a Daytona snapshot, an E2B template and a Modal image alike. GET /environments/<name>/contents returns the build in build, and installed_environment_contents reads the manifest back from a running sandbox.

Endpoints

  • GET /api/runtimes/v1/environments — every platform Environment, with contents: the selected uids and, resolved from their definitions, name, type, revision, mount and permissions; then the caller's User Environments, with uid, origin, owner, promotedVersion, variants and nextCursor.
  • GET /api/runtimes/v1/environments/<name>/contents?provider=<provider> — the contents resolved for a provider (the Environment's own by default): environment, provider, supported and contents, a list of resolved contents each carrying uid, name, type, mount, permissions, revision, sha256, a status of resolved, unsupported or unresolved, and a detail saying what the content becomes or what cannot be met. When the Environment resolves on Datalayer, the result is recorded as a Contents environment source (once per Environment, idempotent on its uid, brought up to date when the build changes) and returned as source.
  • PUT /api/runtimes/v1/environments/<name>/contents — platform administrators only; body {"contents": [{"uid": "...", "mount": "...", "permissions": "ro"}]}; proxied to the Operator, which resolves every uid before patching the RuntimeEnvironment.
  • GET /api/runtimes/v1/runtime-contents — the catalog as a user may see it: uid, name, type, description, permissions, and for a public git repository its source and pinned revision. Credentials, bucket names and filesystem paths are withheld.

Runtimes also serves the User Environments registry (/environments/{uid}, /environment-versions/…, /environment-builds/…, /environment-publications, /environment-quotas, /sandbox-providers) and the durable worker's internal routes under /internal/…. See Environments.

The launch gate

POST /api/runtimes/v1/runtimes refuses a launch — before anything starts — when a content the Environment selects cannot be delivered where the sandbox will run: 422 with code ENVIRONMENT_CONTENT_UNSUPPORTED and a message naming the content, the provider and the semantics that cannot be met (ENVIRONMENT_CONTENT_UNRESOLVED when a selection matches no definition or a git content is not pinned).

When the launch is on Datalayer and every selected content is deliverable, the gate also records the resolved contents as the Environment's environment source — the same idempotent write the inspection endpoint makes — so a launched sandbox's build is on record even if nobody inspected the Environment first. This is best-effort: a launch is never failed because the catalog could not write the record.

Registering the environment source needs DATALAYER_RUNTIMES_API_KEY (the Runtimes identity Contents accepts for it) and DATALAYER_CONTENTS_URL.

Contents attachments​

An account attaches what a sandbox should see through the Contents catalog (POST /api/contents/v1/attachments, naming the sandbox), and Runtimes is where that attachment meets the launch. The Environment Contents above are the Environment's; these are the account's. Runtimes' part, in order:

  1. Prepare. POST /api/runtimes/v1/runtimes with content_attachment_uids: [...] (and a pod_name; without one Runtimes names the sandbox runtime-<ulid> and prepares for that name). Runtimes reads the sandbox's manifest, GET /attachments/manifest/{sandbox_uid}, with the caller's credentials — a uid the manifest does not list is 404 — then has Contents prepare each one, POST /attachments/{uid}/prepare with token_audience: sandbox:<uid>, under its own identity (DATALAYER_RUNTIMES_API_KEY, the attachments:prepare scope). Preparing is what mints a short-lived token for what the sandbox will fetch, or a bridge session for a local mount. Each prepared attachment is stamped with its source's kind, read with the caller's credentials again — a source the caller cannot read refuses the launch — because the Operator renders a Home Folder (files) and a Volume differently and the catalog is where that difference is written. A files source attached under /home/datalayer — a share — comes back from prepare with its source_path, the claim-relative folder the Operator binds; the manifest carries the same field, so a listing and a launch name the same place.
  2. Resolve the Home Folders. When the request asks for the home_folder capability or one attachment is a Home Folder, Runtimes asks IAM for the caller's memberships and sets home_folder_mounts to their own folder plus one per organization and team — mounts supplied by the client are replaced, never trusted. When IAM cannot answer, the runtime gets its own folder only. Without a Home Folder in the request, no home folder is mounted: a Volume alone mounts its Volume.
  3. Refuse before anything runs. Two gates answer 422 with a code before a provider is chosen: ENVIRONMENT_CONTENT_UNSUPPORTED / ENVIRONMENT_CONTENT_UNRESOLVED (the Environment's contents, above), and LOCAL_BRIDGE_UNSUPPORTED for a local-bridge attachment on an environment that cannot run the bridge filesystem. A bridge mount is a userspace filesystem inside the sandbox — fusepy over /dev/fuse — and whether a sandbox has one is a property of its environment, never of the provider: Runtimes reads the environment's features from the Operator's record of it, else from the provider catalog in code_sandboxes, and refuses unless fuse is among them, naming the environment, the provider, what it would need and what it advertises, and offering Synchronize (a copy, called a copy). This gate judges external sandboxes only; the stock external environments declare no fuse. On Datalayer nothing runs in the sandbox: the Operator writes a grant and the Node Mount Gateway runs the bridge filesystem on the node. Nothing refuses a Datalayer launch up front either: with the gateway or its local-bridge switches off, the Operator logs GATEWAY_KIND_OFF and launches the sandbox without the folder (Node Mounts).
  4. Hand over. In the cluster the prepared attachments go to the Operator in content_attachments of POST /api/operator/v1/runtimes, which mounts them and reports each outcome to Contents itself (see Operator). At an external provider (Daytona, E2B, Modal) there is no Operator in the path: the session thread fetches the manifest with the user's credentials, builds the code_sandboxes content manifest — the account's attachments, then the Environment's contents as environment entries, plus contents_url — has code_sandboxes prepare it inside the sandbox (materialized checkouts, bucket access from code, the bridge filesystem run from inside the sandbox), and reports every outcome under DATALAYER_RUNTIMES_API_KEY (the attachments:status scope). A required attachment Contents no longer lists is an error; an optional one is skipped and logged. Without a service key nothing is reported and the attachment stays preparing — a misconfiguration that shows in the catalog rather than hiding.

The attachments a runtime was launched with travel on its record, so a runtime's description lists what it actually mounts, and an external session's manifest can be put back in the form Contents takes.

Which launches wait for a Pod​

A Pod's volumes are fixed when it is created, so a launch that mounted anything used to be given a Pod of its own — scheduling, an image pull and a Jupyter startup probe — while a launch that mounted nothing was handed a prewarmed one. With the Node Mount Gateway deployed and DATALAYER_NODE_MOUNT_GATEWAY_ENABLED set on the Operator, a Home Folder launch is served from the pool as well: the Operator writes the mount set on the pooled Pod and the node agent binds the folders in while it runs.

Nothing in step 2 changes. The memberships are still resolved in Runtimes, from IAM, with the caller's own credentials, and still replace whatever the client sent — the gateway changes when a folder is mounted, never who decides which. A Volume or a Local Mount still gets a Pod of its own, and so does everything when the gateway is off. The Operator's page has the full table: Pool or its own Pod.

A runtime's record carries mount_gateway, so a client can tell before it asks whether this sandbox takes a mount while it runs.

Configuration

VariableMeaning
DATALAYER_CONTENTS_URLWhere the manifest is read and attachments are prepared and reported (runtimes.env in the chart; default the in-cluster datalayer-contents-svc).
DATALAYER_RUNTIMES_API_KEYRuntimes' identity to Contents (attachments:prepare, attachments:status, and registering the environment source): 503 without it as soon as a launch names an attachment. Also Runtimes' key to the Operator for a User Environment launch, a build's smoke test and an environment's live runtimes (503 DL_ENV_UNAVAILABLE without it), and for remote registration. plane up datalayer-runtimes refuses without it.
DATALAYER_OPERATOR_API_KEYThe key on every other call Runtimes makes to the Operator.

An external sandbox answers its own account​

The /api/runtimes/v1/external/{runtime_name}/api/... surface is a Jupyter Server as far as a client is concerned, standing in front of a sandbox at Daytona or E2B. Opening one is scoped to the caller — the record is fetched from the Operator as that user, so one account cannot open another's sandbox by knowing its name.

Until 2026-09-05 the routes that only looked at a sandbox already open, or closed it, took no caller at all. Against r1 with no Authorization header, GET …/api/status and GET …/api/kernels both answered 200, and DELETE …/api/kernels/{id} closed the sandbox for anybody who knew the runtime name — which travels in an agent's _meta, in task records and in logs. The POST …/restart immediately below the delete required a caller; the delete did not.

They all require one now, and answer only the account whose sandbox it is. A caller who is not the owner gets exactly what an address with nothing open gets — an empty kernel list, kernels: 0, a 404 for a named kernel, and a 204 from a close that closed nothing — so a reply never says which runtime names are real.

The kernel channel, WS …/api/kernels/{kernel_id}/channels, follows the same rule since 2026-09-19: it used to check only that the token was valid, so any signed-in account that knew a runtime name could run code in that sandbox. A Runtimes image built before that date still has the gap. A caller who is not the owner gets the close an address with nothing open gets.

Seven routes still take no caller, deliberately: the version handshake, the kernel spec, the empty terminal list, the two checkpoint stubs, the interrupt that always answers 501, and the event stream, held open and silent. Each returns the same thing for every name and touches nothing, so none of them can say whether a sandbox exists.

Sharing a runtime​

A runtime can be shared with other people, teams, organizations and service agents, at one of three nested levels: view reads outputs, update may change files, execute may run code.

RouteWhoWhat it does
GET /api/runtimes/v1/runtimes/{name}/sharingthe ownerThe grants, per level and kind of principal
PUT …/sharing {"access": {"view": {"userUids": […], "teamUids": […], "organizationUids": […], "agentUids": […]}, …}}the ownerReplace the grants at the levels named; levels not named are kept. Leaving no grant at all removes the record
GET …/permissionsanybodyview, update, execute for the caller, owner, and whose it is
GET /api/runtimes/v1/runtimes/{name}a granteeThe runtime as its owner sees it, marked shared_from and shared_access

Ownership is the operator's to say. A runtime is listed for the account that launched it and for nobody else, so every question here starts by asking the operator as the caller: answered, the caller owns it; refused, the record decides, with the caller's memberships from IAM. An agent grant is matched on the token's own agent_uid and never on the person behind it. "Not shared with you" and "no such runtime" answer alike — three false — so asking is not a way to learn which runtimes exist.

The record is one document per shared runtime in the sandbox-snapshots collection, beside the snapshots and never carrying the two fields their queries key on. It lives as long as its runtime: the cleanup loop purges records past the runtime's expiry. The MCP Server is the first caller: its share_sandbox and unshare_sandbox tools write here with the caller's own credential, and its gateway asks permissions on every call a grantee makes.

A token narrowed to resources​

A token carrying authorization_details (IAM's task grants, O1-06) reaches the sandboxes and snapshots it names, by name. A dependency in front of every router but the probes (api/v1/narrowing.py) lets a request naming a sandbox (runtime_name) or a snapshot (id) go on only when the token names it with the action its method needs — 404 otherwise — and refuses 403 anything else a narrowed token asks, a listing or a creation, since a manifest names resources and never the collections they are in. A request whose token is not narrowed, or that carries none, is left to the route as before.

Tear Down Datalayer Runtimes​

If needed, tear down.

plane down datalayer-runtimes

OpenAPI Specification​

The OpenAPI (Swagger) specification is available online.