Skip to main content

☰ 🛂 Datalayer IAM

KubernetesREST API

Datalayer IAM service provides the Identity and Access management to Datalayer and supports a variety of Authentication and Authorisation methods. It allows to manage the following artifacts.

🛂 Personal and Organisation Accounts.

🧑‍🤝‍🧑 Teams.

🛡️ Authorization Policies.

The OAuth 2.1 authorization server​

IAM is where an MCP client authenticates. It serves the endpoints an agent discovers through RFC 8414 metadata — /api/iam/v1/oauth/authorize, /token, /revoke, /register — and the /.well-known/oauth-authorization-server document that points at them. The Jupyter MCP Server is the resource those tokens are issued for, and refuses one issued for anything else.

How a client is registered​

By Client ID Metadata Document: the client_id is a URL that IAM fetches, validates and caches, and the document at it declares the client's name, its redirect URIs and what it may ask for. Advertised as client_id_metadata_document_supported in the metadata, so a client discovers it rather than being told.

Dynamic Client Registration (RFC 7591, /oauth/register) still works and is the deprecated fallback. A document needs no registration call and no stored secret, which is what makes it right for a client that discovers servers at runtime.

The fetch is bounded, and the bounds are the security properties rather than tuning:

Timeout3 s for the whole fetch — connect, send, read
Redirects3, all on the same host
JWKS64 KiB
Logo256 KiB
Cache5 min to 24 h; 1 h where the document says nothing

A client's document lives on the client's own server, so fetching it is IAM reaching out to a third party during an authorization. The timeout and the size caps are what stop that server from holding an authorization open or handing back a download. Redirects stay on the host because the hostname is the client's identity: following one off-host would let a document delegate its identity to somewhere the person approving it never saw.

no-store gets the lower bound, not no caching

A document refetched on every request would let the client's server watch every authorization its client is part of — who signs in, when, how often. The cache exists to prevent that, so a document asking not to be cached is still held for five minutes.

A URL client authenticates at the token endpoint with none or private_key_jwt, never a shared secret: a document anybody can read cannot hold one. The symmetric methods remain for the deprecated registration path.

Redirect URIs, and the one exception​

A redirect URI is matched exactly against the client's document — the whole string as the client wrote it, never a prefix, never a different scheme, path, query or fragment.

The exception is the port of a loopback address, where any port is accepted.

This exception is not optional

RFC 8252 §7.3. A native application — Claude Code, Claude Desktop, any desktop agent — listens on whatever port the operating system hands it at the moment it starts the flow: http://localhost:3118/callback today, something else tomorrow. No document can list those ports, because the client does not know them when it is written.

Matching the port too refuses the client on its own callback:

{"error":"invalid_request",
 "error_description":"Redirect URI [http://localhost:3118/callback] is not registered for this client."}

which reads to whoever hit it as the server being broken. If you see this error, the host and the path are what to check — those are still compared exactly, and 127.0.0.1 is a different address from localhost as far as matching is concerned.

Nothing else is relaxed. A public redirect gets no exception at all: there the port is part of the address the client promised to be at.

PKCE​

Required, S256 only. An authorization request without code_challenge, or with plain, is refused before any browser is redirected — so the consent screen only ever shows a request that would succeed.

Scopes​

What the resource parameter names wins over what the client asked for. The resource is chosen by the person setting the client up, and it is the only say they have over a client that sends whatever scopes it likes.

The OpenID Connect scopes — openid, profile, email, offline_access — pass through so one sign-in answers both questions. None of them grants anything on the MCP server.

Grant types​

grant_types_supported names three, and the token endpoint accepts exactly those three — one list in services/oauth_provider.py, read by the OAuth metadata, the OpenID document and the unsupported_grant_type refusal alike.

GrantWho uses it
authorization_codeEvery MCP client, with PKCE
refresh_tokenThe same clients, to stay connected
urn:ietf:params:oauth:grant-type:token-exchangeDatalayer services only
Token exchange is advertised, and refused to everyone but services

RFC 8693. The audience of an access token is what stops one minted for the MCP gateway being replayed against another API — and the gateway genuinely does need to call the Spacer on the user's behalf, holding a token that names the gateway. It exchanges that token for one naming the Spacer.

It is in the metadata although only a trusted service may use it: the document describes what the server supports, and who may use it is what the refusal says — invalid_client, precisely. A capability left out of the description because it is restricted is a capability nobody can discover is there, and this one was omitted for exactly that reason until a test held the two lists against each other.

client_credentials and jwt-bearer are not served. Service agents authenticate by a different route — see below.

Organization, team and personal MCP policy​

/api/iam/v1/mcp-policies/{scope}/{subject_uid} serves and stores the layers the Jupyter MCP Server enforces, at three scopes: organization, team and personal. An organization's owner writes its layer, a team's owner writes the team's within it, and a person writes their own.

Six rules, and no others:

RuleValue
toolDenylistTool names refused
toolAllowlistTool names permitted, everything else refused
allowedClientsClient ids — a CIMD URL admits that client and no other
maxCallsPerMinutePer-subject rate cap
maxCreditsPerDayThe daily budget
maxConcurrentSandboxesSandboxes running at once
A rule IAM does not know is refused, not stored

The catalogue is kept beside the gateway's own ENFORCED_RULES. A rule the gateway enforces but IAM does not list cannot be saved — harmless, and visible immediately. The other way round is a policy page that lies: a rule stored, displayed, and enforced by nothing.

So a mistyped toolDenyList is an error the administrator sees, rather than a setting that quietly does nothing.

The last three are the quotas. They are policy rules rather than a surface of their own, because a second place to write a limit would be a second answer to what an organization is allowed. The gateway reads them and refuses against them; GET /api/mcp/v1/organizations/{uid}/usage puts each beside its use.

Alert rules​

McpAlertRule, stored per organization. A rule names a condition, an operator, a threshold, a window and a destination. IAM refuses a rule it cannot evaluate — an unknown condition, a window of zero, a threshold of nan — rather than storing it: a refused rule is visible, while a stored one nobody can evaluate is a condition that looks watched and is not.

Service agents​

/api/iam/v1/mcp-service-agents/…, owned by an organization: created, rotated and revoked by its owners. A pipeline, a CI job or a bot holds its own key and spends the organization's credits under its own agent_uid — so the work does not stop when the engineer who set it up leaves, and its spend is attributed to it rather than to them.

They do not use client_credentials. The gateway presents the key to POST …/authenticate, which answers with the principal, and the endpoint is restricted to trusted services: anything that may call it may test keys.

A bad key is a 401, never a 200 with an empty principal

A caller that reads a truthful empty answer as an anonymous success is how an unauthenticated agent gets a session.

org_uid and team_uid claims​

Chosen at consent and stamped into the token. The gateway keys policy, quotas, alerts and the audit on org_uid, so a token without one is a personal-scope session. IAM validates the choice against membership before stamping it: a person cannot act for an organization they do not belong to, and a team is refused if it is not that organization's.

What is not here yet​

Named plainly, because the MCP plan refers to it and an operator should know:

  • DPoP. Not implemented, and deliberately not built ahead of the Agent Identity Working Group's finalized MCP profile. Sender-constrained tokens are the right answer to a stolen bearer token, and building against a draft that then changes would leave a deployed proof format nobody else speaks.

Where IAM's own audit rows go​

IAM records its security decisions — a token issued, refused, exchanged or revoked; a client document fetched or thrown out of the cache; an MCP policy set or removed — in the same McpAuditEvent shape the gateway uses, so the two halves of the story can be read together.

By default they go to the datalayer_iam.audit logger, one JSON line each, for a log pipeline to forward. That is the right default: it works with no Solr, in a container that only has stdout.

It also means those rows are not queryable. "Who changed this policy, and when" has nowhere to be asked, and an auditor ends up reading half the story from the console and half from a log aggregator — which in practice means reading half.

To put them in mcp-audit beside the gateway's rows, so GET /api/mcp/v1/audit answers both:

env:
  - name: DATALAYER_IAM_AUDIT_SINK_CLASS
    value: datalayer_iam.services.audit_solr_sink:SolrAuditSink
Written to the log as wellAlways — it is what a failed collection write is recovered from
A failed collection writeLogged, not raised
A retried writeAnswers the first row; the row's uid is its idempotency key
TimestampWhen IAM recorded the decision, not when Solr accepted it
The swallow is deliberate

These rows are written on the path of every token IAM issues. Raising on a failed audit write would turn a Solr blip into a refused sign-in.

A row missing from the collection is recoverable from the log. A platform nobody can sign in to is not. If you alert on anything here, alert on the could not be written to mcp-audit warning rather than on a gap in the collection.

Deploy Datalayer IAM​

plane up datalayer-iam
plane ls

Check the availability of the Datalayer IAM Pods.

kubectl get pods -n datalayer-api -l app=iam

Check the logs of the Datalayer IAM Pods.

kubectl logs -n datalayer-api -l app=iam -f

Check the availability of the Datalayer IAM Certificate.

kubectl describe certificate ${DATALAYER_RUN_HOST}-datalayer-api-cert-secret -n datalayer-api

Check the availability of the Datalayer IAM Endpoints.

open https://${DATALAYER_RUN_HOST}/api/iam/version
open https://${DATALAYER_RUN_HOST}/api/iam/v1/ping

Tear Down Datalayer IAM​

If needed, tear down.

plane down datalayer-iam

OpenAPI Specification​

The OpenAPI (Swagger) specification is available online.

IAM Cases​

The following diagrams describe the authentication (Authn) and authorization (Authz) in various cases.

All interactions between JupyterLab and the Datalayer services are over TLS/SSL (HTTP or WebSocket) via the Ingress.

Authenticate from JupyterLab with Username and Password​

Authenticate from JupyterLab with a 3rd Party Token​

Access a Datalayer Runtime from JupyterLab​

Create a Runtimes Runtime from JupyterLab​

Access a Runtimes Runtime from JupyterLab​

IAM at Ingress Level​

In order to use Datalayer IAM as a Nginx Ingress middleware checking the user identity and authorization, the Ingress specification must have the following annotation (any service can be protected, aka forcing authentication and passing policies, using the following annotation on the Ingress).

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    nginx.ingress.kubernetes.io/auth-url: "http://datalayer-iam-svc.datalayer-api.svc.cluster.local:9700/api/iam/v1/auth"
    nginx.ingress.kubernetes.io/auth-snippet: |
      proxy_set_header X-Forwarded-Method $request_method;
      proxy_set_header X-Forwarded-Proto $scheme;
      proxy_set_header X-Forwarded-Host $host;
      proxy_set_header X-Forwarded-Uri $scheme://$host$request_uri;