Runtime Pools

A RuntimePool keeps size Pods of one Runtime Environment warm, so a launch that fits takes one instead of waiting for a Pod to be created, scheduled and probed. When a Pod is assigned to a user it is injected with the User Secrets. Which launches may take a pooled Pod, and which pay for a cold start, is Pool or its own Pod on the Operator page.

Short names: rtp, rp, rps, rpool, rpools. The complete specification is available here.

Runtime Pool Specification

Field	Meaning
environment	The RuntimeEnvironment this pool is of, by name
size	How many Pods are kept warm. The cluster has to be provisioned for the sum of every pool's size
burningRate	Credits a second a runtime of this pool burns
resources	The Pods' requests and limits. When set, they override the Environment's resourcesRanges
cull.seconds	How long a runtime of this pool lives before the Operator culls it
affinity, nodeSelector, tolerations	Where the Pods are scheduled — the runtime nodes, and the node variant the pool is sized for

A plane needs at least one pool, even where every launch is cold: the Operator culls expired runtimes from the timer the pools run.

A pooled Pod keeps the spec it was born with for as long as it waits to be assigned. A change to the Environment or the Operator reaches warm Pods only as they are used and replaced, so after one, expect the next launch or two to come from a Pod created before it.

Manage the Runtime Pools

kubectl get runtime-pools.datalayer.io -A
kubectl get runtime-pools.datalayer.io -o yaml -A

Runtime Pool Example

cat <<EOF | kubectl apply -f -
apiVersion: datalayer.io/v1
kind: RuntimePool
metadata:
  name: python-simple-pool
  namespace: datalayer-runtimes
  labels:
    project.datalayer.io/name: dev
spec:
  environment: python-simple-env
  size: 3
  affinity:
    nodeAffinity:
      requiredDuringSchedulingIgnoredDuringExecution:
        nodeSelectorTerms:
        - matchExpressions:
          - key: role.datalayer.io/runtime
            operator: In
            values:
            - "true"
          - key: node.datalayer.io/variant
            operator: In
            values:
            - medium
          - key: node.datalayer.io/xpu
            operator: In
            values:
            - cpu
  burningRate: 0.01
  cull:
    seconds: 999999
EOF