Skip to main content

Runtime Contents

Runtime Contents Specification​

You can create content definitions ready to be used in Runtime Environments.

AWS S3 Buckets and Git Repositories contents are supported for now.

The content specifications often rely on secrets the cluster manager will need to create.

The complete specification is available here and the following sections show simple examples.

Every content carries a spec.uid — a ULID assigned once, when the content is defined. Runtime Environments select contents by that uid, never by name, so a content can be renamed or re-described without every Environment that uses it silently pointing elsewhere. A git content also carries the revision (commit sha) it is checked out at: one without a revision is refused at launch. The Operator's page explains what happens at launch.

Apply the platform definitions​

The platform's own definitions live in the Services repository under plane/etc/specs/runtime-contents/ (one file per content, each with its uid). They are applied with kubectl; no Plane command applies them, and redeploying the Operator (plane reup datalayer-operator) leaves them as they are — after an upgrade of the Operator that changed the specification, re-apply them:

kubectl apply -n datalayer-runtimes -f plane/etc/specs/runtime-contents/
kubectl get runtime-contents -n datalayer-runtimes -o custom-columns=NAME:.metadata.name,UID:.spec.uid,TYPE:.spec.type,REVISION:.spec.revision

Every row must show a UID; a content without one cannot be selected by any Environment, and the Operator logs RuntimeContent '<name>' has no spec.uid.

Manage the Contents​

kubectl get runtime-contents.datalayer.io -A
kubectl get runtime-contents.datalayer.io -o yaml -A

AWS S3 Example​

In this example, the authentication must be stored in the s3-secret secret providing the following key.

  • access_key_id
  • secret_access_key
  • region
cat <<EOF | kubectl apply -f -
apiVersion: datalayer.io/v1
kind: RuntimeContent
metadata:
  name: s3-satellite-sentinels-images-content
  namespace: datalayer-runtimes
spec:
  uid: 01M0YX0MXYH92ETBXWPH7A8205
  description: Satellite images from Sentinels mission.
  type: s3
  source: datalayer-dev
  permissions: ro
  secret: s3-secret
EOF
kubectl get runtime-contents.datalayer.io -A
kubectl get runtime-contents.datalayer.io -o yaml -A

Git Repository Example​

In this example, the Git repository must be public.

cat <<EOF | kubectl apply -f -
apiVersion: datalayer.io/v1
kind: RuntimeContent
metadata:
  name: jakevdp-sklearn-tutorial-content
  namespace: datalayer-runtimes
spec:
  uid: 01M0YX0MXYRP0Q29YWYRE5THZE
  description: Jake VanderPlas scikit-learn tutorial
  type: git
  source: https://github.com/jakevdp/sklearn_tutorial.git
  revision: 5098cee2a638c56c311aca0c18987e407fe127fd
EOF

A new uid is minted with python -c "import ulid; print(ulid.new())" (or any ULID generator); the revision is git ls-remote <repository> HEAD at the moment you pin it.