Skip to content

Welcome to Provider Datalab

Provider Datalab is a PaaS-style building block for platform operators. It turns one Datalab claim into an operator-governed cloud workspace with an online IDE, object-storage access, managed databases, document stores, key-value/cache stores, vector databases, and an optional Docker registry.

The primary reader for these docs is the platform operator. The operator decides which teams may run which workloads, which services are available, how egress is controlled, which state is backed up, and how lifecycle is governed. Engineers and data users should still be able to read the examples and understand the contract they are asking the platform to fulfill. Sponsors and governance stakeholders should be able to see where accountability lives: policy, capacity, audit, and durable data are platform responsibilities, not hidden inside a user terminal.

A named Datalab session gives users a browser editor, terminal, persistent workspace, and approved service access. A session runs in a dedicated Kubernetes namespace and may include a vCluster when it needs a separate Kubernetes API. Users receive only the workspace access, credentials, and permissions assigned to that session. Platform operators continue to manage RBAC, Pod Security, NetworkPolicies, quotas, backups, ingress, and lifecycle controls.

Architecture

The current Educates runtime provides the Datalab browser editor and terminal. Platform ingress can protect it through delegated Keycloak authentication. ADR-001 defines the Datalab resource as the public contract and keeps runtime resources as internal implementation details.

Provider Datalab requires Crossplane v2 or later. It provides a tenant-facing Datalab API and compositions that connect systems you already operate: Kubernetes namespaces, ingress, identity, object-storage credentials, persistent volumes, database operators, cache and vector-store operators, and the Educates runtime.

Provider Datalab does not create object-storage buckets. Use Provider Storage or another storage process to create buckets and credentials. Provider Datalab consumes those credentials and wires storage access into the lab.

Operator Contract

For an operator, a Datalab is not just a notebook or a pod. It is the contract for a tenant-facing service:

  • You define the platform boundary in EnvironmentConfig: ingress, authentication, storage endpoint, quotas, security defaults, and optional backend services.
  • A tenant, GitOps process, or higher-level API submits a Datalab claim describing users, sessions, files, storage credentials, runtime permissions, and requested data services.
  • Crossplane compositions create or configure the required Kubernetes, identity, storage access, and backend resources.
  • The resulting resources stay visible to the operator, so lifecycle, policy, capacity, and backup responsibility are clear.

This is the main design point: Provider Datalab makes self-service smooth without hiding state from the platform team. In the current API, a named session is a long-lived environment, while its workspace Pod is replaceable. The Pod provides the browser editor, terminal, tools, and preconfigured session access. Large, isolated, or repeatable work belongs on separate task workloads. Its /home/eduk8s workspace is backed by the session's stable PVC; persistence of other session applications is not yet proven. Databases, buckets, persistent volumes, and other stateful services remain platform concerns.

For governance, this gives sponsors a concrete review surface: a Datalab can be approved, costed, secured, and retired as a named platform service instead of becoming unmanaged compute plus scattered credentials.

What It Provides

At its core, Provider Datalab provides:

  • A Datalab Composite Resource Definition (XRD).
  • Compositions for Crossplane v2 or later that create environments with sessions, storage access, vClusters, identity wiring, and optional managed backends.
  • A default datalab-educates runtime that launches VS Code Server, terminals, a storage browser, and common tools such as awscli and rclone.
  • Optional Keycloak-managed access, including confidential clients, runtime OAuth2 credential Secrets, groups, roles, role scope mappings, role bindings, service-account API access, and memberships.
  • Support for delegated authentication through the surrounding platform, for example NGINX external auth or APISIX OIDC protection at the ingress layer.
  • Optional platform-managed services from the same Datalab claim: PostgreSQL databases, MongoDB document stores, Redis key-value/cache stores, Qdrant vector stores, and a Docker registry.

For end users, this means a simple workspace experience: they can open a familiar online IDE, access storage and credentials that have already been wired in, and work with higher-level services without understanding every underlying Kubernetes resource. For software engineers, it means the platform contract is declarative and reviewable instead of a long checklist of manual setup steps.

Features

  • PaaS-style service abstraction Offer online IDEs, storage access, databases, caches, vector stores, and registries through one Kubernetes resource.
  • Operator-visible provisioning Keep generated resources inspectable and governable instead of burying durable state inside user sessions.
  • Multi-tenant runtime isolation Run each Datalab inside a namespace or, where useful, with a dedicated virtual Kubernetes control plane (vCluster).
  • Integrated or delegated identity Use Keycloak-managed workspace access where appropriate, or set auth.type: delegated and delegate authentication to the ingress layer. Generated Datalab clients are confidential, include a service-account-only ws_api role for automation, and can add configured service audiences to access tokens.
  • Storage integration Consume object-storage credentials from Provider Storage or another storage process, and mount them into the lab.
  • Extensible by design Built on Crossplane, ready to connect additional operator-owned services without changing the user-facing API.

Installation

To install the configuration package into your Crossplane environment, e.g. based on Educates, use:

apiVersion: pkg.crossplane.io/v1
kind: Configuration
metadata:
  name: datalab-educates
spec:
  package: ghcr.io/versioneer-tech/provider-datalab/educates0.10.0-rc.1
  skipDependencyResolution: true

Quickstart

Minimal Example

apiVersion: pkg.internal/v1beta2
kind: Datalab
metadata:
  name: team-wonderland
spec:
  users:
  - alice
  sessions:
  - name: default
  vcluster: true

This provisions a vCluster within a dedicated Kubernetes namespace and starts the Educates tooling stack (including VS Code Server and a terminal), together with bundled utilities. The declared session also gets a durable workspace PVC that remains available if the session is later set to state: stopped. A storage browser is available with storage automatically mounted, and additional tools such as awscli and rclone are preinstalled to support typical data lab tasks like coding, data exploration, and wrangling.

Access to the datalab is intended for Alice, since she currently is the only user associated with this lab. Depending on the platform configuration, access can be enforced by Keycloak-managed resources or by delegated ingress authentication.

Combined with a small, cluster-specific EnvironmentConfig (realm, ingress domain/class, storage secret), the platform handles the rest—provisioning the chosen runtime, mounting credentials, and preloading content.

The same claim can also request stateful platform services:

spec:
  databases:
    pg0:
      names:
      - analytics
      storage: 1Gi
      backupStorage: 3Gi
  documentStores:
    prod:
      storage: 1Gi
  cacheStores:
    prod:
      storage: 1Gi
  vectorStores:
    prod:
      storage: 1Gi
  registry:
    enabled: true
    storage: 3Gi

Those resources are provisioned through the platform's installed operators and stay visible as managed infrastructure. That is what lets the operator decide how they are backed up, monitored, upgraded, and retired.

Note

The datalab-educates configuration package uses the shared Datalab Composite Resource Definition.

More Examples

Check the examples folder in the GitHub repository for complete scenarios, including:

  • Datalabs with multiple users
  • Datalabs with integrated storage
  • Identity-aware environments
  • Datalabs with managed databases, document stores, cache stores, vector stores, and registries