Skip to main content

Deploy with OpenTofu

The aperturedata/aperturedb-terraform package is an OCI module for OpenTofu. It creates native Kubernetes resources for distributed ApertureDB in an existing cluster. The default topology has three database pods, three ZooKeeper pods, two database gateways, two HTTP proxies, one checkpointer, a Web UI, and telemetry.

Prerequisites​

  • OpenTofu 1.10 or later for OCI module sources. Use tofu for the OCI source shown below, even though the package name contains terraform.

  • Kubernetes 1.23 or later and a kubeconfig for the intended cluster.

  • An object storage bucket and credentials; see Deploying ApertureDB.

  • Persistent volume provisioning for ZooKeeper and a 100Gi monitoring volume when telemetry is enabled.

  • Sufficient cluster capacity: each database pod and the checkpointer requests 8Gi of memory by default, and each Lenz pod requests 1Gi, in addition to the other components.

  • LoadBalancer support for the example below.

Configure the Kubernetes provider. The HTTP service is included; no ingress controller or Helm provider is needed. Use a separate namespace per deployment because application resource names are fixed within a namespace.

Configure the module​

In a new directory, create main.tf. Replace <package-version> with the published package tag you want, and replace the bucket name and hostname. This example references secrets you create separately in the namespace:

terraform {
  required_version = ">= 1.10.0"
  required_providers {
    kubernetes = {
      source  = "hashicorp/kubernetes"
      version = "~> 2.30"
    }
  }
}

provider "kubernetes" {
  config_path = "~/.kube/config"
}

module "aperturedb" {
  source = "oci://registry-1.docker.io/aperturedata/aperturedb-terraform?tag=<package-version>"

  namespace        = "aperturedb"
  create_namespace = false
  master_key       = { existing_secret = "adb-master-key" }

  storage = {
    type            = "s3"
    bucket          = "<your-bucket-name>"
    existing_secret = "adb-storage"
  }

  service = { type = "LoadBalancer" }
  http = {
    host       = "aperturedb.example.com"
    tls        = true
  }

  telemetry = {
    enabled = true
    auth    = { existing_secret = "adb-monitoring" }
    grafana = { existing_secret = "adb-grafana" }
  }
}

output "database_load_balancer" {
  value = module.aperturedb.lenz_external_load_balancer
}

output "http_load_balancer" {
  value = module.aperturedb.http_load_balancer
}

Create the namespace and secrets​

Create the namespace if it does not already exist:

kubectl create namespace aperturedb

Create four local files: master-password.txt, monitoring-password.txt, grafana-password.txt, and storage-credentials.json. Each password file must contain a different strong password without a trailing newline. The JSON file contains the S3 credentials described in Deploying ApertureDB. Then run:

chmod 600 master-password.txt monitoring-password.txt grafana-password.txt storage-credentials.json
kubectl -n aperturedb create secret generic adb-master-key \
  --from-file=password=master-password.txt
kubectl -n aperturedb create secret generic adb-monitoring \
  --from-file=password=monitoring-password.txt
kubectl -n aperturedb create secret generic adb-grafana \
  --from-file=password=grafana-password.txt
kubectl -n aperturedb create secret generic adb-storage \
  --from-file=ADB_STORAGE_CREDENTIALS=storage-credentials.json

For Google Cloud Storage, set storage.type = "gs" and put the service account JSON in storage-credentials.json. The OpenTofu storage secret key is ADB_STORAGE_CREDENTIALS; the Helm chart uses a different key.

Alternatively, the module can create its namespace with create_namespace = true and create secrets from inline values: master_key.password, storage.credentials (a map containing ADB_STORAGE_CREDENTIALS), telemetry.auth.password, and telemetry.grafana.admin_password. Inline credentials are stored in OpenTofu state. Protect state, plan files, and local credential files, and keep them out of source control.

For private workload images, create a registry secret in the namespace and set image_pull_secrets = ["my-registry-secret"] in the module.

Configure TLS​

The module uses an existing TLS secret, inline certificate and key, cert-manager, or a generated self-signed certificate, in that order. To use a certificate covering your web and TCP hostnames:

kubectl -n aperturedb create secret tls adb-tls \
  --cert=cert.pem --key=key.pem

Add tls = { existing_secret = "adb-tls" } to the module. Other options are tls.cert and tls.key, or tls.cert_manager with a configured issuer. Without these settings, the self-signed fallback is for evaluation; clients must explicitly trust it or use an evaluation-only certificate verification override. The generated private key is stored in OpenTofu state.

Plan, apply, and connect​

Initialize the module and providers, review the plan, then apply it:

tofu init
tofu validate
tofu plan -out=aperturedb.tfplan
tofu apply aperturedb.tfplan
kubectl -n aperturedb get pods,svc,pvc
tofu output database_load_balancer
tofu output http_load_balancer

Point the DNS record for http.host at the address from http_load_balancer. Native clients use lenz_external_load_balancer. The TLS certificate must cover all hostnames used by web and native clients.

  • Web UI: https://aperturedb.example.com/.

  • REST API: https://aperturedb.example.com/api/.

  • Native clients: the lenz-external service’s external address on port 55555, with TLS enabled. In-cluster clients can use lenz on that port.

  • Grafana: https://aperturedb.example.com/grafana/ with telemetry enabled. Log in as admin using the Grafana secret’s password.

Use admin with the master key in the runnable client check at Connect and verify the deployment. See Deployment settings and process boundaries for server and gateway settings. Check pod events, PVC status, and container logs if startup does not complete. A pending external address requires checking LoadBalancer support; HTTP routing also depends on DNS and http.host.

NodePort access​

For direct access through cluster nodes, configure distinct, available HTTP and HTTPS NodePorts and allow them through your firewall:

http = {
  host = "aperturedb.example.com"
  service = {
    type            = "NodePort"
    http_node_port  = 30080
    https_node_port = 30443
  }
}

Point http.host at a reachable node. With these example ports, the Web UI is at https://aperturedb.example.com:30443/; append api/ or grafana/ for the API or dashboards. HTTP on port 30080 redirects to that HTTPS address. Choose ports from your cluster’s configured NodePort range, normally 30000–32767. To expose native clients through NodePort as well, set service.type to NodePort and inspect lenz-external for its allocated port.

Behind an external load balancer serving ports 80/443, use NodePort and set http.public_port to 443. Leave the HTTP and HTTPS NodePorts at zero for automatic allocation. Redirects and Grafana URLs use public port 443. Both database and HTTP Services are always created; use ClusterIP for private access.

Workflow routes and request limits​

Use http.routes to expose workflow HTTP services under the deployment’s web hostname. Each route specifies a path prefix, an in-cluster service name, and its HTTP port. For example:

http = {
  host = "aperturedb.example.com"
  routes = [{
    path = "/notebooks"
    host = "workflow-notebooks-ingresses"
    port = 8888
  }]
}

Paths must be unique and omit the trailing slash; /api and /grafana are reserved. Requests retain their path prefix and query string. WebSockets and streamed responses are supported, and route updates take effect automatically.

http.body_limit limits each request body to 256 MiB by default. Oversized requests receive 413. http.timeout_seconds controls upstream inactivity and defaults to 300 seconds; workflow routes and Grafana allow one hour between response reads. Active requests have up to five minutes to finish during shutdown. Long-lived sessions should reconnect after an update.

Valid replacements in the configured TLS Secret are loaded automatically. Existing connections can finish with the previous certificate; invalid replacements leave the last valid certificate active.

Customize, upgrade, and remove​

Common module settings include aperturedb.replicas, lenz.replicas, http.replicas, global.storage_class, and telemetry.persistence.storage_class. Telemetry is enabled by default with a fixed 100Gi monitoring volume; set telemetry = { enabled = false } to disable it. inactive = true scales application workloads to zero while retaining their resources.

Grafana, Loki, and Prometheus share the web Pod, alongside the Web UI when it is enabled. Promtail runs in each database Pod and forwards its logs to Loki. telemetry.retention_days controls metrics and log retention and defaults to 28 days.

The ?tag= value in source selects the module package and matching versions of all ApertureData service images.

Review the compatibility notes in Release Notes for your target version, then complete Changing a deployment. The following commands refresh dependencies and apply the configuration:

tofu init -upgrade
tofu plan -out=aperturedb.tfplan
tofu apply aperturedb.tfplan

init -upgrade refreshes provider selections within the configured constraints. Review changes to the provider lock file before applying them.

To remove the resources managed by this configuration, run tofu destroy and review its proposed deletions. It removes the active monitoring PVC, whose underlying data follows the storage reclaim policy. ZooKeeper PVCs created by StatefulSet claim templates remain when their namespace remains. The module does not delete the object storage bucket. In this example the namespace and separately created secrets also remain; if you instead let the module manage the namespace, destroying it deletes the namespace and its remaining resources, including retained claims.