Skip to main content

Deploy with OpenTofu

The aperturedata/aperturedb-terraform package is an OCI module for OpenTofu. It creates native Kubernetes resources for distributed ApertureDB in an existing cluster. The default topology has three database pods, three ZooKeeper pods, two database gateways, two HTTP proxies, one checkpointer, a Web UI, and telemetry.

Prerequisites​

  • OpenTofu 1.10 or later for OCI module sources. Use tofu for the OCI source shown below, even though the package name contains terraform.

  • Kubernetes 1.23 or later and a kubeconfig for the intended cluster.

  • An object storage bucket and credentials; see Deploying ApertureDB.

  • Persistent volume provisioning for ZooKeeper and a 100Gi monitoring volume when telemetry is enabled.

  • Sufficient cluster capacity: each database pod and the checkpointer requests 8Gi of memory by default, and each Lenz pod requests 1Gi, in addition to the other components.

  • LoadBalancer support for the example below.

Configure the Kubernetes provider. The HTTP service is included; no ingress controller or Helm provider is needed. Use a separate namespace per deployment because application resource names are fixed within a namespace.

Configure the module​

In a new directory, create main.tf. Replace <package-version> with the published package tag you want, and replace the bucket name and hostname. This example references secrets you create separately in the namespace:

terraform {
required_version = ">= 1.10.0"
required_providers {
kubernetes = {
source = "hashicorp/kubernetes"
version = "~> 2.30"
}
}
}

provider "kubernetes" {
config_path = "~/.kube/config"
}

module "aperturedb" {
source = "oci://registry-1.docker.io/aperturedata/aperturedb-terraform?tag=<package-version>"

namespace = "aperturedb"
create_namespace = false
master_key = { existing_secret = "adb-master-key" }

storage = {
type = "s3"
bucket = "<your-bucket-name>"
existing_secret = "adb-storage"
}

service = { type = "LoadBalancer" }
http = {
host = "aperturedb.example.com"
tls = true
}

telemetry = {
enabled = true
auth = { existing_secret = "adb-monitoring" }
grafana = { existing_secret = "adb-grafana" }
}
}

output "database_load_balancer" {
value = module.aperturedb.lenz_external_load_balancer
}

output "http_load_balancer" {
value = module.aperturedb.http_load_balancer
}

Create the namespace and secrets​

Create the namespace if it does not already exist:

kubectl create namespace aperturedb

Create four local files: master-password.txt, monitoring-password.txt, grafana-password.txt, and storage-credentials.json. Each password file must contain a different strong password without a trailing newline. The JSON file contains the S3 credentials described in Deploying ApertureDB. Then run:

chmod 600 master-password.txt monitoring-password.txt grafana-password.txt storage-credentials.json
kubectl -n aperturedb create secret generic adb-master-key \
--from-file=password=master-password.txt
kubectl -n aperturedb create secret generic adb-monitoring \
--from-file=password=monitoring-password.txt
kubectl -n aperturedb create secret generic adb-grafana \
--from-file=password=grafana-password.txt
kubectl -n aperturedb create secret generic adb-storage \
--from-file=ADB_STORAGE_CREDENTIALS=storage-credentials.json

For Google Cloud Storage, set storage.type = "gs" and put the service account JSON in storage-credentials.json. The OpenTofu storage secret key is ADB_STORAGE_CREDENTIALS; the Helm chart uses a different key.

Alternatively, the module can create its namespace with create_namespace = true and create secrets from inline values: master_key.password, storage.credentials (a map containing ADB_STORAGE_CREDENTIALS), telemetry.auth.password, and telemetry.grafana.admin_password. Inline credentials are stored in OpenTofu state. Protect state, plan files, and local credential files, and keep them out of source control.

For private workload images, create a registry secret in the namespace and set image_pull_secrets = ["my-registry-secret"] in the module.

Configure TLS​

The module uses an existing TLS secret, inline certificate and key, cert-manager, or a generated self-signed certificate, in that order. To use a certificate covering your web and TCP hostnames:

kubectl -n aperturedb create secret tls adb-tls \
--cert=cert.pem --key=key.pem

Add tls = { existing_secret = "adb-tls" } to the module. Other options are tls.cert and tls.key, or tls.cert_manager with a configured issuer. Without these settings, the self-signed fallback is for evaluation; clients must explicitly trust it or use an evaluation-only certificate verification override. The generated private key is stored in OpenTofu state.

Plan, apply, and connect​

Initialize the module and providers, review the plan, then apply it:

tofu init
tofu validate
tofu plan -out=aperturedb.tfplan
tofu apply aperturedb.tfplan
kubectl -n aperturedb get pods,svc,pvc
tofu output database_load_balancer
tofu output http_load_balancer

Point the DNS record for http.host at the address from http_load_balancer. Native clients use lenz_external_load_balancer. The TLS certificate must cover all hostnames used by web and native clients.

  • Web UI: https://aperturedb.example.com/.

  • REST API: https://aperturedb.example.com/api/.

  • Native clients: the lenz-external service’s external address on port 55555, with TLS enabled. In-cluster clients can use lenz on that port.

  • Grafana: https://aperturedb.example.com/grafana/ with telemetry enabled. Log in as admin using the Grafana secret’s password.

Use admin with the master key in the runnable client check at Connect and verify the deployment. See Deployment settings and process boundaries for server and gateway settings. Check pod events, PVC status, and container logs if startup does not complete. A pending external address requires checking LoadBalancer support; HTTP routing also depends on DNS and http.host.

NodePort access​

For direct access through cluster nodes, configure distinct, available HTTP and HTTPS NodePorts and allow them through your firewall:

http = {
host = "aperturedb.example.com"
service = {
type = "NodePort"
http_node_port = 30080
https_node_port = 30443
}
}

Point http.host at a reachable node. With these example ports, the Web UI is at https://aperturedb.example.com:30443/; append api/ or grafana/ for the API or dashboards. HTTP on port 30080 redirects to that HTTPS address. Choose ports from your cluster’s configured NodePort range, normally 30000–32767. To expose native clients through NodePort as well, set service.type to NodePort and inspect lenz-external for its allocated port.

Behind an external load balancer serving ports 80/443, use NodePort and set http.public_port to 443. Leave the HTTP and HTTPS NodePorts at zero for automatic allocation. Redirects and Grafana URLs use public port 443. Both database and HTTP Services are always created; use ClusterIP for private access.

Workflow routes and request limits​

Use http.routes to expose workflow HTTP services under the deployment’s web hostname. Each route specifies a path prefix, an in-cluster service name, and its HTTP port. For example:

http = {
host = "aperturedb.example.com"
routes = [{
path = "/notebooks"
host = "workflow-notebooks-ingresses"
port = 8888
}]
}

Paths must be unique and omit the trailing slash; /api and /grafana are reserved. Requests retain their path prefix and query string. WebSockets and streamed responses are supported, and route updates take effect automatically.

http.body_limit limits each request body to 256 MiB by default. Oversized requests receive 413. http.timeout_seconds controls upstream inactivity and defaults to 300 seconds; workflow routes and Grafana allow one hour between response reads. Active requests have up to five minutes to finish during shutdown. Long-lived sessions should reconnect after an update.

Valid replacements in the configured TLS Secret are loaded automatically. Existing connections can finish with the previous certificate; invalid replacements leave the last valid certificate active.

Settings reference​

Set only the attributes you want to change in the module block. Omitted attributes of an object keep their defaults, so zookeeper = { min_session_timeout = 30000 } keeps the other ZooKeeper settings.

Network​

Network settings

Attribute

Default

Purpose

service.type

LoadBalancer

Type of the lenz-external native database Service: LoadBalancer, NodePort, or ClusterIP.

service.port

55555

Published native database port.

service.node_port

0

Fixed node port when service.type is NodePort; 0 assigns one automatically.

service.annotations, service.load_balancer_class, service.load_balancer_source_ranges

Empty

Load balancer annotations, class, and allowed client CIDRs for lenz-external.

http.host

None; required

Web hostname used for redirects, Grafana URLs, and generated certificates.

http.extra_hosts

[]

Additional web hostnames, also included in generated and cert-manager certificates.

http.tls

true

Serve HTTPS and redirect HTTP to it; false serves HTTP only.

http.public_port

0

Public web port behind an external load balancer; 0 derives it from the Service.

http.body_limit

268435456

Maximum HTTP request body size in bytes.

http.timeout_seconds

300

Upstream inactivity timeout for API and Web UI requests.

http.routes

[]

Workflow routes; see Workflow routes and request limits.

http.service.type

LoadBalancer

Type of the http-proxy Service.

http.service.http_node_port, http.service.https_node_port

0, 0

Fixed HTTP and HTTPS node ports; see NodePort access.

http.service.annotations, http.service.load_balancer_class, http.service.load_balancer_source_ranges

Empty

Load balancer annotations, class, and allowed client CIDRs for http-proxy.

Native client connections always require TLS; the module has no equivalent of the Compose LNZ_FORCE_SSL setting.

Credentials, TLS, and images​

Credential, TLS, and image settings

Attribute

Default

Purpose

namespace

None; required

Namespace for all resources.

create_namespace

false

Create the namespace and delete it on tofu destroy.

master_key.password, master_key.existing_secret

None; one required

Database admin password, inline or from an existing Secret.

storage.type

s3

Object storage provider: s3 or gs.

storage.bucket

""

Object storage bucket for database content.

storage.credentials, storage.existing_secret

{}, ""

Storage credentials, as a map containing ADB_STORAGE_CREDENTIALS or from an existing Secret.

storage.extra_env

{}

Additional non-secret storage variables for the database, checkpointer, and tools pods.

tls.existing_secret

""

Existing kubernetes.io/tls Secret for web and native TLS.

tls.cert, tls.key

"", ""

Inline PEM certificate chain and private key.

tls.cert_manager.enabled, tls.cert_manager.issuer_ref.name, tls.cert_manager.issuer_ref.kind

false, "", ClusterIssuer

Issue the certificate with cert-manager.

image_pull_secrets

[]

Names of existing registry Secrets for private images.

registry_credentials.enabled, registry_credentials.registry, registry_credentials.username, registry_credentials.password

false, Docker Hub, "", ""

Create the aperturedb-registry pull Secret from inline credentials.

image.pull_policy

IfNotPresent

Pull policy for ApertureData images.

See Create the namespace and secrets and Configure TLS for Secret keys and certificate precedence. The ?tag= value in source selects the module package and matching versions of all ApertureData service images.

Capacity and logging​

Capacity and logging settings

Attribute

Default

Purpose

aperturedb.max_concurrency

512

Database concurrency per gateway process. Each database accepts this many clients plus 8 for internal clients.

aperturedb.task_executor_threads

100

Database executor threads.

aperturedb.storage_max_gi

4096

Graph-store storage limit in GiB; 0 means unlimited.

aperturedb.storage_operations_limit

100

Concurrent object storage operations per database.

aperturedb.descriptorset_flusher_interval

1800

Seconds between descriptor set cleanup operations.

aperturedb.session_token_max_age_sec

36000

Session token lifetime in seconds.

aperturedb.refresh_token_max_age_sec

360000

Refresh token lifetime in seconds.

aperturedb.health_check

true

Enable the database health port and use it for readiness checks.

aperturedb.debuggable

false

Database debug mode.

lenz.timeout_factor

4

Scales the gateway’s network timeouts; raise it for slow clients or very large requests.

aperturedb.resources

Requests 10m CPU, 8Gi; limits 64 CPU, 128Gi

Resources per database pod.

checkpointer.resources

Requests 10m CPU, 8Gi; limits 64 CPU, 128Gi

Resources for the checkpointer pod.

lenz.resources

Requests 10m CPU, 1Gi; limits 64 CPU, 32Gi

Resources per gateway pod.

http.resources

Requests 10m CPU, 64Mi; memory limit 512Mi

Resources per HTTP proxy pod.

aperturedb.extra_env, checkpointer.extra_env, lenz.extra_env

{}

Additional environment variables for that container, such as ADB_HTTPS_* settings.

lenz.workflows_secret

""

Existing Secret mounted into each gateway at /etc/lenz/workflows.

zookeeper.tick_time

8000

ZooKeeper tick, in milliseconds, from 1 to 10000.

zookeeper.min_session_timeout

20000

Lowest ZooKeeper session timeout, in milliseconds. A database that loses ZooKeeper for longer loses its session.

The module sets the other database settings listed for Compose, including ADB_LOG_LEVEL=0 and ADB_KVGD_DB_SYNC=false. See Configuration File for database configuration details.

Workloads and storage​

Workload and storage settings

Attribute

Default

Purpose

aperturedb.replicas

3

Database pods.

lenz.replicas

2

Gateway pods.

http.replicas

2

HTTP proxy pods.

checkpointer.replicas

1

Checkpointer pods.

zookeeper.replicas

3

ZooKeeper pods. Use an odd number.

inactive

false

Scale all workloads to zero while keeping their resources and volumes.

global.storage_class

""

Default storage class for all volumes; empty uses the cluster default.

zookeeper.persistence.size, zookeeper.persistence.storage_class

1Gi, ""

Volume size and storage class per ZooKeeper pod.

zookeeper.prefix

db_root

ZooKeeper path for this deployment’s coordination data.

aperturedb.node_selector, checkpointer.node_selector, lenz.node_selector, http.node_selector, zookeeper.node_selector, webui.node_selector

{}

Node labels that each component’s pods require.

webui.enabled

true

Deploy the Web UI.

tools.enabled, tools.replicas

true, 0

Administration pod; scale it up when needed.

Monitoring​

Monitoring settings

Attribute

Default

Purpose

telemetry.enabled

true

Deploy Prometheus, Grafana, Loki, and Promtail log collection.

telemetry.retention_days

28

Metrics and log retention in days.

telemetry.auth.password, telemetry.auth.existing_secret

None; one required with telemetry

Monitoring password used to collect metrics, separate from the master key.

telemetry.grafana.admin_password, telemetry.grafana.existing_secret

None; one required with telemetry

Grafana admin password.

telemetry.persistence.storage_class

""

Storage class for the shared 100Gi monitoring volume.

Grafana, Loki, and Prometheus share the web Pod, alongside the Web UI when it is enabled. Promtail runs in each database Pod and forwards its logs to Loki.

Upgrade and remove​

Review the compatibility notes in Release Notes for your target version, then complete Changing a deployment. The following commands refresh dependencies and apply the configuration:

tofu init -upgrade
tofu plan -out=aperturedb.tfplan
tofu apply aperturedb.tfplan

init -upgrade refreshes provider selections within the configured constraints. Review changes to the provider lock file before applying them.

To remove the resources managed by this configuration, run tofu destroy and review its proposed deletions. It removes the active monitoring PVC, whose underlying data follows the storage reclaim policy. ZooKeeper PVCs created by StatefulSet claim templates remain when their namespace remains. The module does not delete the object storage bucket. In this example the namespace and separately created secrets also remain; if you instead let the module manage the namespace, destroying it deletes the namespace and its remaining resources, including retained claims.