Configuration

All configuration options for Brokoli.

Environment variables

Database configuration

SQLite (default)

SQLite is the default. No setup needed.

broked serve --db ./broked.db

Or set via environment:

export BROKOLI_DB=./broked.db

When to use SQLite: Single-user, development, small teams, low-to-medium pipeline volumes.

PostgreSQL

For production deployments with multiple users or high pipeline volumes:

broked serve --db "postgres://user:password@localhost:5432/brokoli?sslmode=disable"

Create the database first:

CREATE DATABASE brokoli;
CREATE USER brokoli WITH PASSWORD 'your-password';
GRANT ALL PRIVILEGES ON DATABASE brokoli TO brokoli;

Brokoli creates all required tables automatically on first start.

When to use PostgreSQL: Production, multiple users, high-volume pipelines, when you need concurrent writes.

Connection string format

postgres://user:password@host:port/dbname?sslmode=disable

SSL modes: disable, require, verify-ca, verify-full.

CORS

By default, CORS allows all origins (development mode). In production, restrict to your domain:

export BROKOLI_CORS_ORIGINS=https://brokoli.example.com,https://admin.example.com

Rate limiting

Built-in rate limiting: 200 requests/second per IP for API endpoints, 10 requests/second for authentication endpoints.

Rate limiting is automatic and not configurable. WebSocket connections and health checks are excluded.

Request timeout

API requests have a 60-second default timeout. Long-running operations (pipeline runs, dry-runs, backfills) are excluded from the timeout.

Encryption

Connection passwords and secret variables are encrypted with AES-256-GCM.

• The key is stored in {db_path}.key (e.g., broked.db.key)
• Generated automatically on first run
• 32 bytes (256-bit)

Warning: Back up the .key file alongside your database. Without it, encrypted secrets cannot be recovered.

Concurrency

export BROKOLI_MAX_CONCURRENT_RUNS=8

Controls how many pipelines can run simultaneously. When at capacity, new runs queue until a slot opens.

Within each run, up to 4 nodes execute in parallel (independent branches of the DAG).

Logging

Brokoli logs to stdout. Each request is logged with method, path, status code, and duration:

2024/01/15 06:00:00 POST /api/pipelines/abc/run 202 12ms
2024/01/15 06:00:02 Scheduled run triggered for pipeline abc

For structured logging, pipe stdout to a log aggregator.

CLI flags vs environment variables

CLI flags take precedence over environment variables:

Production checklist

• Set BROKOLI_DATA_DIRS to restrict file access
• Set BROKOLI_CORS_ORIGINS to your domain
• Set BROKOLI_JWT_SECRET explicitly (don't rely on auto-generated)
• Use PostgreSQL for multi-user setups
• Back up broked.db.key and .brokoli-jwt-secret
• Run behind a reverse proxy with TLS
• Create admin user immediately after first start
• Set BROKOLI_MAX_CONCURRENT_RUNS based on available CPU/memory