Python SDK
Python SDK Installation
Install the Brokoli Python SDK to define pipelines as code.
Install the Brokoli Python SDK to define pipelines as code.
Requirements
- Python 3.9 or later
- A running Brokoli server (for deployment)
Install
pip install brokoliVerify
import brokoli
print(brokoli.__version__)Or from the command line:
brokoli --versionCLI commands
The SDK includes a CLI for managing pipelines:
| Command | Description |
|---|---|
brokoli deploy | Deploy a pipeline to the server |
brokoli validate | Validate a pipeline file without deploying |
brokoli compile | Output pipeline JSON to stdout |
brokoli export | Export pipeline JSON to a file |
Configuration
Set the server URL via environment variable or CLI flag:
export BROKOLI_SERVER=http://localhost:8080
export BROKOLI_API_KEY=brk_your_key_hereOr per-command:
brokoli deploy my_pipeline.py --server http://localhost:8080 --api-key brk_xxxOptional dependencies
For Code nodes that process large datasets, install these for faster data transfer:
pip install pyarrow pandasBrokoli auto-detects pyarrow and pandas at runtime and uses them for file-based data transfer when datasets exceed 10,000 rows.
Next steps
- Pipeline Class -- build pipelines with the
Pipelineclass - Decorators -- use
@taskand@conditiondecorators - Examples -- complete pipeline examples