Webhooks

Trigger pipelines via HTTP webhooks and receive lifecycle notifications.

Webhook triggers

Any pipeline can be triggered via a webhook endpoint instead of (or in addition to) cron scheduling.

Setup

1. Generate a webhook token for the pipeline:

   curl -X PUT http://localhost:8080/api/pipelines/{id} \
     -H "Content-Type: application/json" \
     -d '{"webhook_token": "my-secret-token"}'

2. Trigger the pipeline by POSTing to the webhook endpoint:

   curl -X POST http://localhost:8080/api/pipelines/{id}/webhook \
     -H "X-Webhook-Token: my-secret-token" \
     -H "Content-Type: application/json" \
     -d '{"params": {"source": "github"}}'

Authentication

Webhook endpoints require a valid X-Webhook-Token header that matches the pipeline's webhook_token. Requests without a valid token receive a 403 Forbidden response.

Token security: Treat webhook tokens like API keys. Use a strong random string and rotate periodically.

Payload format

The request body is optional. If provided, it can include runtime parameters:

{
  "params": {
    "date": "2024-01-15",
    "source": "external-system"
  }
}

Parameters are available in node configs via ${param.key}.

Use cases

• Git push hooks: Trigger a pipeline when code is pushed to a repository
• External events: Run a pipeline when an external system completes a process
• Manual triggers: Give external teams a simple URL to start a pipeline

Lifecycle hooks

Pipelines can fire HTTP webhooks at key lifecycle events. Configure hooks in the pipeline's hooks field:

{
  "hooks": {
    "on_start": {
      "type": "webhook",
      "url": "https://hooks.slack.com/services/T.../B.../xxx",
      "enabled": true
    },
    "on_success": {
      "type": "webhook",
      "url": "https://api.example.com/notify",
      "enabled": true
    },
    "on_failure": {
      "type": "webhook",
      "url": "https://api.pagerduty.com/incidents",
      "enabled": true
    }
  }
}

Hook events

Hook payload

Each hook POSTs a JSON payload:

{
  "event": "on_failure",
  "pipeline_id": "abc-123",
  "pipeline": "Daily ETL",
  "run_id": "run-456",
  "timestamp": "2024-01-15T06:00:02Z",
  "error": "node Transform (transform1) failed: filter condition invalid"
}

The error field is only present in failure events.

Timeout and reliability

• Hooks have a 10-second timeout
• Failed hooks are logged but do not affect the pipeline run
• Hooks fire asynchronously and do not block pipeline execution

Combining webhooks with scheduling

A pipeline can have both a cron schedule and a webhook token. The schedule runs automatically, and the webhook provides on-demand triggering:

{
  "schedule": "0 6 * * *",
  "webhook_token": "trigger-on-demand-too"
}