This is the multi-page printable view of this section. Click here to print.

Return to the regular view of this page.

Configuration

1: Configure OpenTelemetry for SonataFlow Workflows
2: Configure Log Aggregation for Serverless Workflows
3: Make workflow able to use authentication request on run
4: Configure workflow for token exchange
5: Configure workflow for token propagation

1 - Configure OpenTelemetry for SonataFlow Workflows

Configure OpenTelemetry for SonataFlow Workflows

This document provides comprehensive guidance for enabling and configuring OpenTelemetry observability in SonataFlow workflows deployed on OpenShift/Kubernetes clusters. You’ll learn how to configure your workflows for distributed tracing, metrics collection, and log aggregation using industry-standard observability tools.

Prerequisites

SonataFlow Operator installed and configured
Kubernetes/OpenShift cluster with appropriate resources
Access to deploy observability infrastructure
Basic understanding of OpenTelemetry concepts

Overview

The OpenTelemetry integration for SonataFlow provides:

Distributed Tracing: Track workflow execution across multiple services and steps
Metrics Collection: Monitor performance, duration, and success rates
Log Aggregation: Centralized logging with trace correlation
Context Propagation: Maintain trace context across workflow boundaries and async operations

Note: The OpenTelemetry feature is available in SonataFlow runtime through the sonataflow-addons-quarkus-opentelemetry addon, which provides comprehensive observability capabilities with minimal configuration overhead.

Architecture Overview

SonataFlow workflows emit telemetry data using the OpenTelemetry Protocol (OTLP) to collectors or directly to observability platforms:

┌──────────────────────┐
│ SonataFlow Workflows │
│  (Quarkus + OTEL)    │
└─────────┬────────────┘
          │ OTLP (gRPC/HTTP)
          │
    ┌─────┴─────┐
    │           │
    │ Traces    │ Logs
    │ :4317     │ :3100/otlp
    ▼           ▼
┌─────────┐ ┌─────────┐
│ Jaeger  │ │  Loki   │
│(Tracing)│ │ (Logs)  │
└─────────┘ └─────────┘
    │           │
    └─────┬─────┘
          │
          ▼
    ┌─────────┐
    │ Grafana │
    │(Visualize)│
    └─────────┘

Optional: OpenTelemetry Collector
for advanced processing, filtering,
and multi-backend export

Configuration

Enable OpenTelemetry Extension

To enable OpenTelemetry in your SonataFlow workflow, add the extension to your project:

1. Add Extension Dependency

Add the SonataFlow OpenTelemetry addon in the QUARKUS_EXTENSIONS environment variable when building the workflow image:

export QUARKUS_EXTENSIONS="${QUARKUS_EXTENSIONS},org.apache.kie.sonataflow:sonataflow-addons-quarkus-opentelemetry"

2. Configure Workflow Properties

The following properties must be configured in the ConfigMap that holds your workflow’s application.properties. When a SonataFlow CR is deployed, it automatically generates a {workflow-name}-props ConfigMap that contains these properties.

Basic OpenTelemetry Configuration

Add the following properties to your workflow’s application.properties:

# Application Identity
quarkus.application.name=my-workflow
quarkus.application.version=1.0.0

# OpenTelemetry Configuration
quarkus.otel.enabled=true
quarkus.otel.traces.enabled=true
quarkus.otel.metrics.enabled=true
quarkus.otel.logs.enabled=true

# Service Resource Attributes
quarkus.otel.resource.attributes=\
  service.name=my-workflow,\
  service.namespace=workflows,\
  service.version=1.0.0,\
  deployment.environment=production

# SonataFlow Specific Configuration
# Master switch for SonataFlow OpenTelemetry integration
sonataflow.otel.enabled=true
# Service identification (uses Quarkus application name/version as defaults)
sonataflow.otel.service-name=${quarkus.application.name:kogito-workflow-service}
sonataflow.otel.service-version=${quarkus.application.version:unknown}
# Enable span creation for workflow states
sonataflow.otel.spans.enabled=true
# Enable process lifecycle events (start, complete, error, state transitions)
sonataflow.otel.events.enabled=true

Note: The sonataflow.otel.enabled property controls SonataFlow-specific instrumentation. This works in conjunction with quarkus.otel.enabled which controls the underlying Quarkus OpenTelemetry integration. Both should be enabled for full observability.

# Context Propagation
quarkus.otel.propagators=tracecontext,baggage,jaeger

# SonataFlow-specific context headers (in addition to standard OpenTelemetry headers)
# X-TRANSACTION-ID: Correlates all workflow executions within a business transaction
# X-TRACKER-*: Custom tracking headers for additional context (e.g., X-TRACKER-CORRELATION-ID)

# Instrumentation
quarkus.datasource.jdbc.telemetry=true
quarkus.otel.instrument.rest=true
quarkus.otel.instrument.grpc=true

Exporter Configuration

Choose one of the following export strategies based on your observability platform:

Option 1: OTLP Exporter (Recommended)

# OTLP Exporter - Direct to Jaeger
quarkus.otel.exporter.otlp.endpoint=http://jaeger-collector.observability.svc.cluster.local:4317
quarkus.otel.exporter.otlp.protocol=grpc
quarkus.otel.traces.exporter=cdi

# Batch Processing for Production
quarkus.otel.bsp.schedule.delay=5s
quarkus.otel.bsp.max.export.batch.size=512
quarkus.otel.bsp.export.timeout=2s
quarkus.otel.bsp.max.queue.size=2048

Option 2: Direct Export to External Platform

# Example: Direct export to Jaeger
quarkus.otel.exporter.otlp.endpoint=http://jaeger-collector:4317
quarkus.otel.exporter.otlp.protocol=grpc
quarkus.otel.traces.exporter=cdi

Externalized Configuration

For production deployments, use environment variables to externalize configuration:

# Externalized Configuration
quarkus.otel.exporter.otlp.endpoint=${OTEL_EXPORTER_OTLP_ENDPOINT:http://localhost:4317}
quarkus.otel.exporter.otlp.headers=${OTEL_EXPORTER_OTLP_HEADERS:}
quarkus.application.name=${OTEL_SERVICE_NAME:my-workflow}
quarkus.otel.resource.attributes=${OTEL_RESOURCE_ATTRIBUTES:deployment.environment=dev}

Observability Tools Configuration

This section provides working examples for two essential observability tools that integrate seamlessly with SonataFlow’s OpenTelemetry implementation.

Tool 1: Jaeger Distributed Tracing

Jaeger provides distributed tracing visualization for SonataFlow workflows.

Deploy Jaeger on OpenShift/Kubernetes

All-in-One Deployment (Development/Testing):

apiVersion: v1
kind: Namespace
metadata:
  name: jaeger-system
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: jaeger
  namespace: jaeger-system
  labels:
    app: jaeger
spec:
  replicas: 1
  selector:
    matchLabels:
      app: jaeger
  template:
    metadata:
      labels:
        app: jaeger
    spec:
      containers:
      - name: jaeger
        image: jaegertracing/all-in-one:1.59
        env:
        - name: COLLECTOR_OTLP_ENABLED
          value: "true"
        ports:
        - containerPort: 16686
          name: query
        - containerPort: 4317
          name: otlp-grpc
        - containerPort: 4318
          name: otlp-http
        resources:
          requests:
            memory: "256Mi"
            cpu: "100m"
          limits:
            memory: "512Mi"
            cpu: "500m"
        readinessProbe:
          httpGet:
            path: /
            port: 14269
          initialDelaySeconds: 5
        livenessProbe:
          httpGet:
            path: /
            port: 14269
          initialDelaySeconds: 10
---
apiVersion: v1
kind: Service
metadata:
  name: jaeger-collector
  namespace: jaeger-system
  labels:
    app: jaeger
spec:
  selector:
    app: jaeger
  ports:
  - name: otlp-grpc
    port: 4317
    targetPort: 4317
  - name: otlp-http
    port: 4318
    targetPort: 4318
  type: ClusterIP
---
apiVersion: v1
kind: Service
metadata:
  name: jaeger-query
  namespace: jaeger-system
  labels:
    app: jaeger
spec:
  selector:
    app: jaeger
  ports:
  - name: query-http
    port: 16686
    targetPort: 16686
  type: ClusterIP

OpenShift Route for UI Access:

apiVersion: route.openshift.io/v1
kind: Route
metadata:
  name: jaeger-query
  namespace: jaeger-system
spec:
  to:
    kind: Service
    name: jaeger-query
  port:
    targetPort: query-http
  tls:
    termination: edge
    insecureEdgeTerminationPolicy: Redirect

Workflow Configuration for Jaeger:

# Direct connection to Jaeger
quarkus.otel.exporter.otlp.endpoint=http://jaeger-collector.jaeger-system.svc.cluster.local:4317
quarkus.otel.exporter.otlp.protocol=grpc
quarkus.otel.traces.exporter=cdi

# Additional Jaeger-specific propagation
quarkus.otel.propagators=tracecontext,baggage,jaeger

Production Deployment with Elasticsearch:

For production environments, use the Jaeger Operator with Elasticsearch storage:

apiVersion: jaegertracing.io/v1
kind: Jaeger
metadata:
  name: jaeger-production
  namespace: observability
spec:
  strategy: production
  storage:
    type: elasticsearch
    elasticsearch:
      nodeCount: 3
      storage:
        storageClassName: gp3
        size: 50Gi
      resources:
        requests:
          cpu: 500m
          memory: 4Gi
        limits:
          cpu: 1000m
          memory: 8Gi
  collector:
    replicas: 2
    resources:
      requests:
        cpu: 200m
        memory: 256Mi
      limits:
        cpu: 500m
        memory: 512Mi

Tool 2: Loki Log Aggregation

Grafana Loki provides scalable log aggregation with native OpenTelemetry support, enabling comprehensive log analysis and correlation with traces.

Loki natively supports OpenTelemetry Protocol (OTLP) for direct log ingestion from SonataFlow workflows.

Deploy Loki on OpenShift/Kubernetes

Loki Configuration for OpenTelemetry:

apiVersion: v1
kind: ConfigMap
metadata:
  name: loki-config
  namespace: observability
data:
  loki-config.yaml: |
    auth_enabled: false

    server:
      http_listen_port: 3100
      grpc_listen_port: 9096

    common:
      path_prefix: /loki
      storage:
        filesystem:
          chunks_directory: /loki/chunks
          rules_directory: /loki/rules
      replication_factor: 1
      ring:
        instance_addr: 127.0.0.1
        kvstore:
          store: inmemory

    distributor:
      otlp_config:
        # Default resource attributes as index labels
        default_resource_attributes_as_index_labels:
          - service.name
          - service.namespace
          - deployment.environment
          - k8s.namespace.name
          - k8s.cluster.name

    limits_config:
      # Enable structured metadata (default in Loki 3.0+)
      allow_structured_metadata: true
      # Maximum number of index labels per stream
      max_label_names_per_series: 15

    schema_config:
      configs:
        - from: 2024-01-01
          store: tsdb
          object_store: filesystem
          schema: v13  # Required for OTLP support
          index:
            prefix: index_
            period: 24h

Loki Deployment:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: loki
  namespace: observability
  labels:
    app: loki
spec:
  replicas: 1
  selector:
    matchLabels:
      app: loki
  template:
    metadata:
      labels:
        app: loki
    spec:
      securityContext:
        fsGroup: 10001
        runAsUser: 10001
        runAsNonRoot: true
      containers:
      - name: loki
        image: grafana/loki:3.0.0
        args:
          - -config.file=/etc/loki/loki-config.yaml
        ports:
        - containerPort: 3100
          name: http-metrics
        - containerPort: 9096
          name: grpc
        resources:
          requests:
            cpu: 500m
            memory: 1Gi
          limits:
            cpu: 1000m
            memory: 2Gi
        volumeMounts:
        - name: config
          mountPath: /etc/loki
        - name: storage
          mountPath: /loki
        livenessProbe:
          httpGet:
            path: /ready
            port: 3100
          initialDelaySeconds: 45
        readinessProbe:
          httpGet:
            path: /ready
            port: 3100
          initialDelaySeconds: 45
      volumes:
      - name: config
        configMap:
          name: loki-config
      - name: storage
        emptyDir: {}
---
apiVersion: v1
kind: Service
metadata:
  name: loki
  namespace: observability
  labels:
    app: loki
spec:
  selector:
    app: loki
  ports:
  - name: http-metrics
    port: 3100
    targetPort: 3100
  - name: grpc
    port: 9096
    targetPort: 9096
  type: ClusterIP

Workflow Configuration for Loki

Direct Connection to Loki (Recommended for simplicity):

# OpenTelemetry Configuration
quarkus.otel.enabled=true
quarkus.otel.traces.enabled=true
quarkus.otel.metrics.enabled=true
quarkus.otel.logs.enabled=true

# OTLP Exporter - Send logs to Loki, traces to Jaeger
quarkus.otel.exporter.otlp.logs.endpoint=http://loki.observability.svc.cluster.local:3100/otlp
quarkus.otel.exporter.otlp.traces.endpoint=http://jaeger-collector.observability.svc.cluster.local:4317
quarkus.otel.exporter.otlp.protocol=grpc

# JSON Logging for better structure
quarkus.log.console.json=true
quarkus.log.console.json.pretty-print=false

# Include trace correlation in logs
quarkus.log.console.format=%d{yyyy-MM-dd HH:mm:ss,SSS} %-5p [%c{3.}] (%t) traceId=%X{traceId}, spanId=%X{spanId} %s%e%n

# Resource attributes for Loki labels
quarkus.otel.resource.attributes=\
  service.name=greeting-workflow,\
  service.namespace=workflows,\
  deployment.environment=production

Optional: OpenTelemetry Collector for Advanced Processing

For production environments requiring log processing, enrichment, or multi-backend export, you can optionally deploy an OpenTelemetry Collector between your workflows and the observability backends.

Benefits of using a Collector:

Log enrichment with Kubernetes metadata
Filtering and sampling
Multi-destination export (e.g., logs to both Loki and external SIEM)
Centralized processing and transformation

Quick Collector Setup:

# Change workflow configuration to send to collector instead
quarkus.otel.exporter.otlp.endpoint=http://otel-collector.observability.svc.cluster.local:4317

Collector Configuration:

# Collector routes to both Jaeger and Loki
exporters:
  otlp/jaeger:
    endpoint: jaeger-collector:4317
  otlphttp/loki:
    endpoint: http://loki:3100/otlp

service:
  pipelines:
    traces:
      receivers: [otlp]
      processors: [batch]
      exporters: [otlp/jaeger]
    logs:
      receivers: [otlp]
      processors: [batch]
      exporters: [otlphttp/loki]

For complete collector deployment manifests, see the OpenTelemetry Collector documentation.

Generated Telemetry Data

When your SonataFlow workflow runs with OpenTelemetry enabled, it generates comprehensive observability data:

Trace Spans

SonataFlow automatically creates spans for workflow states. The spans are:

Grouped by workflow state - Each distinct workflow state gets its own span, reducing noise from individual node executions. Process start events are attached to the first state span, and process complete events are attached to the final state span.
Flat hierarchy for all workflow spans - All workflow spans (including subflows) share the same root span context and appear as flat siblings under the HTTP request span. This creates a cleaner trace visualization where subflow spans are NOT nested children of the subprocess invocation state, but rather siblings with the main workflow spans.
Named consistently - Span names follow the pattern sonataflow.process.<processId>.execute

Span Attributes

Each span includes these SonataFlow-specific attributes:

Attribute	Description	Example
`sonataflow.process.instance.id`	Unique workflow instance identifier	`greeting-abc123-456-789`
`sonataflow.process.id`	Workflow definition ID	`greeting`
`sonataflow.process.version`	Workflow version	`1.0.0`
`sonataflow.process.instance.state`	Current process state	`ACTIVE`, `COMPLETED`, `ERROR`
`sonataflow.workflow.state`	Current workflow state name	`ChooseOnLanguage`
`sonataflow.transaction.id`	Transaction ID from `X-TRANSACTION-ID` header or process instance ID	`tx-12345`
`service.name`	Service name from configuration	`greeting-workflow`
`service.version`	Service version from configuration	`1.0.0`

Custom Tracker Attributes: When X-TRACKER-* headers are provided, they appear as sonataflow.tracker.* attributes. For example, X-TRACKER-CORRELATION-ID: abc123 becomes sonataflow.tracker.correlation.id: abc123.

Process Lifecycle Events

The following events are automatically added to spans:

Event Name	Description	Attributes
`process.instance.start`	Workflow execution begins	`process.instance.id`, `trigger`, `reference.id`
`process.instance.complete`	Workflow execution ends	`process.instance.id`, `outcome`, `duration.ms`
`process.instance.error`	Workflow encounters an error	`process.instance.id`, `error.message`, `error.type`
`state.started`	Workflow state execution begins	`event.description`
`state.completed`	Workflow state execution ends	`event.description`
`log.message`	Application log during workflow execution	`level`, `logger`, `message`, `thread.name`, `thread.id`

Context Propagation via HTTP Headers

SonataFlow extracts and propagates context from these HTTP headers:

Header	Purpose	Example
`X-TRANSACTION-ID`	Correlate multiple workflow executions in a business transaction	`X-TRANSACTION-ID: order-tx-12345`
`X-TRACKER-*`	Custom tracking context (converted to span attributes)	`X-TRACKER-USER-ID: user123`

These headers are sanitized (max 100 characters, special characters removed) and stored as span attributes for correlation and debugging.

Metrics

Metrics are exported for:

Process duration and success rates
Node execution times
Error rates and types
Resource utilization

What’s More: Visualization and Monitoring

Once your telemetry data is flowing, you can leverage various visualization and monitoring approaches:

Jaeger UI Exploration

Access the Jaeger UI to:

Trace Search: Find traces by service, operation, or time range
Service Map: Visualize service dependencies and call patterns
Performance Analysis: Identify bottlenecks and latency issues
Error Investigation: Trace error propagation through workflow steps

Example Jaeger query for your workflow:

service:greeting-workflow operation:workflow.execute

Grafana Dashboards

Create dashboards to monitor:

Workflow Success Rate: Percentage of successful executions
Execution Duration: P50, P95, P99 latencies by workflow and node
Error Rate Trends: Track error patterns over time
Resource Utilization: Monitor workflow resource consumption

Loki Log Exploration

Grafana Explore Queries for SonataFlow logs:

# All logs from greeting workflow
{service_name="greeting-workflow"}

# Error logs only
{service_name="greeting-workflow"} |= "ERROR"

# Logs for specific workflow node
{service_name="greeting-workflow"} | json | node_name="ChooseOnLanguage"

# Logs with trace ID correlation
{service_name="greeting-workflow"} | json | trace_id != "" | line_format "{{.trace_id}}: {{.message}}"

# Count workflow executions by language choice
sum by (language) (count_over_time({service_name="greeting-workflow"} |~ "language.*English|Spanish" [5m]))

# Error rate by workflow
sum by (service_name) (rate({service_namespace="workflows"} |= "ERROR" [5m])) /
sum by (service_name) (rate({service_namespace="workflows"} [5m]))

# Workflow execution duration analysis
{service_name="greeting-workflow"}
  | json
  | duration_ms != ""
  | line_format "Duration: {{.duration_ms}}ms for {{.workflow_id}}"

# Correlation between traces and logs
{service_name="greeting-workflow"}
  | json
  | trace_id="your-trace-id-here"
  | line_format "{{.timestamp}} [{{.level}}] {{.message}}"

Example Loki Dashboard Panels:

Log Volume Timeline

sum by (service_name) (count_over_time({service_namespace="workflows"}[1m]))

Error Logs Table

{service_namespace="workflows"} |= "ERROR" | json

Workflow Success vs Failure

# Success
sum(count_over_time({service_name="greeting-workflow"} |~ "completed successfully" [5m]))

# Failures
sum(count_over_time({service_name="greeting-workflow"} |= "ERROR" [5m]))

Language Choice Distribution (Greeting-specific)

sum by (language) (count_over_time({service_name="greeting-workflow"} |~ "English|Spanish" [1h]))

Alerting

Set up alerts for:

High error rates in critical workflow steps
Unusual execution duration increases
Service dependency failures
Resource exhaustion patterns

Log Correlation

Correlate logs using trace IDs:

Search logs by trace ID to see complete execution context
Track data flow through complex workflow states
Debug issues with full observability context

Advanced Analysis

Use the collected data for:

Performance optimization - Identify slow workflow nodes
Capacity planning - Understand resource usage patterns
Business insights - Track workflow completion rates and user experience
Troubleshooting - Root cause analysis with full trace context

Troubleshooting

Common Issues

No Traces Appearing

Check workflow configuration:

# Verify OpenTelemetry is enabled
kubectl get cm onboarding-workflow-props -n workflows -o yaml

# Check pod logs for OpenTelemetry initialization
kubectl logs -n workflows deployment/onboarding-workflow | grep -i "otel\|trace"

Verify connectivity:

# Test Jaeger endpoint from workflow pod
kubectl exec -n workflows deployment/greeting -- curl -v http://jaeger-collector.observability.svc.cluster.local:4317

Authentication Issues

For platforms requiring authentication, configure headers:

quarkus.otel.exporter.otlp.headers=authorization=Bearer ${API_TOKEN}

Test authentication:

curl -v -X POST \
  -H "Authorization: Bearer YOUR_TOKEN" \
  -H "Content-Type: application/x-protobuf" \
  https://your-endpoint.com/v1/traces

High Memory Usage

Configure memory limits in collector:

processors:
  memory_limiter:
    check_interval: 1s
    limit_mib: 1000
    spike_limit_mib: 200

Context Lost Between Steps

Ensure proper propagation:

# Include all required propagators
quarkus.otel.propagators=tracecontext,baggage,jaeger

# Enable JSON logging to verify trace IDs
quarkus.log.console.json=true

Verification Steps

1. Check OpenTelemetry addon is loaded:

kubectl logs -n workflows deployment/onboarding-workflow | grep "sonataflow-addons-quarkus-opentelemetry"

2. Verify trace export:

# Look for successful trace exports
kubectl logs -n workflows deployment/greeting | grep -i "export\|batch"

3. Check Jaeger health:

# Verify Jaeger is receiving data
kubectl logs -n observability deployment/jaeger | grep -i "span\|trace"

4. Test with debug exporter:

# Temporarily enable console logging
%dev.quarkus.otel.traces.exporter=logging

Additional Resources

2 - Configure Log Aggregation for Serverless Workflows

Configure structured JSON logging and log aggregation for SonataFlow workflows to enable comprehensive monitoring, debugging, and observability across your workflow instances.

Prerequisites

OpenShift Container Platform 4.8+ or Kubernetes 1.21+
SonataFlow workflows deployed via SonataFlow Operator
Cluster admin permissions for deploying log aggregation stack
Basic knowledge of JSON logging and log aggregation tools

Overview

SonataFlow workflows support structured JSON logging with automatic process instance correlation through:

• Process Instance Context: Automatic processInstanceId correlation in all log entries • Structured Format: JSON logs optimized for machine processing and aggregation • Multi-tenancy Support: Log isolation by workflow and process instance

JSON Log Structure

When properly configured, workflow logs are emitted as JSON with the following structure:

{
  "timestamp": "2025-11-24T10:30:45.123Z",
  "level": "INFO",
  "loggerName": "org.kie.kogito.workflow.engine",
  "message": "Workflow step completed successfully",
  "threadName": "executor-thread-1",
  "mdc": {
    "processInstanceId": "abc-123-def-456"
  }
}

Configuration

Workflow Configuration

To enable JSON logging for your workflows, configure the following properties in the {workflow-name}-props ConfigMap:

# Enable JSON logging with Quarkus JSON logging extension
quarkus.log.console.json=true
quarkus.log.console.json.pretty-print=false

# Include all MDC context fields in JSON output
# - processInstanceId: Set automatically by SonataFlow/Kogito
# - traceId, spanId: Set by Quarkus OpenTelemetry (requires quarkus.otel.enabled=true)
quarkus.log.console.json.print-details=true

# Configure log levels for workflow components
quarkus.log.category."org.kie.kogito".level=DEBUG
quarkus.log.category."io.serverlessworkflow".level=INFO
quarkus.log.category."org.kie.kogito.process".level=INFO

# Optional: Enable additional context logging
quarkus.log.category."org.kie.kogito.services.context".level=DEBUG

Required Extension

Add the Quarkus JSON logging extension in the QUARKUS_EXTENSIONS environment variable when building the workflow image:

export QUARKUS_EXTENSIONS="${QUARKUS_EXTENSIONS},io.quarkus:quarkus-logging-json"

Note: This extension is required for JSON log formatting. Without it, logs will remain in plain text format.

File-Based JSON Logging

In some deployment scenarios, you may need to output logs to a file instead of (or in addition to) the console. This is useful when:

Using sidecar containers that read log files (e.g., Fluent Bit file input)
Integrating with legacy log collection systems that expect file-based logs
Debugging locally with persistent log files
Collecting logs from environments where stdout/stderr collection is limited

Basic File Logging Configuration

Add the following properties to enable JSON logging to a file:

# Enable file logging
quarkus.log.file.enable=true
quarkus.log.file.path=/var/log/sonataflow/workflow.log

# Enable JSON format for file output
quarkus.log.file.json=true
quarkus.log.file.json.pretty-print=false

# Include MDC context fields in JSON output
# - processInstanceId: Set automatically by SonataFlow/Kogito
# - traceId, spanId: Set by Quarkus OpenTelemetry (requires quarkus.otel.enabled=true)
quarkus.log.file.json.print-details=true

# Set log level for file output
quarkus.log.file.level=INFO

File Rotation Configuration

For production environments, configure log rotation to prevent disk space issues:

# Enable file logging with rotation
quarkus.log.file.enable=true
quarkus.log.file.path=/var/log/sonataflow/workflow.log

# JSON format
quarkus.log.file.json=true
quarkus.log.file.json.pretty-print=false
quarkus.log.file.json.print-details=true

# Rotation settings
quarkus.log.file.rotation.max-file-size=10M
quarkus.log.file.rotation.max-backup-index=5
quarkus.log.file.rotation.file-suffix=.yyyy-MM-dd
quarkus.log.file.rotation.rotate-on-boot=true

This configuration:

Rotates logs when they reach 10MB
Keeps up to 5 backup files
Adds date suffix to rotated files
Rotates on application startup

Combined Console and File Logging

You can enable both console and file JSON logging simultaneously:

# Console JSON logging (for container log collectors)
quarkus.log.console.json=true
quarkus.log.console.json.pretty-print=false
quarkus.log.console.json.print-details=true

# File JSON logging (for file-based collectors)
quarkus.log.file.enable=true
quarkus.log.file.path=/var/log/sonataflow/workflow.log
quarkus.log.file.json=true
quarkus.log.file.json.pretty-print=false
quarkus.log.file.json.print-details=true
quarkus.log.file.rotation.max-file-size=10M
quarkus.log.file.rotation.max-backup-index=5

Kubernetes Volume Configuration

When using file-based logging in Kubernetes, ensure the log directory is properly mounted:

apiVersion: sonataflow.org/v1alpha08
kind: SonataFlow
metadata:
  name: my-workflow
spec:
  podTemplate:
    container:
      volumeMounts:
      - name: logs
        mountPath: /var/log/sonataflow
    volumes:
    - name: logs
      emptyDir: {}

For persistent logs or sidecar collection, use a shared volume:

spec:
  podTemplate:
    container:
      volumeMounts:
      - name: shared-logs
        mountPath: /var/log/sonataflow
    initContainers: []
    volumes:
    - name: shared-logs
      emptyDir:
        sizeLimit: 500Mi

Validation

Before deploying log aggregation, verify that JSON logging is working correctly:

1. Check Workflow Pod Logs

After applying the configuration, restart your workflow pod and check the log output:

# Get workflow pod name
oc get pods -n sonataflow-infra -l sonataflow.org/workflow-app=your-workflow

# Check logs for JSON format
oc logs -n sonataflow-infra your-workflow-pod-name | head -5

Expected output (JSON format):

{"timestamp":"2025-11-24T10:30:45.123Z","level":"INFO","logger":"io.quarkus","message":"Profile prod activated","MDC":{}}

If you see plain text instead:

2025-11-24 10:30:45,123 INFO  [io.quarkus] Profile prod activated

This indicates the quarkus-logging-json extension is not properly included in your workflow image.

2. Verify MDC Context

Look for workflow-specific logs that include processInstanceId:

# Search for logs with process instance context
oc logs your-workflow-pod-name | grep processInstanceId

Expected: JSON logs containing "MDC":{"processInstanceId":"abc-123-..."}

If MDC fields are empty or missing: The JSON logging is working, but process context is not being set. This may indicate:

Workflow has not processed any instances yet
Custom MDC configuration may be needed depending on your SonataFlow version

3. Test with a Simple Workflow Execution

Trigger a workflow execution and verify the logs contain process correlation:

# Trigger workflow (method depends on your setup)
curl -X POST http://your-workflow-url/your-workflow

# Check logs immediately after
oc logs your-workflow-pod-name --tail=20 | grep processInstanceId

ConfigMap Example

Here’s a complete example of a workflow ConfigMap with JSON logging enabled:

apiVersion: v1
kind: ConfigMap
metadata:
  name: greetings-props
  namespace: sonataflow-infra
data:
  application.properties: |
    # JSON logging configuration
    quarkus.log.console.json=true
    quarkus.log.console.json.pretty-print=false
    quarkus.log.console.json.print-details=true

    # Log levels
    quarkus.log.category."org.kie.kogito".level=DEBUG
    quarkus.log.category."io.serverlessworkflow".level=INFO

Log Aggregation Solutions

Recommended Stack: Promtail + Loki + Grafana (PLG)

The PLG stack provides a modern, cloud-native solution optimized for Kubernetes environments.

💡 For detailed production deployment guides: See the comprehensive PLG Observability Documentation which includes complete Helm values, YAML manifests, dashboard configurations, and advanced query examples.

Architecture

SonataFlow Pods → Promtail (log collection) → Loki (storage) → Grafana (visualization)

Quick Deployment

For a quick start, deploy the PLG stack using Helm:

# Add Grafana Helm repository
helm repo add grafana https://grafana.github.io/helm-charts
helm repo update

# Create namespace
oc new-project sonataflow-observability

# Deploy Loki stack
helm install loki-stack grafana/loki-stack \
  --namespace sonataflow-observability \
  --set loki.persistence.enabled=true \
  --set loki.persistence.size=20Gi \
  --set promtail.config.logLevel=info \
  --set grafana.enabled=true

📋 For production deployment: Use the complete Helm values configuration with proper resource limits, security contexts, and OpenShift-specific settings.

Promtail Configuration

Configure Promtail to discover and parse SonataFlow logs. You can choose between scraping container stdout (default) or custom JSON log files.

Option A: Scrape Container Stdout (Default)

This configuration uses Kubernetes service discovery to collect logs from container stdout:

apiVersion: v1
kind: ConfigMap
metadata:
  name: promtail-config
  namespace: sonataflow-observability
data:
  config.yml: |
    server:
      http_listen_port: 3101

    clients:
      - url: http://loki:3100/loki/api/v1/push

    scrape_configs:
    - job_name: sonataflow-workflows
      kubernetes_sd_configs:
      - role: pod
        namespaces:
          names: ["sonataflow-infra"]

      relabel_configs:
      - source_labels: [__meta_kubernetes_pod_label_sonataflow_org_workflow_app]
        action: keep
        regex: (.+)

      - source_labels: [__meta_kubernetes_pod_name]
        target_label: pod

      - source_labels: [__meta_kubernetes_pod_label_sonataflow_org_workflow_app]
        target_label: workflow

      pipeline_stages:
      - json:
          expressions:
            timestamp: timestamp
            level: level
            logger: logger
            message: message
            processInstanceId: mdc.processInstanceId
            traceId: mdc.traceId
            spanId: mdc.spanId

      - labels:
          level:
          logger:
          processInstanceId:
          traceId:

Option B: Scrape JSON Log Files

When using file-based JSON logging, configure Promtail as a sidecar to read from the shared log volume:

apiVersion: v1
kind: ConfigMap
metadata:
  name: promtail-sidecar-config
  namespace: sonataflow-infra
data:
  config.yml: |
    server:
      http_listen_port: 3101

    clients:
      - url: http://loki.sonataflow-observability.svc.cluster.local:3100/loki/api/v1/push

    positions:
      filename: /var/log/positions.yaml

    scrape_configs:
    - job_name: sonataflow-json-files
      static_configs:
      - targets:
          - localhost
        labels:
          job: sonataflow-workflows
          __path__: /var/log/sonataflow/*.log

      pipeline_stages:
      - json:
          expressions:
            timestamp: timestamp
            level: level
            logger: loggerName
            message: message
            processInstanceId: mdc.processInstanceId
            traceId: mdc.traceId
            spanId: mdc.spanId

      - labels:
          level:
          logger:
          processInstanceId:
          traceId:

      - timestamp:
          source: timestamp
          format: RFC3339Nano

Promtail Sidecar Deployment:

Add Promtail as a sidecar container in your SonataFlow CR:

apiVersion: sonataflow.org/v1alpha08
kind: SonataFlow
metadata:
  name: my-workflow
  namespace: sonataflow-infra
spec:
  podTemplate:
    container:
      volumeMounts:
      - name: shared-logs
        mountPath: /var/log/sonataflow
    containers:
    - name: promtail-sidecar
      image: grafana/promtail:2.9.0
      args:
        - -config.file=/etc/promtail/config.yml
      volumeMounts:
      - name: shared-logs
        mountPath: /var/log/sonataflow
        readOnly: true
      - name: promtail-config
        mountPath: /etc/promtail
      - name: positions
        mountPath: /var/log
      resources:
        requests:
          cpu: 50m
          memory: 64Mi
        limits:
          cpu: 100m
          memory: 128Mi
    volumes:
    - name: shared-logs
      emptyDir:
        sizeLimit: 500Mi
    - name: promtail-config
      configMap:
        name: promtail-sidecar-config
    - name: positions
      emptyDir: {}

Query Examples

Filter logs by process instance:

{job="sonataflow-workflows"} | json | processInstanceId="abc-123-def-456"

Find workflow errors:

{job="sonataflow-workflows", workflow="onboarding"} | json | level="ERROR"

Trace correlation:

{job="sonataflow-workflows"} | json | traceId="4bf92f3577b34da6a3ce929d0e0e4736"

Process instance timeline:

{job="sonataflow-workflows"} | json | processInstanceId="abc-123-def-456" | line_format "{{.timestamp}} [{{.level}}] {{.message}}"

🔍 50+ additional query examples: See LogQL Query Reference for comprehensive examples including SLA monitoring, capacity planning, and audit trails.

Alternative Stack: Fluent Bit + OpenSearch

For organizations preferring Elasticsearch-compatible solutions:

Deployment

# Deploy OpenSearch cluster
helm repo add opensearch https://opensearch-project.github.io/helm-charts/
helm install opensearch opensearch/opensearch \
  --namespace logging \
  --set clusterName=sonataflow-logs \
  --set nodeGroup=master \
  --set masterService=opensearch \
  --set replicas=3

# Deploy Fluent Bit
helm install fluent-bit fluent/fluent-bit \
  --namespace logging \
  --set outputs.es.host=opensearch \
  --set outputs.es.index=sonataflow-logs

Fluent Bit Configuration

You can configure Fluent Bit to collect logs from container stdout or from custom JSON log files.

Option A: Scrape Container Logs (Default)

This configuration collects logs from the standard Kubernetes container log location:

apiVersion: v1
kind: ConfigMap
metadata:
  name: fluent-bit-config
data:
  custom_parsers.conf: |
    [PARSER]
        Name        sonataflow_json
        Format      json
        Time_Key    timestamp
        Time_Format %Y-%m-%dT%H:%M:%S.%L%z    

  fluent-bit.conf: |
    [INPUT]
        Name              tail
        Path              /var/log/containers/*sonataflow*.log
        Parser            sonataflow_json
        Tag               kube.sonataflow.*
        Refresh_Interval  5
        Mem_Buf_Limit     50MB

    [FILTER]
        Name                kubernetes
        Match               kube.sonataflow.*
        Kube_URL            https://kubernetes.default.svc:443
        Kube_CA_File        /var/run/secrets/kubernetes.io/serviceaccount/ca.crt
        Kube_Token_File     /var/run/secrets/kubernetes.io/serviceaccount/token
        Keep_Log            On
        Merge_Log           On

    [OUTPUT]
        Name            es
        Match           kube.sonataflow.*
        Host            opensearch
        Port            9200
        Index           sonataflow-logs-%Y.%m.%d
        Type            _doc
        Logstash_Format On

Option B: Scrape JSON Log Files (Sidecar)

When using file-based JSON logging, deploy Fluent Bit as a sidecar to read from the shared log volume:

apiVersion: v1
kind: ConfigMap
metadata:
  name: fluent-bit-sidecar-config
  namespace: sonataflow-infra
data:
  custom_parsers.conf: |
    [PARSER]
        Name        sonataflow_json
        Format      json
        Time_Key    timestamp
        Time_Format %Y-%m-%dT%H:%M:%S.%L%z    

  fluent-bit.conf: |
    [SERVICE]
        Flush         1
        Log_Level     info
        Parsers_File  /fluent-bit/etc/custom_parsers.conf

    [INPUT]
        Name              tail
        Path              /var/log/sonataflow/*.log
        Parser            sonataflow_json
        Tag               sonataflow.*
        Refresh_Interval  5
        Mem_Buf_Limit     50MB
        Read_from_Head    True
        DB                /var/log/flb_sonataflow.db

    [FILTER]
        Name          modify
        Match         sonataflow.*
        Add           kubernetes.namespace_name ${NAMESPACE}
        Add           kubernetes.pod_name ${POD_NAME}

    [OUTPUT]
        Name            es
        Match           sonataflow.*
        Host            opensearch.logging.svc.cluster.local
        Port            9200
        Index           sonataflow-logs-%Y.%m.%d
        Type            _doc
        Logstash_Format On
        Retry_Limit     5

Fluent Bit Sidecar Deployment:

Add Fluent Bit as a sidecar container in your SonataFlow CR:

apiVersion: sonataflow.org/v1alpha08
kind: SonataFlow
metadata:
  name: my-workflow
  namespace: sonataflow-infra
spec:
  podTemplate:
    container:
      volumeMounts:
      - name: shared-logs
        mountPath: /var/log/sonataflow
    containers:
    - name: fluent-bit-sidecar
      image: fluent/fluent-bit:2.2
      env:
      - name: NAMESPACE
        valueFrom:
          fieldRef:
            fieldPath: metadata.namespace
      - name: POD_NAME
        valueFrom:
          fieldRef:
            fieldPath: metadata.name
      volumeMounts:
      - name: shared-logs
        mountPath: /var/log/sonataflow
        readOnly: true
      - name: fluent-bit-config
        mountPath: /fluent-bit/etc
      - name: fluent-bit-db
        mountPath: /var/log
      resources:
        requests:
          cpu: 50m
          memory: 64Mi
        limits:
          cpu: 100m
          memory: 128Mi
    volumes:
    - name: shared-logs
      emptyDir:
        sizeLimit: 500Mi
    - name: fluent-bit-config
      configMap:
        name: fluent-bit-sidecar-config
    - name: fluent-bit-db
      emptyDir: {}

OpenSearch Query Examples

Process instance logs:

{
  "query": {
    "term": {
      "MDC.processInstanceId": "abc-123-def-456"
    }
  }
}

Error aggregation:

{
  "query": {
    "bool": {
      "must": [
        {"term": {"level": "ERROR"}},
        {"range": {"@timestamp": {"gte": "now-1h"}}}
      ]
    }
  },
  "aggs": {
    "by_workflow": {
      "terms": {"field": "kubernetes.labels.sonataflow_org/workflow-app"}
    }
  }
}

What’s More: Visualization and Alerting

Grafana Dashboards

Create comprehensive monitoring dashboards with:

• Workflow Overview: Active process instances, completion rates, error percentages • Process Timeline: Step-by-step execution visualization per process instance • Performance Metrics: Average duration, throughput, resource consumption • Error Analysis: Error distribution, stack traces, failure patterns

📊 Ready-to-use Dashboard: Get a complete Grafana dashboard with 10 pre-built panels from Grafana Dashboard Configuration.

Sample Dashboard Panels

Active Process Instances:

count(count by (processInstanceId) ({job="sonataflow-workflows"} | json | processInstanceId != ""))

Error Rate by Workflow:

rate({job="sonataflow-workflows", level="ERROR"} | json[5m])

Process Duration Distribution:

histogram_quantile(0.95,
  rate({job="sonataflow-workflows"} | json | message="Workflow completed" | duration > 0[5m])
)

Alerting Rules

Configure alerts for critical workflow conditions:

Workflow Failures

groups:
- name: sonataflow.rules
  rules:
  - alert: WorkflowHighErrorRate
    expr: rate({job="sonataflow-workflows", level="ERROR"}[5m]) > 0.1
    for: 2m
    labels:
      severity: warning
    annotations:
      summary: "High error rate in SonataFlow workflows"
      description: "Error rate is {{ $value }} errors per second"

  - alert: WorkflowInstanceStuck
    expr: |
      time() - max by (process_instance_id) (
        {job="sonataflow-workflows"} | json | unwrap timestamp[1h]
      ) > 3600      
    labels:
      severity: critical
    annotations:
      summary: "Workflow instance {{ $labels.process_instance_id }} appears stuck"

Long-Running Processes

  - alert: LongRunningWorkflow
    expr: |
      time() - min by (process_instance_id) (
        {job="sonataflow-workflows"} | json | message="Workflow started" | unwrap timestamp[24h]
      ) > 7200      
    labels:
      severity: warning
    annotations:
      summary: "Workflow {{ $labels.process_instance_id }} running longer than 2 hours"

Integration with External Systems

OpenTelemetry Tracing

Correlate logs with distributed traces:

# Enable OpenTelemetry integration
quarkus.otel.exporter.otlp.traces.endpoint=http://jaeger-collector:14268/api/traces
quarkus.otel.service.name=${workflow.name}
quarkus.otel.resource.attributes=service.namespace=sonataflow-infra

Notification Systems

Integrate with Slack, PagerDuty, or email:

route:
  group_by: ['alertname', 'workflow']
  group_wait: 10s
  group_interval: 10s
  repeat_interval: 1h
  receiver: 'web.hook'

receivers:
- name: 'web.hook'
  slack_configs:
  - api_url: 'YOUR_SLACK_WEBHOOK_URL'
    channel: '#workflow-alerts'
    title: 'SonataFlow Alert'
    text: '{{ range .Alerts }}{{ .Annotations.summary }}{{ end }}'

Troubleshooting

Common Issues

Logs Still in Plain Text Format

Problem: Logs appear as traditional text instead of JSON after configuration.

Solutions:

Verify quarkus-logging-json extension is in your workflow’s pom.xml
Rebuild and redeploy the workflow container image
Check that quarkus.log.console.json=true is correctly set in ConfigMap
Restart the workflow pod after ConfigMap changes

No Process Instance Context in Logs

Problem: JSON logs work but processInstanceId is always empty or missing.

Solutions:

Verify workflow instances are actually running (check workflow status)
Ensure quarkus.log.console.json.print-details=true is set
Check if your SonataFlow version automatically populates MDC (may require custom implementation)

Promtail Not Collecting Logs

Problem: Loki shows no data from workflow pods.

Solutions:

Verify pod label selector in Promtail configuration matches your workflow pods
Check Promtail logs: oc logs -l app=promtail
Ensure Promtail has proper RBAC permissions to read pod logs
Verify namespace configuration in scrape_configs

High Resource Usage

Problem: JSON logging causes performance issues.

Solutions:

Adjust log levels to reduce volume: quarkus.log.category."org.kie.kogito".level=WARN
Configure log rotation and retention policies
Use asynchronous logging: quarkus.log.console.async=true
Monitor storage and network bandwidth usage

Notes and Considerations

• Resource Planning: JSON logging increases log volume by 20-30% compared to plain text • Retention Policies: Configure appropriate log retention (typically 30-90 days for workflow logs) • Index Optimization: Use time-based indices for better performance with large log volumes • Security: Ensure log aggregation respects namespace isolation and RBAC policies • Backup Strategy: Include log indices in disaster recovery planning • Cost Management: Monitor storage costs, especially with high-throughput workflows

Additional Resources

• Complete PLG Observability Guide: Comprehensive documentation with production-ready configurations • Quick Start Guide: 15-minute deployment walkthrough • YAML Manifests: All Kubernetes resources for manual deployment • Advanced Promtail Configuration: Detailed log parsing and collection setup

For detailed setup instructions and advanced configurations, refer to the comprehensive observability documentation and OpenShift logging best practices.

3 - Make workflow able to use authentication request on run

Starting RHDH 1.7.3, the orchestrator plugin let you specify in the worfklow’s data input schema file the required login for the workflow to work against external services.

Prerequisites

Keycloak or another OIDC provider that supports OAuth 2.0 Token Exchange
A workflow calling an OpenAPI client generated from an OpenAPI specification file using an oauth2 security scheme

Configure data input schemas property

To enable RHDH Orchestrator plugin to dynamically prompt for authentication you need to set the authSetup property and add each authentication provider needed under authTokenDescriptors:

"authSetup": {
    "type": "string",
    "ui:widget": "AuthRequester",
    "ui:props": {
        "authTokenDescriptors": [
            {
            "provider": "oidc",
            "customProviderApiId": "internal.auth.oidc",
            "tokenType": "oauth"
            }
        ]
    }
}

In the above example, we are making sure that upon execution the RHDH will request the user to login in the OIDC provider configured in RHDH.

In the workflow, you will then be able to use the token by using the header X-Authorization-Oidc. See Token Propagation and Token Exchange pages for example in using such header.

You can specify other providers such as github or gitlab, the header’s name is formatted as follow: X-Authorization-<provider>.

4 - Configure workflow for token exchange

Token Exchange lets a workflow swap the incoming end‑user token for a new access token tailored to a downstream OpenAPI‑secured service. Use it when you must not forward the original token or when workflows run long enough that the original token may expire.

See the upstream reference for full details: Token Exchange for OpenAPI services in SonataFlow (https://sonataflow.org/serverlessworkflow/main/security/token-exchange-for-openapi-services.html).

Prerequisites

Keycloak or another OIDC provider that supports OAuth 2.0 Token Exchange
A workflow calling an OpenAPI client generated from an OpenAPI specification file using an oauth2 security scheme

Build

When building the workflow image, ensure the following extensions are present (e.g., via QUARKUS_EXTENSION for the internal builder):

io.quarkus:quarkus-oidc-client-filter
org.kie:kie-addons-quarkus-token-exchange

Optional for persistence of exchanged tokens:

org.kie:kogito-quarkus-serverless-workflow-jdbc-token-persistence

See https://github.com/rhdhorchestrator/orchestrator-demo/blob/main/scripts/build.sh#L180 to see how we do it.

Configuration

1) Define an OAuth2 security scheme in your OpenAPI specification file

The OpenAPI operation(s) you call must be secured by an oauth2 scheme. The OIDC client name is derived from this scheme name (sanitized by replacing non‑alphanumerics with _).

Example:

openapi: 3.0.3
paths:
  /secured:
    get:
      operationId: callService
      responses:
        "200": { description: OK }
      security:
        - service-oauth: []
components:
  securitySchemes:
    service-oauth:
      type: oauth2
      flows:
        clientCredentials:
          authorizationUrl: https://<idp>/realms/<realm>/protocol/openid-connect/auth
          tokenUrl: https://<idp>/realms/<realm>/protocol/openid-connect/token
          scopes: {}

2) Configure `application.properties`

Replace placeholders with your values.

# Base URL for the generated client (service id is the sanitized OpenAPI file id)
quarkus.rest-client.<service_id>.url=http://<downstream-service>

# Enable Token Exchange for this auth name (sanitized from OpenAPI scheme name)
sonataflow.security.auth.<auth_name>.token-exchange.enabled=true

# Proactive refresh and monitor (optional, default values are shown below)
sonataflow.security.auth.<auth_name>.token-exchange.proactive-refresh-seconds=300
sonataflow.security.auth.token-exchange.monitor-rate-seconds=60

# Ensure the incoming user token is available to the workflow service
# If the end-user token is sent in a custom header (e.g., from RHDH Orchestrator plugin),
# point Quarkus to that header so the SecurityIdentity is established.
auth-server-url=https://<keycloak>/realms/<yourRealm>
client-id=<client ID>
client-secret=<client secret>

quarkus.oidc.auth-server-url=${auth-server-url}
quarkus.oidc.client-id=${client-id}
quarkus.oidc.credentials.secret=${client-secret}
quarkus.oidc.token.header=X-Authorization-<provider>
quarkus.oidc.token.issuer=any

# OIDC client for Token Exchange corresponding to the auth scheme (name sanitized)
quarkus.oidc-client.<auth_name>.discovery-enabled=false
quarkus.oidc-client.<auth_name>.auth-server-url=${auth-server-url}/protocol/openid-connect/auth
quarkus.oidc-client.<auth_name>.token-path=${auth-server-url}/protocol/openid-connect/token
quarkus.oidc-client.<auth_name>.client-id=${client-id}
quarkus.oidc-client.<auth_name>.grant.type=exchange
quarkus.oidc-client.<auth_name>.credentials.client-secret.method=basic
quarkus.oidc-client.<auth_name>.credentials.client-secret.value=${client-secret}

# Persist inbound headers so tokens survive wait/resume and restarts (recommended)
kogito.persistence.headers.enabled=true

With:

service_id: sanitized OpenAPI file id (e.g., simple-server.yaml -> simple_server_yaml).
auth_name: sanitized OpenAPI oauth2 security scheme (e.g., service-oauth -> service_oauth).
provider: the RHDH provider name; the Orchestrator plugin sends user tokens as X-Authorization-{provider}: {token}.

Configuration reference

SonataFlow Token Exchange guide: Token Exchange for OpenAPI services
SonataFlow configuration properties (headers persistence): Core configuration properties
Quarkus OIDC Client: OpenID Connect (OIDC) client
Quarkus OIDC Client Filter (REST Client): REST Client OIDC client filter
Quarkiverse OpenAPI Generator client configuration: Client configuration
Token propagation with REST Client (contrast with exchange): Token propagation

3) Interplay with token propagation

Do not enable token propagation for the same <auth_name> if you need token exchange. If both are enabled, propagation takes precedence and no exchange is performed.

If you do need propagation in another specification file having a similar scheme name, configure per scheme name on the specification level:

quarkus.openapi-generator.<another_service_id>.auth.<auth_name>.token-propagation=true
quarkus.openapi-generator.<another_service_id>.auth.<auth_name>.header-name=X-Authorization-<provider>

Caching and persistence

When enabled, exchanged tokens are cached per process instance and auth name, with proactive refresh before expiry. By default, an in‑memory cache is used. To persist cache entries, add the JDBC persistence extension in the QUARKUS_EXTENSIONS when building the image: see https://sonataflow.org/serverlessworkflow/latest/cloud/operator/build-and-deploy-workflows.html#passing-build-arguments-to-internal-workflow-builder

For local debug/dev, you can add it in your local pom.xml file:

<dependency>
  <groupId>org.kie</groupId>
  <artifactId>kogito-quarkus-serverless-workflow-jdbc-token-persistence</artifactId>
  <!-- Configure a JDBC DataSource as usual -->
  <!-- The extension provides a CDI TokenCacheRepository implementation -->
</dependency>

Examples

Invoke workflow with an Authorization header

curl -X POST \
  http://localhost:8080/<workflow_id> \
  -H "Authorization: Bearer $USER_ACCESS_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"input":"value"}'

Invoke workflow with RHDH custom header

If your client sends the token via the Orchestrator plugin header, and you set quarkus.oidc.token.header=X-Authorization-<provider>:

curl -X POST \
  http://localhost:8080/<workflow_id> \
  -H "X-Authorization-<provider>: Bearer $USER_ACCESS_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"input":"value"}'

Notes

Security scheme names are global in an OpenAPI specification file; all operations using the same scheme share the same OIDC client and token‑exchange configuration.
Prefer enabling kogito.persistence.headers.enabled=true for long‑running workflows so the incoming token is available after wait/resume or restarts.
For a general comparison and configuration of forwarding the original token, see the token propagation guide in this folder.

Configuring OIDC properties at SonataFlowPlatform level (Cluster‑wide OIDC configuration)

To avoid duplication, follow the platform‑level OIDC setup here: Configuring OIDC properties at SonataFlowPlatform level. This centralizes incoming request authentication and $WORKFLOW.identity. For Token Exchange, you still need to enable it per auth scheme using sonataflow.security.auth.<auth_name>.token-exchange.enabled=true.

5 - Configure workflow for token propagation

By default, the RHDH Orchestrator plugin adds headers for each token in the ‘authTokens’ field of the POST request that is used to trigger a workflow execution. Those headers will be in the following format: X-Authorization-{provider}: {token}. This allows the user identity to be propagated to the third parties and externals services called by the workflow. To do so, a set of properties must be set in the workflow application.properties file.

Prerequisites

Having a Keycloak instance running with a client
Having RHDH with the latest version of the Orchestrator plugins
Having a workflow using openapi spec file to send REST requests to a service. Using custom REST function within the workflow will not propagate the token; it is only possible to propagate tokens when using openapi specification file.

Build

When building the workflow’s image, you will need to make sure the following extensions are present in the QUARKUS_EXTENSION:

io.quarkus:quarkus-oidc-client-filter # needed for propagation
io.quarkus:quarkus-oidc # neded for token validity check thus accessing $WORKFLOW.identity

See https://github.com/rhdhorchestrator/orchestrator-demo/blob/main/scripts/build.sh#L180 to see how we do it.

Configuration

By default, the workflow is not persisting the request headers in the database. Therefore, any token in the header will be lost if the workflow flushes its context (e.g: sleeps, goes idle, is resumed, …) as the headers will not be restored to the context from the database.

By setting the property kogito.persistence.headers.enabled to true in the application.properties file or in the config map representing it on the cluster, the workflow will persist the headers. This will enable the workflow to keep using the token from the headers even after it was interupted and restored.

You can exclude headers from being persisted using kogito.persistence.headers.excluded. See https://sonataflow.org/serverlessworkflow/main/core/configuration-properties.html and/or https://sonataflow.org/serverlessworkflow/main/use-cases/advanced-developer-use-cases/persistence/persistence-with-postgresql.html#ref-postgresql-persistence-configuration for more information.

Oauth2

In the OpenAPI spec file(s) where you want to propagate the incoming token, define the security scheme used by the endpoints you’re interested in. All endpoints may use the same security scheme if configured globally. e.g

components:
  securitySchemes:
    BearerToken:
     type: oauth2
     flows:
       clientCredentials:
         tokenUrl: http://<keycloak>/realms/<yourRealm>/protocol/openid-connect/token
         scopes: {}
     description: Bearer Token authentication

In the application.properties of your workflow, for each security scheme, add the following:

auth-server-url=https://<keycloak>/realms/<yourRealm>
client-id=<client ID>
client-secret=<client secret>

# Properties to check for identity, needed to use $WORKFLOW.identity within the workflow
quarkus.oidc.auth-server-url=${auth-server-url}
quarkus.oidc.client-id=${client-id}
quarkus.oidc.credentials.secret=${client-secret}
quarkus.oidc.token.header=X-Authorization-<provider>
quarkus.oidc.token.issuer=any # needed in case the auth server url is not the same as the one configured; e.g: localhost VS the k8S service

# Properties for propagation
quarkus.oidc-client.BearerToken.auth-server-url=${auth-server-url}
quarkus.oidc-client.BearerToken.token-path=${auth-server-url}/protocol/openid-connect/token
quarkus.oidc-client.BearerToken.discovery-enabled=false
quarkus.oidc-client.BearerToken.client-id=${client-id}
quarkus.oidc-client.BearerToken.grant.type=client
quarkus.oidc-client.BearerToken.credentials.client-secret.method=basic
quarkus.oidc-client.BearerToken.credentials.client-secret.value=${client-secret}

quarkus.openapi-generator.<spec_file_yaml_or_json>.auth.<security_scheme>.token-propagation=true
quarkus.openapi-generator.<spec_file_yaml_or_json>.auth.<security_scheme>.header-name=X-Authorization-<provider>

With:

spec_file_yaml_or_json: the name of the spec file configured with _ as separator. E.g: if the file name is simple-server.yaml the normalized property name will be simple_server_yaml. This should be the same for every security scheme defined in the file.
security_scheme: the name of the security scheme for which propagates the token located in the header defined by the header-name property. In our example it would be BearerToken.
provider: the name of the expected provider from which the token comes from. As explained above, for each provider in RHDH, the Orchestrator plugin is adding a header with the format X-Authorization-{provider}: {token}.
keycloak: the URL of the running Keycloak instance.
yourRealm: the name of the realm to use.
client ID: the ID of the Keycloak client to use to authenticate against the Keycloak instance.

See https://sonataflow.org/serverlessworkflow/latest/security/authention-support-for-openapi-services.html#ref-authorization-token-propagation and https://quarkus.io/guides/security-openid-connect-client-reference#token-propagation-rest for more information about token propagation.

Setting the quarkus.oidc.* properties will enforce the token validity check against the OIDC provider. Once successful, you will be able to use $WORKFLOW.identity in the workflow definition in order to get the identity of the user. See https://quarkus.io/guides/security-oidc-bearer-token-authentication and https://quarkus.io/guides/security-oidc-bearer-token-authentication-tutorial for more information.

Bearer token

In the OpenAPI spec file(s) where you want to propagate the incoming token, define the security scheme used by the endpoints you’re interested in. All endpoints may use the same security scheme if configured globally. e.g

components:
  securitySchemes:
    SimpleBearerToken:
     type: http
     scheme: bearer

In the application.properties of your workflow, for each security scheme, add the following:

auth-server-url=https://<keycloak>/realms/<yourRealm>
client-id=<client ID>
client-secret=<client secret>

# Properties to check for identity, needed to use $WORKFLOW.identity within the workflow
quarkus.oidc.auth-server-url=${auth-server-url}
quarkus.oidc.client-id=${client-id}
quarkus.oidc.credentials.secret=${client-secret}
quarkus.oidc.token.header=X-Authorization-<provider>
quarkus.oidc.token.issuer=any # needed in case the auth server url is not the same as the one configured; e.g: localhost VS the k8S service

quarkus.openapi-generator.<spec_file_yaml_or_json>.auth.<security_scheme>.token-propagation=true
quarkus.openapi-generator.<spec_file_yaml_or_json>.auth.<security_scheme>.header-name=X-Authorization-<provider>

With:

spec_file_yaml_or_json: the name of the spec file configured with _ as separator. E.g: if the file name is simple-server.yaml the normalized property name will be simple_server_yaml. This should be the same for every security scheme defined in the file.
security_scheme: the name of the security scheme for which propagates the token located in the header defined by the header-name property. In our example it would be SimpleBearerToken.
provider: the name of the expected provider from which the token comes from. As explained above, for each provider in RHDH, the Orchestrator plugin is adding a header with the format X-Authorization-{provider}: {token}.

Basic auth

Basic auth token propagation is not currently supported. A pull request has been opened to add support for it: https://github.com/quarkiverse/quarkus-openapi-generator/pull/1078

With Basic auth, the $WORKFLOW.identity is not available.

Instead you could access the header directly: $WORKFLOW.headers.X-Authorization-{provider} and decode it:

functions:
- name: getIdentity
  type: expression
  operation: '.identity=($WORKFLOW.headers["x-authorization-basic"] | @base64d | split(":")[0])' # mind the lower case!!

You can see a full example here: https://github.com/rhdhorchestrator/workflow-token-propagation-example.

Configuring OIDC properties at SonataFlowPlatform level (Cluster-wide OIDC configuration)

This short guide shows how to inject the Quarkus OIDC settings once at platform‑scope so that all present and future workflows automatically authenticate incoming requests and expose $WORKFLOW.identity.

Prerequisites

Namespace where the workflows run
Keycloak Realm URL
Client‑ID
Client‑secret

There is an assumption that the workflows and the platform are installed in the sonataflow-infra here.

export TARGET_NS=‘sonataflow-infra’ # target namespace of workflows and sonataflowplatform CR

Keep the client secret in a Secrets vault; don’t embed it as clear‑text in the CR.

Create the supporting objects

Secret: holds the confidential client secret

e.g

oc create secret generic oidc-client-secret \
  -n $TARGET_NS \
  --from-literal=cred=swf-client-secret  # This is a sample value. You need to replace it with actual value.

Patch the SonataFlowPlatform CR

Create patch.yaml (or paste inline):

e.g

#### All the values below need to be replaced by actual values.
spec:
  properties:
    flow:
    - name: quarkus.oidc.auth-server-url
      value: https://keycloak-host/realms/dev
    - name: quarkus.oidc.client-id
      value: swf-client
    - name: quarkus.oidc.token.header
      value: X-Authorization
    - name: quarkus.oidc.token.issuer
      value: any
    - name: quarkus.oidc.credentials.secret
      valueFrom:
        secretKeyRef:
          key: cred
          name: oidc-client-secret

Apply the patch:

e.g

oc patch sonataflowplatform <Platform CR name> \
  -n $TARGET_NS \
  --type merge \
  -p "$(cat patch.yaml)"

Wait a few seconds for the operator reconcile loop.

Verify the managed properties

e.g

oc get sonataflowplatform <Platform CR name> -n $TARGET_NS -o yaml

You should see all five keys.

Restart running workflow deployments once so Quarkus reloads the file:

e.g

oc rollout restart deployment -l sonataflow.org/workflow -n $TARGET_NS