Polyglot Orchestration

Real-World Use Case: Multi-Environment Deployment Pipeline

A financial services company needs to deploy applications across dev, staging, and production environments with language-agnostic workflows.

1. Hybrid Shell & Python Workflows

orchestration.json

{
  "name": "deploy-financial-app",
  "version": 1,
  "steps": [
    {
      "name": "validate-deployment",
      "action_type": "shell",
      "code": "docker pull myapp:latest && \
                   docker inspect myapp:latest | jq '.RepoTags'",
      "target": "prod-servers"
    },
    {
      "name": "check-compliance",
      "action_type": "python",
      "code": "import hashlib\nwith open('/app/binary', 'rb') as f:\n  sha256 = hashlib.sha256(f.read()).hexdigest()\n  assert sha256 == '{{vault.expected_hash}}', 'Binary mismatch!'",
      "timeout": 30
    },
    {
      "name": "notify-audit",
      "action_type": "http",
      "method": "POST",
      "url": "https://audit.internal/api/v1/deployments",
      "headers": {
        "Authorization": "Bearer {{vault.audit_token}}"
      },
      "body": {
        "environment": "production",
        "version": "{{vault.app_version}}",
        "timestamp": "2025-02-20T06:00:00Z"
      }
    }
  ]
}

Why This Works

Step 1 - Native shell for Docker validation with real-time image inspection
Step 2 - Python for cryptographic compliance checking with vault variable injection
Step 3 - HTTP for external audit logging with Bearer token from vault

No custom DSL. No limitations. Use the right tool for each job.

2. Nested Orchestrations for Reusable Workflows

Building Blocks Approach

Define base orchestrations, then compose them into complex workflows without duplication:

health-check.json - Generic health check across any service
deploy-service.json - Deploy + health check + rollback
full-pipeline.json - Multi-service orchestration calling the above

Orchestra 10 services with 3 nested calls instead of 100 steps.

Nested Composition

// health-check.json (reusable)
{
  "name": "health-check",
  "steps": [
    {
      "code": "curl -sf {{service_url}}/health",
      "action_type": "shell"
    }
  ]
}

// full-pipeline.json (composes 3 services)
{
  "name": "deploy-all",
  "steps": [
    {
      "name": "deploy-web",
      "action_type": "orchestration",
      "orchestration_id": "deploy-service",
      "params": {
        "service": "web-api"
      }
    },
    {
      "name": "deploy-worker",
      "action_type": "orchestration",
      "orchestration_id": "deploy-service",
      "params": {
        "service": "worker"
      }
    }
  ]
}

3. Data Processing with Multiple Languages

ETL Pipeline

// Orchestration combining all languages
{
  "name": "etl-pipeline",
  "steps": [
    {
      "name": "extract-data",
      "action_type": "shell",
      "code": "aws s3 sync s3://data-lake /tmp/data --profile prod"
    },
    {
      "name": "transform",
      "action_type": "python",
      "code": "import pandas as pd\ndf = pd.read_csv('/tmp/data/raw.csv')\ndf['timestamp'] = pd.to_datetime(df['timestamp'])\ndf.to_parquet('/tmp/data/transformed.parquet')"
    },
    {
      "name": "load",
      "action_type": "http",
      "method": "POST",
      "url": "https://dw.internal/api/ingest",
      "files": {
        "data": "/tmp/data/transformed.parquet"
      }
    }
  ]
}

ETL at Scale

Extract - Bash for S3 operations and cloud CLI tools
Transform - Python for data manipulation with libraries like Pandas, NumPy, Polars
Load - HTTP to push data to data warehouse APIs

No duct-taping languages together. Everything runs in the orchestration engine with shared vault variables.

Additional Capabilities

🔗 Cross-Language State Sharing

Orchestrations maintain shared state across language boundaries. A Shell step stores data, Python reads it, HTTP submits it—all in one workflow.

State Flow

"steps": [
  { "action_type": "shell", "code": "echo 'processed_count=42' > /tmp/state.txt" },
  { "action_type": "python", "code": "state = open('/tmp/state.txt').read(); print(f'Processed: {state}')" },
  { "action_type": "http", "body": { "metric": "{{vault.state}}" } }
]

⚡ Parallel Language Execution

Run Shell and Python steps in parallel on different nodes while maintaining consistency through quorum-based locking.

Parallel Steps

"steps": [
  { "parallel": "true", "steps": [
    { "action_type": "shell", "target": "nodes-1-5", "code": "backup-database.sh" },
    { "action_type": "python", "target": "nodes-6-10", "code": "validate-integrity.py" }
  ]}
]

🎪 Conditional Language Switching

Branch orchestrations based on conditions evaluated in Python, then execute appropriate Shell or HTTP actions.

Conditional Flow

"steps": [
  { "action_type": "python", "name": "check-quota",
    "code": "usage = check_metrics(); \nif usage > 80: return {'proceed': True}" },
  { "action_type": "shell", "if": "check-quota.proceed",
    "code": "scale-up-cluster.sh" }
]

🌍 Multi-Cloud Language Distribution

Route Shell scripts to Linux nodes, Python to Kubernetes pods, HTTP to cloud APIs—all from one orchestration.

Multi-Cloud

"steps": [
  { "action_type": "shell", "target": "aws-ec2",
    "code": "aws elbv2 describe-load-balancers" },
  { "action_type": "python", "target": "gcp-k8s",
    "code": "kubectl get nodes -o json" },
  { "action_type": "http",
    "url": "https://api.azure.com/subscriptions/..." }
]

🎯

Use the Right Tool

Shell for sysadmin, Python for data, HTTP for APIs. No compromises.

🔄

Composable

Build libraries of reusable orchestrations. Compose them at scale.

🚀

No Learning Curve

Teams use languages they already know. No DSL to master.

🔒

Vault Integration

All languages get secret injection and variable substitution automatically.

Try Polyglot Orchestration Back to Features

Execute Automation in Any Language