If you've been in software engineering for more than a minute, you know the drill. A product manager walks up and says, "We need to sync our database with Salesforce, trigger a Slack alert, update Zendesk, and pipe the metrics into Datadog." On paper, it sounds like a straightforward afternoon of work. In reality, you are about to enter Integration Hell.
You’ll spend the next three days reading mismatched OpenAPI specs, handling quirky OAuth2 flows, writing boilerplate retry logic for rate limits, and wrapping everything in defensive try-catch blocks. And God forbid one of those third-party APIs changes its payload schema next month without warning.
This week, a project caught my eye on Hacker News that promises to kill this pain point once and for all: Ballet. It’s an open-source workflow automation engine designed specifically for developers. Instead of dragging and dropping boxes in a heavy low-code UI (looking at you, Zapier and Make), or writing thousands of lines of brittle SDK glue code, Ballet lets you write clean, code-first workflows while it handles the dirty work of dynamically generating integration code against any API on the fly.
Let’s dive deep into how Ballet works, look at its architectural philosophy, and build a real-world workflow to see if it lives up to the hype.
The Core Problem: Why Integration SDKs Suck
Traditionally, developers have had two choices when connecting systems:
- The SDK Route: Pull in official npm packages or Python libraries for every service. The catch: These libraries are often bloated, outdated, inconsistent in their error-handling patterns, and increase your application's dependency attack surface.
- The HTTP/cURL Route: Write raw
fetchoraxioscalls. The catch: You have to manually write authentication handshakes, rate-limiting backoffs, and type-safe payload definitions.
Ballet introduces a third way. It acts as a lightweight runtime and compiler. It reads API definitions (like OpenAPI/Swagger specs) or learns from raw HTTP curl examples, and exposes them inside a unified, developer-friendly sandbox. You write standard TypeScript or Python to orchestrate the logic, and Ballet handles the underlying protocol execution, state persistence, and error recovery.
Under the Hood: How Ballet Orchestrates Integrations
To understand why Ballet is different from traditional workflow engines like Temporal or Airflow, we need to look at its architecture. Ballet is designed around three main concepts: Connectors, Flows, and the State Engine.
+-------------------------------------------------------------+
| Ballet Engine |
| |
| +--------------------+ +--------------------+ |
| | TypeScript/Py | Executes | State Engine | |
| | Workflow Code | ----------> | (ACID Steps, State)| |
| +--------------------+ +--------------------+ |
| | | |
| | Resolves | Persists |
| v v |
| +-------------------------------------------------------+ |
| | Dynamic Connector Layer | |
| | (Translates JS/Python calls into HTTP/gRPC) | |
| +-------------------------------------------------------+ |
+------------------------------|------------------------------+
|
+-----------+-----------+
| |
v v
[ Stripe API ] [ Internal DB ]
1. Dynamic Connectors
Instead of relying on hardcoded, maintenance-heavy integration modules, Ballet uses a dynamic translation layer. You feed it an OpenAPI schema, a GraphQL endpoint, or even just a set of curl commands. Ballet parses these inputs and dynamically generates a typed, autocompleted client at runtime. If the target API updates, you simply refresh the schema definition, and your workflow code updates instantly without requiring a rewrite of your core logic.
2. Durable Execution (Without the Overhead)
If your workflow crashes halfway through a multi-step billing sync, you don't want to double-charge your customer. Ballet implements a lightweight version of durable execution. It checkpoints the state of your workflow after every external API call. If a network blip occurs or the container restarts, Ballet resumes the execution precisely where it left off, ensuring strict once-only execution patterns.
Hands-On: Building an Automated User Provisioning Flow
Let's get our hands dirty. Imagine a common scenario: when a new developer joins our team, we need to:
- Create their account in GitHub and invite them to our organization.
- Add their user profile to our internal PostgreSQL database.
- Send a welcome message with setup instructions to their Slack.
Here is how we can implement this cleanly in Ballet using its TypeScript SDK.
Step 1: Define the Environment and Connectors
First, we configure our connectors. In Ballet, this can be done via a simple YAML manifest file or directly in code. Here, we'll define our integrations:
// ballet.config.yaml
version: "1"
connectors:
github:
type: openapi
spec: https://api.github.com/specs/v3.json
auth:
type: bearer
token: ${GITHUB_PAT}
slack:
type: http
baseUrl: https://slack.com/api
auth:
type: oauth2
token: ${SLACK_BOT_TOKEN}
database:
type: postgres
connectionString: ${DATABASE_URL}
Step 2: Writing the Workflow
With our connectors declared, Ballet automatically makes them available in our execution environment with full type safety. Here is our workflow script:
import { workflow, connectors } from '@ballet/sdk';
interface NewHireInput {
email: string;
githubUsername: string;
fullName: string;
}
export default workflow('OnboardNewDeveloper', async (input: NewHireInput) => {
const { email, githubUsername, fullName } = input;
// Step 1: Add user to database and retrieve our internal User ID
const dbResult = await connectors.database.query(
'INSERT INTO employees (email, name, role, status) VALUES ($1, $2, $3, $4) RETURNING id',
[email, fullName, 'Software Engineer', 'ONBOARDING']
);
const userId = dbResult.rows[0].id;
// Step 2: Invite to GitHub Org (Dynamic connector reads the OpenAPI spec)
try {
await connectors.github.orgs.createMembershipForUser({
org: 'sysseder-dev',
username: githubUsername,
role: 'member'
});
} catch (error) {
// If the GitHub invite fails, we update the DB state and alert the team
await connectors.database.query(
'UPDATE employees SET status = $1 WHERE id = $2',
['GITHUB_FAILED', userId]
);
throw new Error(`Failed to invite ${githubUsername} to GitHub: ${error.message}`);
}
// Step 3: Look up Slack ID by email
const slackUserLookup = await connectors.slack.get('/users.lookupByEmail', {
params: { email }
});
if (slackUserLookup.ok) {
const slackUserId = slackUserLookup.user.id;
// Step 4: Send the welcome DM
await connectors.slack.post('/chat.postMessage', {
json: {
channel: slackUserId,
text: `Welcome to the team, ${fullName}! 🚀 Head over to https://wiki.sysseder.internal/setup to get started.`
}
});
}
// Step 5: Mark onboarding complete in DB
await connectors.database.query(
'UPDATE employees SET status = $1 WHERE id = $2',
['ACTIVE', userId]
);
return { success: true, userId };
});
Why This Developer Experience Wins
Look closely at the code above. There is no custom HTTP client configuration. There are no manual retry loops written around the GitHub or Slack calls; Ballet's runtime handles exponential backoff automatically based on the HTTP status codes returned (like 429 Too Many Requests).
More importantly, the code is highly readable. Any developer on your team can look at this script and immediately understand the business logic without getting bogged down in the infrastructural plumbing.
Advanced Power: Autogenerating Connectors with "Ballet AI"
One of the most impressive aspects of the Ballet project is its ability to handle APIs that lack any formal OpenAPI spec. We've all had to integrate with legacy enterprise systems where the only "documentation" is a poorly formatted PDF or a raw curl example.
Ballet solves this by letting you pass raw HTTP examples directly to its CLI. It parses the request and response payloads, infers the schema, and generates a functional mock connector for you:
$ ballet connector generate \
--name legacyBilling \
--example-curl "curl -X POST https://legacy.firm/api/v1/charge -d '{\"amt\": 100, \"uid\": \"usr_12\"}'"This command outputs a local connector manifest containing a fully typed schema matching the inferred parameters. You can immediately import it and start using typed autocomplete features in your IDE of choice. This bridges the gap between old-school SOAP/REST legacy systems and modern, clean DX.
The Verdict: Is Ballet Ready for Production?
Ballet is a refreshing take on developer-first automation. It lands in the sweet spot between heavy, enterprise workflow frameworks like Temporal (which require significant infrastructure overhead to run) and low-code tools like Zapier (which quickly become an unmaintainable mess of hidden logic and version control nightmares).
Pros:
- Code-First: Everything lives in git. You can write unit tests, review code via PRs, and run local integration tests easily.
- Type-Safe: Excellent IDE autocomplete experience generated straight from API specs.
- Resilient: Out-of-the-box state management, persistence, and automated retries.
Cons:
- As a relatively new open-source tool, its ecosystem of pre-built community connectors is still growing.
- For highly complex distributed state machines requiring Saga patterns, you might still need the heavy-duty features of Temporal or AWS Step Functions.
Wrapping Up
If you're tired of writing boilerplate integration glue code and want a system that gives you the flexibility of raw code with the infrastructure benefits of an enterprise workflow engine, you should absolutely give Ballet a look.
You can check out their open-source repository, self-host it via Docker, or run it locally to start prototyping your workflows.
Over to you: How are you managing integration workflows in your current stack? Are you writing custom cron jobs, using an enterprise state machine, or relying on third-party SaaS tools? Let me know in the comments below, or drop your thoughts in the sysseder community forums!