Continue to use the tools you already love, but together. Mix and match dbt with Python in containers, ingestion with transformation, ML with analytics.
If you know SQL or Python, you can use Twirl. No need to scatter your code with imports, dsl or decorators.
Twirl’s development mode reads input data from production but write outputs to a safe development sandbox, so you can iterate fast.
If you define schemas, Twirl will check them, and prevent users from accidentally breaking each others' code.
Twirl has first class support for unit testing, making it easy to test edge cases and be confident your code works.
Deploying a job in Twirl is as simple as merging a pull request. Twirl will run a Github Action pushing your new code to the orchestrator.
All data is stored in your cloud account, and all jobs execute there, so no sensitive data ever leaves your account.
Twirl gives you full control over node scheduling, which means different jobs in the same pipeline can run at different frequencies.
Twirl automatically packages your code and dependencies, simplifying deployment and ensuring consistency between development and prod environments.
Twirl’s web app lets you easily check how all your data jobs are doing in one view.
Use tags or Twirl's selection syntax to easily zoom in on relevant jobs
Twirl’s detail view shows you which upstream jobs failed, making it easy to debug stale tables.
Twirl displays parsed log output so you can always see the right level of detail for your jobs.
Keep the tools you already love and use, and experience a seamless way to develop data pipelines across them all.
“Twirl is like dbt but with the added power of a full blown orchestrator... 10/10 would choose again.”
Victor Quinn
CTO and Co-founder at Texture Energy
“Twirl allowed us to move from three tools (dbt, airflow, and prefect) to just one and in doing so we reduced our code base by 30%.”
Tobias Morville
Head of Data at Monta