Skip to content

Compared to

Tool What it does Difference
Metadata-driven copy (Azure Data Factory, Fabric) Control tables drive copy activities. Copy-centric, it stops at ingestion. pipetree’s YAML covers ingestion through modeling.
dbt Config-driven SQL transformations. Transformation only, it does not extract. pipetree has source readers and SCD merge strategies.
AUTO CDC KEYS, SEQUENCE BY, STORED AS SCD TYPE in SQL. Runs inside Lakeflow pipelines only. pipetree runs on any Spark with Delta and can translate to AUTO CDC.

The design is described in the Pipeline Engineering blog series.