File
Apache Avro
Apache Hudi
Apache Iceberg
Apache ORC
Apache Parquet
CSV
Delta Lake
| Attribute | Apache Avro | Apache Hudi | Apache Iceberg |
|---|---|---|---|
| Name | Apache Avro | Apache Hudi | Apache Iceberg |
| Description | Apache Avro is the leading serialization format for record data, and first choice for streaming data pipelines. | Apache Hudi is a transactional data lake platform that brings database and data warehouse capabilities to the data lake. Utilises data stored in either parquet or orc. | Iceberg is a high-performance format for huge analytic tables. Utilises data stored in either parquet, avro, or orc. |
| License | Apache license 2.0 | Apache license 2.0 | Apache license 2.0 |
| Source code | https://github.com/apache/avro | https://github.com/apache/hudi | https://github.com/apache/iceberg |
| Website | https://avro.apache.org/ | https://hudi.apache.org/ | https://iceberg.apache.org/ |
| Year created | 2009 | 2016 | 2017 |
| Company | Apache | Uber | Netflix |
| Language support | java, c++, c#, c, python, javascript, perl, ruby, php, rust | ||
| Use cases | Stream processing, Analytics, Efficient data exchange | Incremental data processing, Data upserts, Change Data Capture (CDC), ACID transactions | Write once read many, Analytics, Efficient storage, ACID transactions |
| Is human readable |
no
|
no
|
no
|
| Orientation | row | column or row | column or row |
| Has type system |
yes
|
yes
|
yes
|
| Has nested structure support |
yes
|
yes
|
yes
|
| Has native compression |
yes
|
yes
|
yes
|
| Has encoding support |
yes
|
yes
|
yes
|
| Has constraint support |
no
|
yes
|
no
|
| Has acid support |
no
|
yes
|
yes
|
| Has metadata |
yes
|
yes
|
yes
|
| Has encryption support |
no
|
maybe
|
maybe
|
| Data processing framework support | Apache Flink, Apache Gobblin, Apache NiFi, Apache Pig, Apache Spark, | Apache Spark, Apache Flink, | Apache Drill, Apache Flink, Apache Gobblin, Apache Pig, Apache Spark, |
| Analytics query support | Apache Impala, Apache Druid, Apache Hive, Apache Pinot, AWS Athena, BigQuery, Clickhouse, Firebolt, | Apache Hive, Apache Impala, AWS Athena, BigQuery, Clickhouse, Presto, Trino, | Apache Impala, Apache Druid, Apache Hive, AWS Athena, BigQuery, Clickhouse, Dremio, DuckDB, Presto, Trino, |
Showing 1 to 21 of 21 entries