icelake
Pure Rust Iceberg Implementation
File Explorer
Download Latest Version (.zip)- ci.yml
- build.sh
- Dockerfile
- pom.xml
- settings.xml
- README.md
- read_iceberg_table.rs
- io.rs
- layer.rs
- mod.rs
- prometheus.rs
- rest.rs
- storage.rs
- mod.rs
- prometheus.rs
- rolling_writer.rs
- data_file_writer.rs
- equality_delete_writer.rs
- mod.rs
- position_delete_writer.rs
- append_only_writer.rs
- equality_delta_writer.rs
- mod.rs
- upsert_writer.rs
- mod.rs
- track_writer.rs
- write.rs
- location_generator.rs
- mod.rs
- scan.rs
- writer_builder.rs
- data_file_writer.rs
- equality_delete_writer.rs
- mod.rs
- position_delete_writer.rs
- sort_writer.rs
- base_file_writer.rs
- mod.rs
- parquet_writer.rs
- track_writer.rs
- dispatcher_writer.rs
- equality_delta_writer.rs
- mod.rs
- partition_writer.rs
- precompute_partition_writer.rs
- prometheus.rs
- builder_helper.rs
- input_wrapper.rs
- location_generator.rs
- mod.rs
- mod.rs
- to_array.rs
- to_primitive.rs
- types.rs
- mod.rs
- to_arrow.rs
- manifest_file.rs
- manifest_list.rs
- mod.rs
- partition_spec.rs
- schema.rs
- snapshot.rs
- sort_order.rs
- table_metadata.rs
- transform.rs
- types.rs
- value.rs
- bucket.rs
- identity.rs
- mod.rs
- temporal.rs
- truncate.rs
- void.rs
- in_memory.rs
- mod.rs
- partition_splitter.rs
- to_avro.rs
- config.rs
- error.rs
- lib.rs
- metrics.rs
- table.rs
- transaction.rs
- docker.rs
- mod.rs
- poetry.rs
- test_generator.rs
- delta_test.rs
- delta_test_v2.rs
- insert_compact_test.rs
- insert_tests.rs
- insert_tests_v2.rs
- precompute_partition_test.rs
- rest_catalog_tests.rs
- scan_test.rs
- Cargo.toml
- README.md
- 1.csv
- 2.csv
- 3.csv
- docker-compose.yaml
- docker-compose.yaml
- mc_init.sh
- minio_server.sh
- .gitignore
- init-table.sql
- insert-table.sql
- inspect-table.sql
- query-table.sql
- run-sql-file.sh
- spark-connect-server.sh
- spark-connect-server2.sh
- 00000-0-1971bd95-8f45-4b60-a703-207eed9b48b2-00001.parquet
- 00000-11-8a16a1fe-46b8-4571-99df-46f7a7d457bf-00001.parquet
- 00000-3-e54bbc88-2087-49ce-abf4-0c4be3636b97-00001.parquet
- 00000-7-45268d71-54eb-476c-b42c-942d880c04a1-00001.parquet
- 00001-1-fb24490d-a162-42a8-9b0e-06b17ee00d87-00001.parquet
- 00001-12-8aef3353-a097-457a-a70b-1f99018548ca-00001.parquet
- 00001-4-eb1a8e9a-3848-4b91-9361-ace1d137ec89-00001.parquet
- 00001-8-0bd1dd3f-91c6-4f7c-b96c-24cfe217ce73-00001.parquet
- 00002-2-4905bb62-7b5a-4b16-991b-e1dff961ca9d-00001.parquet
- 00002-5-0b2f22de-dc14-471d-a71d-4d1c6d3d47cc-00001.parquet
- 00002-9-d3b554b1-5252-43f1-ab41-28b5a91ba963-00001.parquet
- 00003-10-b0fcc339-e01a-4fc1-8f75-09acf00a9839-00001.parquet
- 00000-74d4bfd7-0dee-46ab-85c5-ef90df8b2c20.metadata.json
- 00001-e40071cb-570f-4f62-a206-072677f67865.metadata.json
- 00002-0e3f3480-280c-49c2-8f18-67061f464330.metadata.json
- 00003-762873ab-ac0f-4c0a-a34c-d033c11790cb.metadata.json
- 00004-624c12b5-ca17-4804-91ae-40f8622b406c.metadata.json
- 00005-032145b7-6a0c-4a53-bc3d-b7b571ccab3b.metadata.json
- 172ea76d-6692-4e39-a239-81b8d467c50f-m0.avro
- 172ea76d-6692-4e39-a239-81b8d467c50f-m1.avro
- 172ea76d-6692-4e39-a239-81b8d467c50f-m2.avro
- b51968b3-8837-4c13-a9ec-74d71bac3565-m0.avro
- ba3e3b20-02d8-47b7-8aba-5432b9aa72b9-m0.avro
- cb85d90a-3e9f-44ae-90da-2391c97d9b9c-m0.avro
- snap-2966623707104393227-1-172ea76d-6692-4e39-a239-81b8d467c50f.avro
- snap-402819370943546960-1-cb85d90a-3e9f-44ae-90da-2391c97d9b9c.avro
- snap-6788296308394418127-1-ba3e3b20-02d8-47b7-8aba-5432b9aa72b9.avro
- snap-8380191719297762539-1-b51968b3-8837-4c13-a9ec-74d71bac3565.avro
- README.md
- 00010-1-d5c93668-1e52-41ac-92a6-bba590cbf249-00001.parquet
- 00000-8e744276-bd72-4cb1-a7c8-c5bb250c5fa4.metadata.json
- 00001-d73c6ed6-f422-4ee7-9d41-39fd748f4026.metadata.json
- b60343c9-e792-4e41-993e-0f3cd2aab484-m0.avro
- snap-8205833995881562618-1-b60343c9-e792-4e41-993e-0f3cd2aab484.avro
- check.py
- check_pyiceberg.py
- check_result.py
- execute.py
- init.py
- poetry.lock
- pyproject.toml
- .00000-0-b8982382-f016-467a-84e4-5e6bbe0ff19a-00001.parquet.crc
- .00001-1-b8982382-f016-467a-84e4-5e6bbe0ff19a-00001.parquet.crc
- .00002-2-b8982382-f016-467a-84e4-5e6bbe0ff19a-00001.parquet.crc
- 00000-0-b8982382-f016-467a-84e4-5e6bbe0ff19a-00001.parquet
- 00001-1-b8982382-f016-467a-84e4-5e6bbe0ff19a-00001.parquet
- 00002-2-b8982382-f016-467a-84e4-5e6bbe0ff19a-00001.parquet
- .10d28031-9739-484c-92db-cdf2975cead4-m0.avro.crc
- .snap-1646658105718557341-1-10d28031-9739-484c-92db-cdf2975cead4.avro.crc
- .v1.metadata.json.crc
- .v2.metadata.json.crc
- .version-hint.text.crc
- 10d28031-9739-484c-92db-cdf2975cead4-m0.avro
- snap-1646658105718557341-1-10d28031-9739-484c-92db-cdf2975cead4.avro
- v1.metadata.json
- v2.metadata.json
- version-hint.text
- no_partition_test.toml
- partition_day_test.toml
- partition_hash_test.toml
- partition_hour_test.toml
- partition_identity_test.toml
- partition_month_test.toml
- partition_truncate_test.toml
- partition_year_test.toml
- docker-compose.yml
- README.md
- .gitignore
- Cargo.toml
- CHANGELOG.md
- LICENSE
- README.md
- rust-toolchain.toml
- rustfmt.toml
# Installation Guide
1. Get the code
git clone https://github.com/icelake-io/icelake
Downloads the entire project code from GitHub to your computer.
cd icelake
Moves into the project folder you just downloaded.
2. Docker
Easy RecommendedPrerequisites
- Git Needed to download the project code from GitHub.
- Docker Desktop Needed to build and run containers. Install it and keep it running in the background.
β οΈ This is a large repository, so this method may point to an internal sub-package rather than the actual core product. Check the full README as well.
docker compose -f testdata/docker/iceberg-fs/docker-compose.yaml up -d --build
Runs the command against the services defined in the compose file.
Run docker compose ps to check the containers are Up. If the README mentions a port, open http://localhost:PORT in your browser.
3. Maven (Java)
MediumPrerequisites
- Git Needed to download the project code from GitHub.
- JDK (Java) Required to build and run Java projects.
- Maven The build tool used for the mvn command.
β οΈ This is a large repository, so this method may point to an internal sub-package rather than the actual core product. Check the full README as well.
cd docker/spark
This project's files live in a subfolder, so move into it first.
mvn clean install
Installs dependencies and builds the project using Maven.
A BUILD SUCCESS message means it worked. The output is created under target/.
4. Python
EasyPrerequisites
β οΈ This is a large repository, so this method may point to an internal sub-package rather than the actual core product. Check the full README as well.
pip install .
Installs the package published on PyPI directly β no need to clone the source.
python <μ€νν νμΌλͺ
>.py # READMEμμ μ νν μ€ν νμΌλͺ
μ νμΈνμΈμ
Runs the Python script (or module).
If it runs without errors and prints output in the terminal, it worked.
5. Rust
MediumPrerequisites
- Git Needed to download the project code from GitHub.
- Rust (rustup) Installing via rustup also installs cargo.
cargo build --release
Compiles the Rust project.
cargo run
Builds and then immediately runs the program.
If cargo build finishes without errors, it worked. The executable is created under target/.
// repository documentation
Was this content helpful?
(0 ratings)