iceberg-cxx

KO | EN

gitlite — search

#javascript #python #hacktoberfest #react #ai #typescript #llm #go #golang #android #machine-learning #rust #deep-learning #linux

(★ 93)

Apache Iceberg C++ library

iceberg-cxx Latest Version Download

Download Latest Version (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

Apache Iceberg C++ library

# Apache Iceberg™ C++

You do not need Java to use [Apache Iceberg™](https://iceberg.apache.org/).

It's an alternative for [iceberg-cpp](https://github.com/apache/iceberg-cpp).
The library is used by [Tea](https://github.com/lithium-tech/tea), an open-source extension for Greenplum that allows it to read Iceberg data from S3 compatible storage using HMS and Nessie catalogs.

## Supported features

Source https://iceberg.apache.org/status/

### Data types

| Data type  | Iceberg version | Cxx | Java |
|--------------------------|---|-----|-----|
| boolean                  | 2 | +   | +   |
| int                      | 2 | +   | +   |
| float                    | 2 | +   | +   |
| double                   | 2 | +   | +   |
| decimal                  | 2 | +   | +   |
| date                     | 2 | +   | +   |
| time                     | 2 | +   | +   |
| timestamp                | 2 | +   | +   |
| timestamptz              | 2 | +   | +   |
| timestamp_ns             | 3 | +   | +   |
| timestamptz_ns           | 3 | +   | +   |
| string                   | 2 | +   | +   |
| uuid                     | 2 | +   | +   |
| fixed                    | 2 | +   | +   |
| binary                   | 2 | +   | +   |
| variant                  | 3 | -   | +   |
| list                     | 2 | +   | +   |
| map                      | 2 | -   | +   |
| struct                   | 2 | -   | +   |
| unknown                  | 3 | +   | ?   |

#### Datetime types restrictions

Datetime restrictions are defined in [Iceberg spec](https://iceberg.apache.org/spec/?h=timestamp_ns#parquet).
For `date` underlying type is `int32`. For `time*` it's `int64`.
`timestamp` and `timestamptz` store microseconds from `1970-01-01 00:00:00.000000`.
`timestamp_ns` and `timestamptz_ns` store nanoseconds from `1970-01-01 00:00:00.000000000`.
`tz` suffix means the time is adjusted to UTC.

### Data files format

| File format   | Cxx | Java |
|---------------|-----|-----|
| Parquet       | +   | +   |
| ORC           | -   | +   |
| Puffin        | +   | +   |
| Avro          | -   | +   |

### File IO

| File IO            | Cxx | Java |
|--------------------|-----|-----|
| Local Filesystem   | +   | +   |
| Hadoop Filesystem  | -   | +   |
| S3 Compatible      | +   | +   |
| GCS Compatible     | -   | +   |
| ADLS Compatible    | -   | +   |

### Table Maintenance Operations

Not implemented

### Table Update Operations

Not implemented

### Table read operations

| Operation        | Iceberg version | Cxx | Java |
|------------------------------|-----|-----|-----|
| Plan with data file          | 1,2 | +   | +   |
| Plan with position deletes   |   2 | +   | +   |
| Plan with equality deletes   |   2 | +   | +   |
| Plan with puffin statistics  | 1,2 | -   | +   |
| Read data file               | 1,2 | +   | +   |
| Read with position deletes   |   2 | +   | +   |
| Read with equality deletes   |   2 | +   | +   |

### Table write operations

| Operation     | Iceberg version | Cxx | Java |
|---------------------------|-----|-----|-----|
| Append data               | 1,2 | +   | +   |
| Write position deletes    |   2 | -   | +   |
| Write equality deletes    |   2 | -   | +   |
| Write deletion vectors    |   3 | +   | +   |

### Catalog

| Table Operation           | Nessie | Glue | HMS | REST |
|----------------------------|-----|-----|-----|-----|
| listTable                  | -   | -   | -   | -   |
| createTable                | -   | -   | -   | -   |
| dropTable                  | -   | -   | -   | -   |
| loadTable                  | +-  | -   | +-  | +-  |
| updateTable                | -   | -   | -   | -   |
| renameTable                | -   | -   | -   | -   |
| tableExists                | -   | -   | -   | -   |
| createView                 | -   | -   | -   | -   |
| dropView                   | -   | -   | -   | -   |
| listView                   | -   | -   | -   | -   |
| viewExists                 | -   | -   | -   | -   |
| replaceView                | -   | -   | -   | -   |
| renameView                 | -   | -   | -   | -   |
| listNamespaces             | -   | -   | -   | -   |
| createNamespace            | -   | -   | -   | -   |
| dropNamespace	             | -   | -   | -   | -   |
| namespaceExists            | -   | -   | -   | -   |
| updateNamespaceProperties  | -   | -   | -   | -   |
| loadNamespaceMetadata      | -   | -   | -   | -   |

## Requirements

- C++20 compliant compiler
- CMake 3.20 or higher
- OpenSSL

### Build, Run Test

You have to [download Apache Arrow dependencies](https://arrow.apache.org/docs/15.0/developers/cpp/building.html#offline-builds) first.
```bash
mkdir _deps && cd _deps
git clone --single-branch -b maint-15.0.2 https://github.com/apache/arrow.git
cd arrow && git apply ../../vendor/arrow/fix_c-ares_url.patch && cd ..
./arrow/cpp/thirdparty/download_dependencies.sh ./arrow-thirdparty
```

```bash
mkdir _build
cd _build
ln -s ../_deps/arrow-thirdparty arrow-thirdparty
cmake -GNinja ../
ninja
cd tests/
../iceberg/iceberg-cpp-test
../iceberg/common/fs/iceberg_common_fs_test
./iceberg_local_test
```

iceberg-cxx

iceberg-cxx Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

iceberg-cxx

iceberg-cxx Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

Download README