btrblocks

KO | EN

gitlite — search

#ai-agent #python #mcp-server #dsh-plugin #agentic-ai #automation #genpark #llm-tools #deepseek-harness #ai-agents #dsh #claude-code

★ 286 Open GitHub ↗

BtrBlocks: Efficient Columnar Compression for Data Lakes (SIGMOD 2023 Paper)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# BtrBlocks - Efficient Columnar Compression for Data Lakes

[![Paper](https://img.shields.io/badge/paper-SIGMOD%202023-green)](https://bit.ly/btrblocks)
[![Build](https://github.com/maxi-k/btrblocks/actions/workflows/test.yml/badge.svg?event=push)](https://github.com/maxi-k/btrblocks/actions/workflows/test.yml)

- [Paper](https://bit.ly/btrblocks) (two-column version)
- [Video](https://dl.acm.org/doi/10.1145/3589263) (SIGMOD 2023 presentation)

## Usage

After [building](#building) the library, follow one of the [examples](./tools/examples) to get started.

## Components

- `btrblocks/`: the compression library, schemes, utilities, ...
- `btrfiles/`: helper library for binary files and yaml schema information
- `tools/`: various conversion, measurement and benchmarking tools
- `test/`: rudimentary tests for the library

![Dependency Graph](https://raw.githubusercontent.com/maxi-k/btrblocks/master/doc/dependencies.svg)

## Building

We currently require an x86 platform.
This library was built and tested on linux only.

``` sh
mkdir build 
cd build
cmake ..
```

Then, depending on your usecase, build only the library or any of the tools:
- build everything: `make`
- install static library and headers on your system: `sudo make install`
- build the compression library only: `make btrblocks`
- build the tests `make tester`
- build the in-memory decompression speed benchmark: `make decompression_speed`
- ...

For a list of all valid targets, run `make help`.

Library was built and tested on Linux (x86, ARM) and MacOS (ARM).

## Contributors

Adnan Alhomssi
David Sauerwein
Maximilian Kuschewski

## License

MIT - See [License File](LICENSE)

btrblocks

Explore Similar Repositories

Was this content helpful?

Recent Feedback

Download README