polars-bio
Blazing-Fast Bioinformatic Operations on Python DataFrames
File Explorer
Download Latest Version (.zip)- config.toml
- apply.md
- archive.md
- proposal.md
- bench_pushdown.py
- SKILL.md
- linkedin-template.md
- SKILL.md
- devcontainer.json
- Dockerfile
- bug_report.yml
- config.yml
- documentation.yml
- feature_request.yml
- benchmark.yml
- claude-code-review.yml
- claude.yml
- publish_documentation.yml
- publish_to_pypi.yml
- release.yml
- PULL_REQUEST_TEMPLATE.md
- ARCHITECTURE.md
- CONCERNS.md
- CONVENTIONS.md
- INTEGRATIONS.md
- STACK.md
- STRUCTURE.md
- TESTING.md
- 01-01-PLAN.md
- 01-01-SUMMARY.md
- 01-02-PLAN.md
- 01-CONTEXT.md
- 01-RESEARCH.md
- ARCHITECTURE.md
- FEATURES.md
- PITFALLS.md
- STACK.md
- SUMMARY.md
- config.json
- PROJECT.md
- REQUIREMENTS.md
- ROADMAP.md
- STATE.md
- bench_wallcpu.sh
- both2.sh
- correctness_verify.sh
- gen_post.py
- matched_bench2.sh
- mem_measure.sh
- parse_all.py
- parse_drift.py
- pb_mem.py
- pb_one.py
- pb_reps.py
- pb_wallcpu.py
- pbk.sh
- pbt2.py
- rebin_compare.py
- reps.sh
- rq_scaling.sh
- run_all.sh
- run_drift.sh
- thread_invariance.sh
- bench.py
- parity.py
- compare_benchmark_results.sh
- generate_comparison_charts.py
- generate_interactive_comparison.py
- parse_benchmark_results.py
- README_BENCHMARKS.md
- recipe.yaml
- build.sh
- meta.yaml
- README.md
- auxiliary.md
- index.md
- operations.md
- reading.md
- sql.md
- writing.md
- polars-bio-overlap-mem.png
- polars-bio-overlap-pd-mem.png
- polars-bio-overlap-pl-mem.png
- bioframe.png
- bioframe_sink.png
- polars-bio.png
- polars-bio_sink.png
- polars-bio_stream_sink.png
- pyranges0.png
- pyranges0_sink.png
- pyranges1.png
- pyranges1_sink.png
- count-overlaps-parallel.png
- count-overlaps-single.png
- coverage-parallel.png
- coverage-single.png
- eccb-2026.png
- format-bam.png
- format-fastq.png
- format-scaling.png
- format-vcf.png
- logo-large-dark.png
- logo-large.png
- logo.png
- nearest-parallel.png
- nearest-single.png
- overlap-parallel.png
- overlap-single.png
- results-nearest-0.1.1.png
- results-overlap-0.1.1.png
- summary-results.png
- bcf-reader-memory.png
- bcf-reader-time.png
- bcf-thread-scaling.png
- all_operations_speedup_comparison.png
- polars_bio_scalability_8_7.png
- all_operations_walltime_comparison.png
- bench-20250-all_operations_speedup_comparison.png
- bench_parallel_speedup_combined_8-7.png
- benchmark_comparison_genomicranges_vs_polars_bio.png
- benchmark_speedup_comparison_genomicranges_vs_polars_bio.png
- combined_benchmark_visualization.png
- combined_multi_testcase.png
- star-history-202595.png
- one_thread_all_operations_2_1.png
- one_thread_all_operations_8_7.png
- thread_scalability_cluster.png
- thread_scalability_complement.png
- thread_scalability_count_overlaps.png
- thread_scalability_coverage.png
- thread_scalability_merge.png
- thread_scalability_nearest.png
- thread_scalability_overlap.png
- thread_scalability_subtract.png
- bam_with_tags.png
- bam_without_tags.png
- fastq_comparison.png
- thread_scaling.png
- vcf_with_info.png
- vcf_without_info.png
- bcf-one-thread.png
- bgen-one-thread.png
- bgen-scaling.png
- pgen-one-thread.png
- scaling-all-formats.png
- general_performance.png
- memory_comparison.png
- seminar.png
- thread_scalability.png
- version_comparison.png
- bcf-genotype-readers-2026-08.md
- benchmark-operations-2025-09.md
- benchmark-operations-2026-02.md
- dataframe-paths-benchmark-2026-04.md
- fastqc-benchmark-2026-07.md
- genomic-formats-benchmark-2026-02.md
- genomic-formats-benchmark-2026-07.md
- gff-read-optimizations-2025-09.md
- release-0.23.0.md
- release-0.26.0.md
- release-0.31.0-vcf-zarr.md
- rewriting-bioinformatics-2026-07.md
- index.md
- cloud.md
- dataframes.md
- index.md
- operations.md
- parallel.md
- reading.md
- sql.md
- writing.md
- .part-00000-47fafbb5-1cab-410c-9461-d10effacf760-c000.snappy.parquet.crc
- .part-00001-47fafbb5-1cab-410c-9461-d10effacf760-c000.snappy.parquet.crc
- _SUCCESS
- part-00000-47fafbb5-1cab-410c-9461-d10effacf760-c000.snappy.parquet
- part-00001-47fafbb5-1cab-410c-9461-d10effacf760-c000.snappy.parquet
- .part-00000-a0d75244-2d87-41eb-a3eb-a18847c7cb87-c000.snappy.parquet.crc
- .part-00001-a0d75244-2d87-41eb-a3eb-a18847c7cb87-c000.snappy.parquet.crc
- _SUCCESS
- part-00000-a0d75244-2d87-41eb-a3eb-a18847c7cb87-c000.snappy.parquet
- part-00001-a0d75244-2d87-41eb-a3eb-a18847c7cb87-c000.snappy.parquet
- example.bam
- example.bed.bgz
- example.fastq.gz
- example.gff3.bgz
- example.vcf
- tutorial.ipynb
- extra.css
- 2026-05-08-vcf-zarr-implementation.md
- 2026-05-09-vcf-zarr-benchmark-notebook-implementation.md
- 2026-05-10-vcf-zarr-typed-values-implementation.md
- 2026-06-26-gff-gtf-attributes-sentinel.md
- 2026-06-27-robust-predicate-pushdown.md
- 2026-07-02-fastqc-per-tile-and-kmer.md
- 2026-07-02-fastqc-phase1-vertical-slice.md
- 2026-08-17-pgen-polars-bio-binding.md
- 2026-05-08-vcf-zarr-design.md
- 2026-05-09-vcf-zarr-benchmark-notebook-design.md
- 2026-06-26-gff-gtf-attributes-sentinel-design.md
- 2026-06-27-robust-predicate-pushdown-design.md
- 2026-07-01-mimalloc-allocator-impact-design.md
- 2026-07-02-fastqc-datafusion-udtf-design.md
- 2026-07-02-fastqc-per-tile-and-kmer-design.md
- 2026-07-04-rewriting-bioinformatics-blog-design.md
- 2026-07-27-adoption-zero-config-first-run-design.md
- 2026-08-17-pgen-polars-bio-binding-design.md
- contact.md
- developers.md
- faq.md
- index.md
- performance.md
- quickstart.md
- requirements.txt
- supplement.md
- talks.md
- versions.json
- .env
- start.sh
- stop.sh
- policy-anonymous.json
- policy-priv.json
- test.fasta
- vep.vcf
- vep.vcf.bgz
- docker-compose.yml
- it_ensembl_vcf_bgz.py
- it_object_storage_io.py
- README.md
- feature_counting_benchmark.ipynb
- vcf_zarr_benchmark.ipynb
- vcf_zarr_small_benchmark.ipynb
- spec.md
- design.md
- proposal.md
- tasks.md
- spec.md
- design.md
- proposal.md
- tasks.md
- spec.md
- proposal.md
- tasks.md
- spec.md
- design.md
- proposal.md
- tasks.md
- spec.md
- design.md
- proposal.md
- tasks.md
- spec.md
- design.md
- proposal.md
- tasks.md
- spec.md
- design.md
- proposal.md
- tasks.md
- spec.md
- design.md
- proposal.md
- tasks.md
- spec.md
- proposal.md
- tasks.md
- spec.md
- design.md
- proposal.md
- tasks.md
- spec.md
- design.md
- proposal.md
- tasks.md
- spec.md
- proposal.md
- tasks.md
- spec.md
- AGENTS.md
- project.md
- header.html
- __init__.py
- _metadata.py
- _path_utils.py
- constants.py
- context.py
- exceptions.py
- fastqc_op.py
- interval_op_helpers.py
- io.py
- logging.py
- metadata_extractors.py
- operations.py
- pileup_op.py
- polars_ext.py
- predicate_translator.py
- pushdown.py
- range_op.py
- range_op_helpers.py
- range_op_io.py
- range_utils.py
- sql.py
- sql_predicate_builder.py
- utils.py
- generate_benchmark_metadata.py
- generate_dataframe_paths_benchmark_figures.py
- update_benchmark_index.py
- context.rs
- fastqc.rs
- lib.rs
- operation.rs
- option.rs
- pileup.rs
- query.rs
- scan.rs
- utils.rs
- write.rs
- reads.csv
- targets.csv
- reads.csv
- targets.csv
- .part-00000-47fafbb5-1cab-410c-9461-d10effacf760-c000.snappy.parquet.crc
- .part-00001-47fafbb5-1cab-410c-9461-d10effacf760-c000.snappy.parquet.crc
- _SUCCESS
- part-00000-47fafbb5-1cab-410c-9461-d10effacf760-c000.snappy.parquet
- part-00001-47fafbb5-1cab-410c-9461-d10effacf760-c000.snappy.parquet
- .part-00000-a0d75244-2d87-41eb-a3eb-a18847c7cb87-c000.snappy.parquet.crc
- .part-00001-a0d75244-2d87-41eb-a3eb-a18847c7cb87-c000.snappy.parquet.crc
- _SUCCESS
- part-00000-a0d75244-2d87-41eb-a3eb-a18847c7cb87-c000.snappy.parquet
- part-00001-a0d75244-2d87-41eb-a3eb-a18847c7cb87-c000.snappy.parquet
- 10x_pbmc_tags.bam
- 10x_pbmc_tags.bam.bai
- mapq255.bam
- multi_chrom.bam
- multi_chrom.bam.bai
- NA12878_10k.bam
- NA12878_10k.bam.bai
- NA12878_10k_samtools_depth.tsv.gz
- nanopore_custom_tags.bam
- nanopore_custom_tags.bam.bai
- test.bam
- test.bam.bai
- annotations.bb
- annotations.bed
- chrom.sizes
- large_signal.bw
- signal.bedgraph
- signal.bw
- VALIDATION.md
- antku_small.bcf
- antku_small.bcf.csi
- ensembl-2.bcf
- ensembl-2.bcf.csi
- ensembl.bcf
- genotype_missing.bcf
- genotype_missing.bcf.csi
- info_bare_key.bcf
- info_bare_key.bcf.csi
- info_bare_key_realdata.bcf
- info_bare_key_realdata.bcf.csi
- info_invalid_flag_value.bcf
- info_invalid_flag_value.bcf.csi
- info_missing_array.bcf
- info_missing_array.bcf.csi
- multi_chrom.bcf
- multi_chrom.bcf.csi
- multisample.bcf
- multisample.bcf.csi
- multisample_large.bcf
- multisample_large.bcf.csi
- README.md
- single_sample_collision.bcf
- single_sample_collision.bcf.csi
- vep.bcf
- vep.bcf.csi
- vep_annotate_test.bcf
- vep_annotate_test.bcf.csi
- chr16_fragile_site.bed
- chr16_fragile_site.bed.bgz
- ENCFF001XKR.bed.gz
- test.bed
- missing_calls.bgen
- multisample.bgen
- multisample.sample
- chr20.fa
- chr20.fa.fai
- test_chr20.cram
- huffman_byte_encoding.cram
- multi_chrom.cram
- multi_chrom.cram.crai
- nanopore_custom_tags.cram
- nanopore_custom_tags.cram.crai
- test.cram
- test.cram.crai
- example.fasta
- test.fasta
- adapter_mix.nogroup.fastqc_data.txt
- dup_mix.nogroup.fastqc_data.txt
- example.nogroup.fastqc_data.txt
- kmer_mix.nogroup.kmers.fastqc_data.txt
- per_tile_mix.nogroup.fastqc_data.txt
- adapter_mix.fastq
- dup_mix.fastq
- example.fastq
- example.fastq.bgz
- example.fastq.bgz.gzi
- example.fastq.gz
- kmer_mix.fastq
- multimember_clean.fastq.gz
- multimember_pigz.fastq.gz
- multimember_split.fastq.gz
- per_tile_mix.fastq
- sample_no_index.fastq.bgz
- sample_parallel.fastq.bgz
- sample_parallel.fastq.bgz.gzi
- test.fastq
- wrong_extension.fastq.gz
- 1_multi_chrom.gff3.gz
- 1_multi_chrom.gff3.gz.tbi
- chrY_test_subset.gff3.bgz
- gencode.v38.annotation.gff3
- gencode.v38.annotation.gff3.bgz
- gencode.v38.annotation.gff3.bgz.gzi
- gencode.v38.annotation.gff3.gz
- multi_chrom.gff3.gz
- multi_chrom.gff3.gz.tbi
- wrong_extension.gff3.gz
- test.gtf
- multi_chrom.pairs.gz
- multi_chrom.pairs.gz.tbi
- test.pairs
- dosage.pgen
- dosage.psam
- dosage.pvar
- oracle.pgen
- oracle.psam
- oracle.pvar
- phase.pgen
- phase.psam
- phase.pvar
- unused_alt.pgen
- unused_alt.psam
- unused_alt.pvar
- test.sam
- antku_small.vcf.gz
- ensembl-2.vcf
- ensembl.vcf
- genotype_missing.vcf
- info_bare_key.vcf
- info_bare_key_realdata.vcf
- info_invalid_flag_value.vcf
- info_missing_array.vcf
- multi_chrom.vcf.gz
- multi_chrom.vcf.gz.tbi
- multisample.vcf
- multisample.vcf.gz
- single_sample_collision.vcf
- vep.vcf
- vep.vcf.bgz
- vep.vcf.gz
- vep_annotate_test.vcf
- wrong_extension.vcf.bgz
- wrong_extension.vcf.gz
- .zarray
- .zattrs
- 0
- .zarray
- .zattrs
- 0
- .zarray
- .zattrs
- 0
- .zarray
- .zattrs
- 0
- .zarray
- .zattrs
- 0.0
- .zarray
- .zattrs
- .zarray
- .zattrs
- 0.0
- .zarray
- .zattrs
- 0.0
- .zarray
- .zattrs
- 0
- .zarray
- .zattrs
- 0
- .zarray
- .zattrs
- 0
- .zarray
- .zattrs
- 0.0
- .zarray
- .zattrs
- 0
- .zarray
- .zattrs
- .zarray
- .zattrs
- 0
- .zarray
- .zattrs
- 0
- .zarray
- .zattrs
- 0
- .zattrs
- .zgroup
- input.csv
- reads.csv
- targets.csv
- reads.csv
- targets.csv
- _expected.py
- conftest.py
- test_bcf_io.py
- test_bgen_io.py
- test_bioframe.py
- test_comprehensive_metadata.py
- test_context_options.py
- test_coordinate_system_metadata.py
- test_custom_tag_inference.py
- test_execution_plan_validation.py
- test_fasta_write.py
- test_fastq_write.py
- test_fastqc.py
- test_fastqc_correctness.py
- test_fastqc_golden.py
- test_fastqc_parity.py
- test_filter_select_attributes_bug_fix.py
- test_gff_eager_vs_lazy.py
- test_io.py
- test_io_bam.py
- test_io_bbi.py
- test_io_bbi_parity.py
- test_io_bbi_parity_dependency.py
- test_io_bbi_streaming.py
- test_io_bed.py
- test_io_cram.py
- test_io_cram_huffman.py
- test_io_fasta.py
- test_io_fastq.py
- test_io_gff.py
- test_io_gtf.py
- test_io_indexed.py
- test_io_pairs.py
- test_io_vcf.py
- test_issue_395_eager_lazy_segfault.py
- test_lazy_streaming_fix.py
- test_lazyframe_partitioning.py
- test_native.py
- test_optimization_bug_fix.py
- test_overlap_algorithms.py
- test_overlap_output_mode.py
- test_pandas.py
- test_parallel_io.py
- test_partitioned_range_operation_regressions.py
- test_pgen_io.py
- test_pileup.py
- test_pileup_samtools.py
- test_polars.py
- test_polars_bio_projection_validation.py
- test_polars_ext.py
- test_predicate_in_between.py
- test_predicate_pushdown.py
- test_predicate_pushdown_chrY_start.py
- test_predicate_translator_units.py
- test_projection_performance.py
- test_projection_pushdown.py
- test_pushdown_equivalence.py
- test_pushdown_helpers.py
- test_source_metadata.py
- test_streaming.py
- test_suffix_handling.py
- test_user_scenario.py
- test_vcf_format_columns.py
- test_vcf_info_bare_keys.py
- test_vcf_info_missing_values.py
- test_vcf_parsing.py
- test_vcf_projection_pushdown.py
- test_vcf_read_options.py
- test_vcf_write.py
- test_vcf_zarr_io.py
- test_warnings.py
- test_wide_dataframes.py
- .gitignore
- .pre-commit-config.yaml
- .readthedocs.yaml
- AGENTS.md
- Cargo.lock
- Cargo.toml
- CHANGELOG.md
- CLAUDE.md
- CODE_OF_CONDUCT.md
- HANDOVER-pgen-benchmarks.md
- HANDOVER-pgen-perf.md
- LICENSE
- Makefile
- mkdocs.yml
- polars-bio.iml
- pyproject.toml
- README.md
- requirements.txt
- rust-toolchain.toml
- rustfmt.toml
- uv.lock
# Installation Guide
1. Get the code
git clone https://github.com/biodatageeks/polars-bio
Downloads the entire project code from GitHub to your computer.
cd polars-bio
Moves into the project folder you just downloaded.
2. Docker
Easy RecommendedPrerequisites
- Git Needed to download the project code from GitHub.
- Docker Desktop Needed to build and run containers. Install it and keep it running in the background.
docker compose -f it/docker-compose.yml up -d --build
Runs the command against the services defined in the compose file.
Run docker compose ps to check the containers are Up. If the README mentions a port, open http://localhost:PORT in your browser.
3. Python
EasyPrerequisites
pip install -r requirements.txt
Installs the Python libraries listed in requirements.txt (or similar).
jupyter notebook
Launches Jupyter in your browser so you can open and run the notebook (.ipynb) files.
If it runs without errors and prints output in the terminal, it worked.
4. Rust
MediumPrerequisites
- Git Needed to download the project code from GitHub.
- Rust (rustup) Installing via rustup also installs cargo.
cargo build --release
Compiles the Rust project.
cargo run
Builds and then immediately runs the program.
If cargo build finishes without errors, it worked. The executable is created under target/.
5. Make
MediumPrerequisites
- Git Needed to download the project code from GitHub.
- Make Usually pre-installed on Linux/macOS. On Windows, install separately (e.g. via MSYS2 or WSL).
make
Compiles the code based on the generated build configuration to produce an executable.
If it finishes without errors, it worked. Try running the generated executable directly.
// repository documentation
Was this content helpful?
(0 ratings)
