KO
|
EN
gitlite — search
Search
#typescript
#javascript
#react
#hacktoberfest
#python
#swift
#blockchain
#embedded
#nodejs
#website
#csharp
#authentication
csv2parquet
★ 65
Open GitHub ↗
Convert a CSV to a parquet file.
Download README (.md)
Explore Similar Repositories
parquet-floor
:
A lightweight Java library that facilitates reading and writing Apache Parquet files without Hadoop dependencies
hyparquet-writer
:
Apache Parquet file writer in JavaScript
node-parquet
:
NodeJS module to access apache parquet format files
vscode-parquet-viewer
:
A VS Code extension to view Apache Parquet files as JSON
elixir-arrow
:
Experimental Elixir bindings for Apache Arrow including Parquet and DataFusion
// repository documentation
Was this content helpful?
★ 0
(0 ratings)
Select Rating:
★
★
★
★
★
Submit Feedback
Recent Feedback
×
Download README
Do you want to download the
README.md
file for
csv2parquet
?
Download (.md)
# csv2parquet [](https://travis-ci.org/cldellow/csv2parquet) [](https://codecov.io/gh/cldellow/csv2parquet) Convert a CSV to a parquet file. You may also find [sqlite-parquet-vtable](https://github.com/cldellow/sqlite-parquet-vtable) or [parquet-metadata](https://github.com/cldellow/parquet-metadata) useful. ## Installing If you just want to use the tool: ``` sudo pip install pyarrow csv2parquet ``` If you want to clone the repo and work on the tool, install its dependencies via pipenv: ``` pipenv install ``` ## Usage Next, create some Parquet files. The tool supports CSV and TSV files. ``` usage: csv2parquet [-h] [-n ROWS] [-r ROW_GROUP_SIZE] [-o OUTPUT] [-c CODEC] [-i INCLUDE [INCLUDE ...] | -x EXCLUDE [EXCLUDE ...]] [-R RENAME [RENAME ...]] [-t TYPE [TYPE ...]] csv_file positional arguments: csv_file input file, can be CSV or TSV optional arguments: -h, --help show this help message and exit -n ROWS, --rows ROWS The number of rows to include, useful for testing. -r ROW_GROUP_SIZE, --row-group-size ROW_GROUP_SIZE The number of rows per row group. -o OUTPUT, --output OUTPUT The parquet file -c CODEC, --codec CODEC The compression codec to use (brotli, gzip, snappy, zstd, none) -i INCLUDE [INCLUDE ...], --include INCLUDE [INCLUDE ...] Include the given columns (by index or name) -x EXCLUDE [EXCLUDE ...], --exclude EXCLUDE [EXCLUDE ...] Exclude the given columns (by index or name) -R RENAME [RENAME ...], --rename RENAME [RENAME ...] Rename a column. Specify the column to be renamed and its new name, eg: 0=age or person_age=age -t TYPE [TYPE ...], --type TYPE [TYPE ...] Parse a column as a given type. Specify the column and its type, eg: 0=bool? or person_age=int8. Parse errors are fatal unless the type is followed by a question mark. Valid types are string (default), base64, bool, float32, float64, int8, int16, int32, int64, timestamp ``` ## Testing ``` pylint csv2parquet pytest ```
