awesome-data-quality

KO | EN

gitlite — search

#javascript #react #ai #go #deep-learning #linux #docker #kubernetes #awesome-list #vue #security #openai #ios #open-source

awesome-data-quality

(★ 15)

Curated list of tools and frameworks assisting in monitoring data quality

awesome-data-quality 최신버젼 다운로드

최종 버전 다운로드 (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# awesome-data-quality

A curated list of awesome tools for testing and monitoring data quality - typically at the data warehouse/lake or within running data pipelines.

_If you want to contribute to this list (please do), send me a pull request or [contact me](https://mobile.twitter.com/orikabeli)._

## Table of Contents
TBD
### Frameworks and Libraries

#### Open sourced
* [elementary](https://github.com/elementary-data/elementary) - Data monitoring and observability tailored to dbt.
* [mobydq](https://github.com/ubisoft/mobydq) -  tool for data engineering teams to run & automate data quality checks on their data pipeline.
* [ydata-quality](https://github.com/ydataai/ydata-quality) - python library for assessing data quality throughout stages of the data pipeline development.
* [great-expectations](https://github.com/great-expectations/great_expectations) - tool for data testing, documentation, and profiling.
* [deepqu](https://github.com/awslabs/python-deequ) - libray by Amazon for defining unit tests for data with focus on large datasets. Based on Apache Spark.
* [soda](https://github.com/sodadata/soda-core) - enables data testing through extended SQL queries.
* [dqm](https://github.com/piotr-kalanski/data-quality-monitoring) - another data quality monitoring tool implemented using Spark.
* [owl-sanitizer](https://github.com/ronald-smith-angel/owl-data-sanitizer) - yet another Spark based lightweight data validation framework.
* [griffin](https://github.com/apache/griffin) - Data Quality solution for distributed data systems at any scale in both streaming and batch data context. 
* [drunken-data-quality](https://github.com/FRosner/drunken-data-quality)
* [DataQuality for BigData](https://github.com/agile-lab-dev/DataQuality)
* [TopNotch](https://github.com/blackrock/TopNotch)
* [Phasor Data Quality Tracker](https://github.com/GridProtectionAlliance/pdqtracker)
* [DataCleaner](https://github.com/datacleaner/DataCleaner)
* [data-quality](https://github.com/Talend/data-quality)

##### Geared for ML
* [deepchecks](https://github.com/deepchecks/deepchecks) - tool for validating your machine learning models and data. Implemented test suites tailored towards ML models datasets and outputs.
* [evidently](https://github.com/evidentlyai/evidently) - analyze and track data and ML model output quality.

##### Pipelines with data quality included
* [dbt](https://docs.getdbt.com/docs/building-a-dbt-project/tests), [dataform](https://dataform.co/blog/data-assertions) - ELT tools that comes with a handy utility to define tests as SQL queries.

#### Paid
Offering ranges from data to pipelines testing, with focus on real-time monitoring, automation of tests creation & threshold setting, and addditional enterprise features.
* [Bigeye](https://bigeye.com)
* [Soda](https://soda.io)
* [Databand](https://databand.ai)
* [Monte Carlo](https://montecarlodata.com)
* [great expectations](https://greatexpectations.io)
* [Sifflet](https://siffletapp.com)
* [Validio](https://validio.io)
* [Lightup](https://lightup.ai)
* [Lantern](https://lantern.so)
* [Metaplane](https://metaplane.dev)
* [Datafold](https://datafold.com)
* [Acceldata](https://acceldata.io)
* [Anomalo](https://anomalo.com)
* [Marquez](https://marquezproject.github.io)

TODOs
* Add tools for unstructured data (Arthur, Robust)
*

awesome-data-quality

awesome-data-quality 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

awesome-data-quality

awesome-data-quality 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

Download README