DevOps-Python-tools

(β˜… 823)

80+ DevOps & Data CLI Tools - AWS, GCP, GCF Python Cloud Functions, Log Anonymizer, Spark, Hadoop, HBase, Hive, Impala, Linux, Docker, Spark Data Converters & Validators (Avro/Parquet/JSON/CSV/INI/XML/YAML), Travis CI, AWS CloudFormation, Elasticsearch, Solr etc.

File Explorer

Download Latest Version (.zip)
  • .appveyor.yml
  • .checkov.yaml
  • .cirrus.yml
  • .coafile
  • .drone.yml
  • .editorconfig
  • .envrc
  • .envrc-python
  • .flake8
  • .gitallowed
  • .gitignore
  • .gitlab-ci.yml
  • .gitmodules
  • .ipython-notebook-pyspark.00-pyspark-setup.py
  • .ipython-notebook-pyspark.ipython_notebook_config.py.j2
  • .mdl.rb
  • .mdlrc
  • .pre-commit-config.yaml
  • .pylintrc
  • .pyup.yml
  • .sonarcloud.properties
  • .travis.yml
  • .yamllint
  • ambari_ams_metrics.sh
  • ambari_blueprints.py
  • ambari_cancel_all_requests.sh
  • ambari_trigger_service_checks.py
  • anonymize.py
  • anonymize_custom.conf
  • anonymize_ignore.conf
  • anonymize_parallel.sh
  • aws_s3_presign.py
  • aws_users_access_key_age.py
  • aws_users_last_used.py
  • aws_users_pw_last_used.py
  • aws_users_unused_access_keys.py
  • azure-pipelines.yml
  • bash-tools
  • bitbucket-pipelines.yml
  • boot
  • buddy.yml
  • center.py
  • cloudera_navigator_tables_used.py
  • cloudera_navigator_tables_used_postgres.py
  • codefresh.yml
  • crunch_accounting_csv_statement_converter.py
  • docker_pull_all_images.sh
  • docker_pull_all_images_all_tags.sh
  • docker_pull_all_tags.sh
  • docker_registry_show_tags.py
  • dockerfiles_check_git_branches.py
  • dockerfiles_check_git_tags.py
  • dockerhub_search.py
  • dockerhub_show_tags.py
  • find_active_apache_drill.py
  • find_active_apache_drill2.py
  • find_active_cassandra.py
  • find_active_elasticsearch.py
  • find_active_hadoop2_namenode.py
  • find_active_hadoop_namenode.py
  • find_active_hadoop_yarn_resource_manager.py
  • find_active_hbase_master.py
  • find_active_hbase_stargate.py
  • find_active_hbase_thrift.py
  • find_active_impala.py
  • find_active_impala_catalog.py
  • find_active_impala_statestore.py
  • find_active_kubernetes_api.py
  • find_active_oozie.py
  • find_active_presto_coordinator.py
  • find_active_server.py
  • find_active_solrcloud.py
  • find_duplicate_files.py
  • find_missing_files_in_sequence.py
  • gcp_service_account_credential_keys.py
  • getent.py
  • git_check_branches_upstream.py
  • hbase_compact_tables.py
  • hbase_flush_tables.py
  • hbase_generate_data.py
  • hbase_region_requests.py
  • hbase_regions_by_memstore_size.py
  • hbase_regions_by_size.py
  • hbase_regions_least_used.py
  • hbase_regionserver_requests.py
  • hbase_scan_table_column_names.sh
  • hbase_show_table_region_ranges.py
  • hbase_table_region_row_distribution.py
  • hbase_table_regions_by_regionserver.sh
  • hbase_table_regionserver_distribution.sh
  • hbase_table_row_key_distribution.py
  • hbase_uncompacted_regions.sh
  • hdfs_files_native_checksums.jy
  • hdfs_files_stats.jy
  • hdfs_find_replication_factor_1.py
  • hdfs_time_block_reads.jy
  • headtail.py
  • hexanonymize.py
  • hive_compute_table_stats.py
  • hive_foreach_table.py
  • hive_schemas_csv.py
  • hive_tables_column_counts.py
  • hive_tables_list.py
  • hive_tables_locations.py
  • hive_tables_metadata.py
  • hive_tables_null_columns.py
  • hive_tables_null_rows.py
  • hive_tables_row_column_counts.py
  • hive_tables_row_counts.py
  • hive_tables_row_counts_any_nulls.py
  • impala_compute_table_stats.py
  • impala_foreach_table.py
  • impala_schemas_csv.py
  • impala_tables_column_counts.py
  • impala_tables_list.py
  • impala_tables_locations.py
  • impala_tables_metadata.py
  • impala_tables_null_columns.py
  • impala_tables_null_rows.py
  • impala_tables_row_column_counts.py
  • impala_tables_row_counts.py
  • impala_tables_row_counts_any_nulls.py
  • ipython_notebook_pyspark.py
  • Jenkinsfile
  • json_docs_to_bulk_multiline.py
  • json_to_xml.py
  • json_to_yaml.py
  • jython_autoinstall.exp
  • jython_install.sh
  • kics.config
  • LICENSE
  • Makefile
  • opentsdb_import_metric_distribution.py
  • opentsdb_list_metrics.sh
  • opentsdb_list_metrics_hbase.sh
  • pig-text-to-elasticsearch.pig
  • pig-text-to-solr.pig
  • pig_udfs.jy
  • plot_uk_marriage_rates.py
  • pylib
  • python_find_library_path.py
  • pythonpath.py
  • quay_show_tags.py
  • README.md
  • requirements.txt
  • selenium_hub_browser_test.py
  • serf_event_handler.py
  • sonar-project.properties
  • spark_avro_to_parquet.py
  • spark_csv_to_avro.py
  • spark_csv_to_parquet.py
  • spark_json_to_avro.py
  • spark_json_to_parquet.py
  • spark_parquet_to_avro.py
  • sql
  • strip_ansi_escape_codes.py
  • templates
  • timeout.py
  • travis_debug_session.py
  • travis_last_log.py
  • urldecode.py
  • urlencode.py
  • validate_avro.py
  • validate_cson.py
  • validate_csv.py
  • validate_ini.py
  • validate_ini2.py
  • validate_json.py
  • validate_ldap_ldif.py
  • validate_multimedia.py
  • validate_parquet.py
  • validate_properties.py
  • validate_toml.py
  • validate_xml.py
  • validate_yaml.py
  • welcome.py
  • xml_to_json.py
  • xml_to_yaml.py
  • yaml_to_json.py

πŸš€ Installation Guide

1. Get the code
git clone https://github.com/HariSekhon/DevOps-Python-tools

Downloads the entire project code from GitHub to your computer.

cd DevOps-Python-tools

Moves into the project folder you just downloaded.

2. Official Install Script

Easy Recommended
Prerequisites
  • Homebrew A package manager for macOS/Linux.
curl -L https://git.io/python-bootstrap | sh

Downloads and runs the official install script in one line β€” this handles the full setup automatically.

brew install openssl wget

Installs the pre-built package via Homebrew β€” no source build required.

βœ… After installing, open a new terminal and run the program's version command (e.g. --version) to confirm it worked.

Pulled directly from this repo's README.

3. Python

Easy
Prerequisites
  • Git Needed to download the project code from GitHub.
  • Python 3 On Windows, be sure to check 'Add Python to PATH' during installation.
sudo pip install -r requirements.txt

Installs the Python libraries listed in requirements.txt (or similar).

sudo pip install --index https://host.domain.com/api/pypi/repo/simple --trusted host.domain.com -r requirements.txt

Installs the Python libraries listed in requirements.txt (or similar).

sudo pip install --proxy hari:mypassword@proxy-host:8080 -r requirements.txt

Installs the Python libraries listed in requirements.txt (or similar).

sudo pip install --upgrade pip==9.0.1

Installs the Python libraries listed in requirements.txt (or similar).

sudo pip install --upgrade pip

Installs the Python libraries listed in requirements.txt (or similar).

βœ… If it runs without errors and prints output in the terminal, it worked.

Pulled directly from this repo's README.

4. Make

Medium
Prerequisites
  • Git Needed to download the project code from GitHub.
  • Make Usually pre-installed on Linux/macOS. On Windows, install separately (e.g. via MSYS2 or WSL).
make

Compiles the code based on the generated build configuration to produce an executable.

make anonymize.pyc

Compiles the code based on the generated build configuration to produce an executable.

make jython

Compiles the code based on the generated build configuration to produce an executable.

make update

Compiles the code based on the generated build configuration to produce an executable.

make test

Compiles the code based on the generated build configuration to produce an executable.

βœ… If it finishes without errors, it worked. Try running the generated executable directly.

Pulled directly from this repo's README.

Today's picks

Front-End-Checklist β˜… 73,540

πŸ—‚ The essential checklist for modern web development, for humans and AI agents

MDX
superset β˜… 74,282

Apache Superset is a Data Visualization and Data Exploration Platform

Python
tabby β˜… 73,948

A terminal for a more modern age

TypeScript
Prompt-Engineering-Guide β˜… 77,504

πŸ™ Guides, papers, lessons, notebooks and resources for prompt engineering, context enginee

MDX
json-server β˜… 75,692

Get a full fake REST API with zero coding in less than 30 seconds (seriously)

JavaScript
transformers β˜… 164,163

πŸ€— Transformers: the model-definition framework for state-of-the-art machine learning model

Python
// repository documentation