modin
Modin: Scale your Pandas workflows by changing a single line of code
File Explorer
Download Latest Version (.zip)- action.yml
- action.yml
- action.yml
- action.yml
- action.yml
- action.yml
- action.yml
- action.yml
- bug-report.yaml
- feature_request.md
- question.md
- codeql-config.yml
- set_up_sql_server.sh
- ci-notebooks.yml
- ci-required.yml
- ci.yml
- codeql.yml
- fuzzydata-test.yml
- publish-to-pypi.yml
- push-to-main.yml
- dependabot.yaml
- PULL_REQUEST_TEMPLATE.md
- stale.yml
- __init__.py
- csv.py
- parquet.py
- __init__.py
- scalability_benchmarks.py
- __init__.py
- common.py
- compatibility.py
- data_shapes.py
- __init__.py
- benchmarks.py
- __init__.py
- test_utils.py
- asv.conf.dask.json
- asv.conf.json
- asv.conf.unidist.json
- README.md
- build-docker.py
- comment_on_pr.py
- Dockerfile.teamcity-ci
- clone_my_modin.png
- clone_upstream_modin.png
- conda_prompt.png
- contributing.md
- pre-commit
- Dockerfile
- custom.js
- layout.html
- architecture.rst
- contributing.rst
- index.rst
- partition_api.rst
- using_pandas_on_dask.rst
- using_pandas_on_mpi.rst
- using_pandas_on_python.rst
- using_pandas_on_ray.rst
- axis_partition.rst
- dataframe.rst
- index.rst
- dtypes.rst
- index.rst
- axis_partition.rst
- partition.rst
- partition_manager.rst
- dataframe.rst
- index.rst
- algebra.rst
- index.rst
- partition.rst
- partition_manager.rst
- virtual_partition.rst
- dataframe.rst
- index.rst
- axis_partition.rst
- partition.rst
- partition_manager.rst
- dataframe.rst
- index.rst
- axis_partition.rst
- partition.rst
- partition_manager.rst
- dataframe.rst
- index.rst
- generic.rst
- axis_partition.rst
- partition.rst
- partition_manager.rst
- dataframe.rst
- index.rst
- generic.rst
- dispatching.rst
- index.rst
- query_compiler.rst
- index.rst
- parsers.rst
- query_compiler.rst
- index.rst
- pandas.rst
- index.rst
- batch.rst
- index.rst
- pandas.rst
- range_partitioning_groupby.rst
- reshuffling_groupby.rst
- sklearn.rst
- xgboost.rst
- base.rst
- dataframe.rst
- series.rst
- config.rst
- utils.rst
- using_modin.rst
- using_modin_cluster.rst
- using_modin_locally.rst
- modin_vs_dask_vs_koalas.rst
- out_of_core.rst
- pandas.rst
- why_modin.rst
- examples.rst
- faq.rst
- installation.rst
- quickstart.rst
- troubleshooting.rst
- full_axis_function.svg
- partitioning_examples.svg
- repartition_square_frames.svg
- square_partitioning.svg
- drop_duplicates_16cpus.jpg
- drop_duplicates_44cpus.jpg
- merge_h2o_500mb.jpg
- merge_h2o_5gb.png
- nunique_16cpus.jpg
- resample_16cpus.jpg
- unique_16cpus.jpg
- unique_44cpus.jpg
- 10000_meter.png
- api_coverage_comparison.svg
- binary_evaluation.svg
- block_partitions_diagram.png
- component_view.png
- convert_to_pandas.png
- factory_dispatching.svg
- generic_data_egress.svg
- generic_data_ingress.svg
- generic_data_transform.svg
- Import.gif
- map_evaluation.svg
- Modin.gif
- modin_architecture.png
- modin_architecture_diagram.png
- modin_cluster.png
- modin_in_the_cloud_architecture.png
- modin_multicore.png
- Modin_Pandas_Import.gif
- Modin_Speedup.svg
- modin_spreadsheet_example.png
- modin_spreadsheet_mini_demo.gif
- modin_spreadsheets_installation.png
- modin_sql_google_play_table.png
- modin_sql_google_play_ur_table.png
- modin_sql_logo.png
- MODIN_ver2.ico
- MODIN_ver2.png
- MODIN_ver2_hrz.png
- Pandas.gif
- pandas_multicore.png
- pandas_on_dask_data_egress.svg
- pandas_on_dask_data_ingress.svg
- pandas_on_dask_data_transform.svg
- pandas_on_python_data_egress.svg
- pandas_on_python_data_ingress.svg
- pandas_on_python_data_transform.svg
- pandas_on_ray_data_egress.svg
- pandas_on_ray_data_ingress.svg
- pandas_on_ray_data_transform.svg
- pandas_on_unidist_data_egress.svg
- pandas_on_unidist_data_ingress.svg
- pandas_on_unidist_data_transform.svg
- performance-all-supported.svg
- performance-not-all-supported.svg
- powered_by_ray_big.svg
- powered_by_ray_small.svg
- progress_bar.gif
- progress_bar_example.png
- query_planner.png
- quickstart_speedup.svg
- read_csv_benchmark.png
- reduce_evaluation.svg
- simplified_query_flow.svg
- release_notes-0.14.0.rst
- release_notes-0.15.0.rst
- release_notes-0.16.0.rst
- release_notes-template.rst
- dataframe_supported.rst
- defaulting_to_pandas.rst
- index.rst
- io_supported.rst
- older_pandas_compat.rst
- series_supported.rst
- utilities_supported.rst
- batch.rst
- index.rst
- modin_engines.rst
- modin_logging.rst
- modin_metrics.rst
- modin_xgboost.rst
- progress_bar.rst
- spreadsheets_api.rst
- index.rst
- index.rst
- range_partitioning_ops.rst
- benchmarking.rst
- index.rst
- integrations.rst
- conf.py
- contact.rst
- ecosystem.rst
- index.rst
- release-procedure.md
- requirements-doc.txt
- boston_housing.csv
- census_1k.csv
- nyc-taxi_1k.csv
- plasticc_test_set_1k.csv
- plasticc_test_set_metadata_1k.csv
- plasticc_training_set_1k.csv
- plasticc_training_set_metadata_1k.csv
- build-docker-image.sh
- census.py
- Dockerfile
- nyc-taxi.py
- plasticc.py
- taxi.pstat
- altair.ipynb
- bokeh.ipynb
- huggingface.ipynb
- matplotlib.ipynb
- NLTK.ipynb
- plotly.ipynb
- seaborn.ipynb
- sklearn.ipynb
- statsmodels.ipynb
- tensorflow.ipynb
- xgboost.ipynb
- Modin_Taxi.ipynb
- Pandas_Taxi.ipynb
- requirements.txt
- tutorial.ipynb
- exercise_5.ipynb
- exercise_1.ipynb
- exercise_2.ipynb
- exercise_3.ipynb
- exercise_4.ipynb
- test_notebooks.py
- Dockerfile
- requirements.txt
- exercise_5.py
- modin-cluster.yaml
- README.md
- exercise_1.ipynb
- exercise_2.ipynb
- exercise_3.ipynb
- exercise_4.ipynb
- test_notebooks.py
- Dockerfile
- requirements.txt
- exercise_1.ipynb
- exercise_2.ipynb
- exercise_3.ipynb
- exercise_4.ipynb
- test_notebooks.py
- Dockerfile
- jupyter_unidist_env.yml
- README.md
- setup_kernel.py
- utils.py
- convert_to_pandas.png
- hdk_convert_to_pandas.png
- modin_cluster.png
- modin_multicore.png
- MODIN_ver2_hrz.png
- pandas_multicore.png
- read_csv_perf.png
- README.md
- README.md
- modin-scikit-learn-example.ipynb
- quickstart.ipynb
- __init__.py
- __main__.py
- envvars.py
- pubsub.py
- __init__.py
- align.py
- check.py
- common.py
- engines.py
- eval.py
- expr.py
- ops.py
- parsing.py
- scope.py
- __init__.py
- binary.py
- cat.py
- dataframe.py
- datetime.py
- default.py
- groupby.py
- list.py
- resample.py
- rolling.py
- series.py
- str.py
- struct.py
- __init__.py
- binary.py
- fold.py
- groupby.py
- map.py
- operator.py
- reduce.py
- tree_reduce.py
- __init__.py
- dataframe.py
- utils.py
- __init__.py
- dataframe.py
- utils.py
- __init__.py
- __init__.py
- axis_partition.py
- __init__.py
- __init__.py
- dataframe.py
- utils.py
- __init__.py
- buffer.py
- column.py
- dataframe.py
- exception.py
- from_dataframe.py
- __init__.py
- __init__.py
- dtypes.py
- index.py
- __init__.py
- axis_partition.py
- partition.py
- partition_manager.py
- __init__.py
- utils.py
- __init__.py
- __init__.py
- engine_wrapper.py
- utils.py
- __init__.py
- dataframe.py
- __init__.py
- io.py
- __init__.py
- partition.py
- partition_manager.py
- virtual_partition.py
- __init__.py
- __init__.py
- __init__.py
- __init__.py
- dispatcher.py
- factories.py
- __init__.py
- __init__.py
- engine_wrapper.py
- __init__.py
- dataframe.py
- __init__.py
- io.py
- __init__.py
- partition.py
- partition_manager.py
- virtual_partition.py
- __init__.py
- __init__.py
- __init__.py
- __init__.py
- deferred_execution.py
- engine_wrapper.py
- utils.py
- __init__.py
- io.py
- __init__.py
- partition_manager.py
- __init__.py
- __init__.py
- dataframe.py
- __init__.py
- io.py
- __init__.py
- partition.py
- partition_manager.py
- virtual_partition.py
- __init__.py
- __init__.py
- __init__.py
- __init__.py
- engine_wrapper.py
- utils.py
- __init__.py
- io.py
- __init__.py
- partition_manager.py
- __init__.py
- __init__.py
- dataframe.py
- __init__.py
- io.py
- __init__.py
- partition.py
- partition_manager.py
- virtual_partition.py
- __init__.py
- __init__.py
- __init__.py
- __init__.py
- modin_aqp.py
- utils.py
- __init__.py
- column_store_dispatcher.py
- feather_dispatcher.py
- hdf_dispatcher.py
- parquet_dispatcher.py
- __init__.py
- sql_dispatcher.py
- __init__.py
- csv_dispatcher.py
- excel_dispatcher.py
- fwf_dispatcher.py
- json_dispatcher.py
- text_file_dispatcher.py
- utils.py
- __init__.py
- file_dispatcher.py
- io.py
- __init__.py
- doc_utils.py
- query_compiler.py
- query_compiler_calculator.py
- __init__.py
- aggregations.py
- groupby.py
- merge.py
- native_query_compiler.py
- parsers.py
- query_compiler.py
- query_compiler_caster.py
- utils.py
- __init__.py
- __init__.py
- __init__.py
- partitions.py
- __init__.py
- __init__.py
- __init__.py
- pipeline.py
- __init__.py
- __init__.py
- __init__.py
- __init__.py
- __init__.py
- __init__.py
- __init__.py
- __init__.py
- __init__.py
- __init__.py
- glob_dispatcher.py
- __init__.py
- sql_dispatcher.py
- utils.py
- __init__.py
- csv_glob_dispatcher.py
- custom_text_dispatcher.py
- __init__.py
- __init__.py
- parsers.py
- __init__.py
- __init__.py
- __init__.py
- __init__.py
- io.py
- __init__.py
- train_test_split.py
- __init__.py
- __init__.py
- general.py
- __init__.py
- datasets.py
- __init__.py
- utils.py
- xgboost.py
- xgboost_ray.py
- __init__.py
- __init__.py
- class_logger.py
- config.py
- logger_decorator.py
- metrics.py
- __init__.py
- arr.py
- array_creation.py
- array_shaping.py
- constants.py
- indexing.py
- linalg.py
- logic.py
- math.py
- trigonometry.py
- utils.py
- __init__.py
- extensions.py
- __init__.py
- __init__.py
- __init__.py
- __init__.py
- __init__.py
- accessor.py
- base.py
- dataframe.py
- general.py
- groupby.py
- indexing.py
- io.py
- iterator.py
- plotting.py
- resample.py
- series.py
- series_utils.py
- utils.py
- window.py
- __init__.py
- base.py
- dataframe.py
- groupby.py
- lazyframe.py
- series.py
- __init__.py
- classes.py
- functions.py
- __init__.py
- classes.py
- __init__.py
- test_envvars.py
- test_parameter.py
- test_internals.py
- test_gpu_managers.py
- test_internals.py
- test_internals.py
- __init__.py
- test_dispatcher.py
- __init__.py
- test_general.py
- __init__.py
- test_dataloader.py
- __init__.py
- test_default.py
- test_dmatrix.py
- test_xgboost.py
- __init__.py
- test_fuzzydata.py
- test_io_exp.py
- test_pipeline.py
- __init__.py
- test_sanity.py
- test_utils.py
- __init__.py
- test_protocol.py
- __init__.py
- test_general.py
- __init__.py
- __init__.py
- test_array.py
- test_array_arithmetic.py
- test_array_axis_functions.py
- test_array_creation.py
- test_array_indexing.py
- test_array_linalg.py
- test_array_logic.py
- test_array_math.py
- test_array_shaping.py
- utils.py
- part-00000-a7bff54c-2ff4-4654-9783-626542bd3a90-c000.snappy.parquet
- part-00001-a7bff54c-2ff4-4654-9783-626542bd3a90-c000.snappy.parquet
- part-00002-a7bff54c-2ff4-4654-9783-626542bd3a90-c000.snappy.parquet
- 44c5b23d806c4dc8a97d70c4fb2219f5-0.parquet
- 44c5b23d806c4dc8a97d70c4fb2219f5-0.parquet
- test_data0.csv
- test_data1.csv
- part_0.parquet
- part_1.parquet
- part_10.parquet
- part_11.parquet
- part_12.parquet
- part_13.parquet
- part_14.parquet
- part_15.parquet
- part_2.parquet
- part_3.parquet
- part_4.parquet
- part_5.parquet
- part_6.parquet
- part_7.parquet
- part_8.parquet
- part_9.parquet
- __init__.py
- airline.sas7bdat
- blah.csv
- every_other_row_nan.xlsx
- excel_sheetname_title.xlsx
- issue_1930.csv
- issue_2074.csv
- issue_2239.csv
- issue_3119.csv
- issue_4543.csv
- issue_976.csv
- modin_error_book.xlsx
- newlines.csv
- test_border_rows.xlsx
- test_categories.csv
- test_categories.json
- test_data.feather
- test_data.fwf
- test_data.json
- test_data.parquet
- test_delim.csv
- test_different_columns_in_rows.json
- test_empty_rows.xlsx
- test_emptyline.xlsx
- test_null_col.csv
- test_time_parsing.csv
- test_usecols.csv
- __init__.py
- test_binary.py
- test_default.py
- test_indexing.py
- test_iter.py
- test_join_sort.py
- test_map_metadata.py
- test_pickle.py
- test_reduce.py
- test_udf.py
- test_window.py
- __init__.py
- conftest.py
- test_api_reexport.py
- test_base_extensions.py
- test_dataframe_extensions.py
- test_groupby_extensions.py
- test_pd_extensions.py
- test_series_extensions.py
- __init__.py
- test_lazy_import.py
- __init__.py
- test_benchmark_mode.py
- __init__.py
- conftest.py
- test_binary.py
- test_compiler_caster.py
- test_copy_on_write.py
- test_default.py
- test_default_to_pandas_without_warnings.py
- test_general.py
- test_indexing.py
- test_iter.py
- test_join_sort.py
- test_map_metadata.py
- test_pickle.py
- test_window.py
- utils.py
- __init__.py
- conftest.py
- test_api.py
- test_backend.py
- test_concat.py
- test_expanding.py
- test_general.py
- test_groupby.py
- test_io.py
- test_repartition.py
- test_reshape.py
- test_rolling.py
- test_series.py
- utils.py
- test_dataframe.py
- __init__.py
- test_dataframe_api_standard.py
- test_docstring_urls.py
- test_envvar_catcher.py
- test_envvar_npartitions.py
- test_executions_api.py
- test_headers.py
- test_logging.py
- test_metrics.py
- test_partition_api.py
- test_utils.py
- __init__.py
- __main__.py
- _version.py
- conftest.py
- db_conn.py
- error_message.py
- utils.py
- env_unidist_linux.yml
- env_unidist_win.yml
- requirements-no-engine.yml
- __init__.py
- examples.py
- test_doc_checker.py
- __init__.py
- doc_checker.py
- release.py
- kaggle10.py
- kaggle12.py
- kaggle13.py
- kaggle14.py
- kaggle17.py
- kaggle18.py
- kaggle19.py
- kaggle20.py
- kaggle22.py
- kaggle3.py
- kaggle4.py
- kaggle5.py
- kaggle6.py
- kaggle7.py
- kaggle8.py
- kaggle9.py
- run_stress_tests.sh
- test_kaggle_ipynb.py
- .gitattributes
- .gitignore
- .readthedocs.yaml
- CODE_OF_CONDUCT.md
- codecov.yml
- CODEOWNERS
- environment-dev.yml
- LICENSE
- LICENSE_HEADER
- MANIFEST.in
- modin-autoimport-pandas.pth
- mypy.ini
- NOTICE
- README.md
- requirements-dev.txt
- setup.cfg
- setup.py
- versioneer.py
π Installation Guide
1. Get the code
git clone https://github.com/modin-project/modin
Downloads the entire project code from GitHub to your computer.
cd modin
Moves into the project folder you just downloaded.
2. Official Install Script
Easy RecommendedPrerequisites
- Python 3 Python is required to use pip.
pip install "modin[all]" # (Recommended) Install Modin with Ray and Dask engines.
Installs the package published on PyPI directly β no need to clone the source.
pip install "modin[ray]" # Install Modin dependencies and Ray.
Installs the package published on PyPI directly β no need to clone the source.
pip install "modin[dask]" # Install Modin dependencies and Dask.
Installs the package published on PyPI directly β no need to clone the source.
pip install "modin[mpi]" # Install Modin dependencies and MPI through unidist.
Installs the package published on PyPI directly β no need to clone the source.
After installing, open a new terminal and run the program's version command (e.g. --version) to confirm it worked.
Pulled directly from this repo's README.
3. Docker
EasyPrerequisites
- Git Needed to download the project code from GitHub.
- Docker Desktop Needed to build and run containers. Install it and keep it running in the background.
docker build -f ci/teamcity/Dockerfile.teamcity-ci -t modin .
Builds a runnable image based on the Dockerfile.
docker run -p 8080:80 modin
Runs the built image as an actual container.
Run docker compose ps to check the containers are Up. If the README mentions a port, open http://localhost:PORT in your browser.
4. Python
EasyPrerequisites
pip install "modin[all]" # (Recommended) Install Modin with Ray and Dask engines.
Installs the package published on PyPI directly β no need to clone the source.
pip install "modin[ray]" # Install Modin dependencies and Ray.
Installs the package published on PyPI directly β no need to clone the source.
pip install "modin[dask]" # Install Modin dependencies and Dask.
Installs the package published on PyPI directly β no need to clone the source.
pip install "modin[mpi]" # Install Modin dependencies and MPI through unidist.
Installs the package published on PyPI directly β no need to clone the source.
If it runs without errors and prints output in the terminal, it worked.
Pulled directly from this repo's README.
// repository documentation
Was this content helpful?
(0 ratings)