darima

KO | EN

gitlite — search

#javascript #ai #typescript #llm #golang #rust #deep-learning #linux #nodejs #awesome #kubernetes #pytorch #security #mcp

(★ 27)

Distributed ARIMA Models

darima Latest Version Download

Download Latest Version (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# `darima`

Distributed ARIMA Models implemented with Apache Spark

# Introduction

DARIMA is designed to facilitate forecasting ultra-long time series by utilizing the industry-standard MapReduce framework. The algorithm is developed on Spark platform and both Python as well as R interfaces.

See [`darima`](darima) for developed functions used for implementing DARIMA models.
- [`model.py`](darima/model.py) : Train ARIMA models for each subseries and convert the trained models into AR representations (Mapper).
- [`dlsa.py`](darima/dlsa.py) : Combine the local estimators obtained in Mapper by minimizing the global loss function (Reducer).
- [`forecast.py`](darima/forecast.py) : Forecast the next H observations by utilizing the combined estimators.
- [`evaluation.py`](darima/evaluation.py) : Calculate the forecasting accuracy in terms as MASE, sMAPE and MSIS.
- [`R`](darima/R) : R functions designed for modeling, combining and forecasting. [rpy2](https://pypi.org/project/rpy2/) is needed as an interface to use R from Python.

# Modern `dstats` Interface

This directory keeps the original DARIMA implementation and examples for
reference. New development is moving to the unified `dstats` package at
<https://github.com/feng-li/dstats>.

The migrated Spark 4 interface is available from `dstats.darima` and uses the
current PySpark `applyInPandas` API. Automatic ARIMA selection is handled in
Python with `statsforecast.models.AutoARIMA`, so the new interface does not
require `rpy2` or R `forecast::auto.arima`.

Typical usage in the new package is:

```py
from dstats.darima import darima_forecast
from dstats.darima import darima_mapreduce
from dstats.darima import fit_darima_partitions
from dstats.darima import model_eval
```

# System requirements

- `Spark >= 2.3.1`
- `Python >= 3.7.0`
    - `pyspark >= 2.3.1`
    - `rpy2 >= 3.0.4`
    - `scikit-learn >= 0.21.2`
    - `numpy >= 1.16.3`
    - `pandas >= 0.23.4`
- `R >= 3.5.2`
    - `forecast >= 8.5`
    - `polynom = 1.3.9`
    - `dplyr >= 0.8.4`
    - `quantmod >= 0.4.13`
    - `magrittr >= 1.5`

# Usage

## DARIMA
Run the [PySpark](https://spark.apache.org/docs/latest/api/python/index.html) code to forecast the time series of the GEFCom2017 by utilizing DARIMA.

```sh
  ./bash/run_darima.sh
```
or simply run
```py
  PYSPARK_PYTHON=/usr/local/bin/python3.7 ARROW_PRE_0_15_IPC_FORMAT=1 spark-submit ./run_darima.py
```
**Note**: `ARROW_PRE_0_15_IPC_FORMAT=1` is added to instruct `PyArrow >= 0.15.0` to use the legacy IPC format with the older Arrow Java that is in Spark 2.3.x and 2.4.x.

## ARIMA
Run the R code to forecast the time series of the GEFCom2017 by utilizing the `auto.arima()` function (used for comparison).
```sh
  ./bash/auto_arima.sh
```
or simply run
```r
  Rscript auto_arima.R
```

# References

- [Xiaoqian Wang](https://xqnwang.rbind.io), [Yanfei Kang](https://yanfei.site), [Rob J Hyndman](https://robjhyndman.com), & [Feng Li](https://feng.li/) (2022) Distributed ARIMA models for ultra-long time series. International Journal of Forecasting [DOI: 10.1016/j.ijforecast.2022.05.001](https://doi.org/10.1016/j.ijforecast.2022.05.001).

darima

darima Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

darima

darima Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

Download README