Stable-Neural-SDEs

KO | EN

gitlite — search

#javascript #python #ai #golang #machine-learning #rust #deep-learning #linux #nodejs #java #windows #awesome-list #security #open-source

Stable-Neural-SDEs

(★ 77)

[ICLR 2024] Stable Neural Stochastic Differential Equations in Analyzing Irregular Time Series Data

Stable-Neural-SDEs 최신버젼 다운로드

최종 버전 다운로드 (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# Stable Neural Stochastic Differential Equations (Neural SDEs)
This repository contains the PyTorch implementation for the paper [Stable Neural Stochastic Differential Equations in Analyzing Irregular Time Series Data](https://openreview.net/forum?id=4VIgNuQ1pY). Spotlight presentation (Notable Top 5%).

Paper (ICLR 2024): https://openreview.net/forum?id=4VIgNuQ1pY · arXiv: https://arxiv.org/abs/2402.14989

> Oh, Y., Lim, D., & Kim, S. (2024). Stable Neural Stochastic Differential Equations in Analyzing Irregular Time Series Data. The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024. https://openreview.net/forum?id=4VIgNuQ1pY

---

## **Code architecture**
The code for each experiment is meticulously organized into separate folders, aligned with the original references used for implementation.

- `Benchmark_classification`: PhysioNet Sepsis and Speech Commands, implemented from Kidger, P. et al. (2020) [1] (https://github.com/patrick-kidger/NeuralCDE)
- `Benchmark_interpolation`: PhysionNet Mortality, implemented from Shukla, S. et al. (2021) [2] (https://github.com/reml-lab/mTAN)
- `Benchmark_forecasting`: MuJoCo Foresting task, implemented from Jhin, S. et al. (2021) [3] (https://github.com/sheoyon-jhin/ANCDE)
- `torch_ists`: motivated from Kidger, P., et al. (2020) [1], we develop new python/pytorch wrapper for extensive experiments on robustness to missing data. Please refer the up-to-date version: [torch_ists](https://github.com/yongkyung-oh/torch-ists).

[1] Kidger, P., Morrill, J., Foster, J., & Lyons, T. (2020). Neural controlled differential equations for irregular time series. Advances in Neural Information Processing Systems, 33, 6696-6707.

[2] Shukla, S. N., & Marlin, B. M. (2021). Multi-Time Attention Networks for Irregularly Sampled Time Series. In 9th International Conference on Learning Representations (ICLR 2021). https://openreview.net/forum?id=4c0J6lwQ4_

[3] Jhin, S. Y., Shin, H., Hong, S., Jo, M., Park, S., Park, N., ... & Jeon, S. (2021, December). Attentive Neural Controlled Differential Equations for Time-series Classification and Forecasting. In 2021 IEEE International Conference on Data Mining (ICDM) (pp. 250-259). IEEE Computer Society.

--

For example, the `diffusion_model` class in `NSDE/benchmark_classification/models_sde/neuralsde.py` implements the Neural SDEs proposed in the study.

- `input_option`: This setting is crucial for the diffusion term. It dictates whether to combine the original observation with a controlled path or not. This flexibility allows for experimenting with how the model handles various types of input data, particularly in the context of irregular time series.
- `noise_option`: It controls the diffusion function in the SDE. Options include constant, additive, multiplicative, among others. The ability to manipulate the diffusion function is key to exploring how different noise models affect the performance and stability of the proposed Neural SDEs.

**Proposed methods (implementation with combinations)**
- Neural SDE: `neuralsde_3_18`
- Neural LSDE: `neuralsde_2_16`
- Neural LNSDE: `neuralsde_4_17`
- Neural GSDE: `neuralsde_6_17`

**Proposed methods (implementation with simple example)**

Please refer the [tutorial](https://github.com/yongkyung-oh/Stable-Neural-SDEs/tree/main/tutorial) for the pure-theory notebook set. The tutorial set currently contains 10 notebooks.

| Model | Formulation | Example |
|-------|-------------|---------|
| Neural ODE | $dz(t) = f(t, z(t); \theta_f) \, dt$ | [example](https://github.com/yongkyung-oh/Stable-Neural-SDEs/blob/main/tutorial/simple%20OU%20process%20-%20Neural%20ODE.ipynb) |
| Neural CDE | $dz(t) = f(t, z(t); \theta_f) \, dX(t)$ | [example](https://github.com/yongkyung-oh/Stable-Neural-SDEs/blob/main/tutorial/simple%20OU%20process%20-%20Neural%20CDE.ipynb) |
| Neural SDE | $dz(t) = f(t, z(t); \theta_f) \, dt + g(t, z(t); \theta_g) \, dW_t$ | [example](https://github.com/yongkyung-oh/Stable-Neural-SDEs/blob/main/tutorial/simple%20OU%20process%20-%20Neural%20SDE.ipynb) |
| Neural SDE + KLD | $dz(t) = f(t, z(t); \theta_f) \, dt + g(t, z(t); \theta_g) \, dW_t$ | [example](https://github.com/yongkyung-oh/Stable-Neural-SDEs/blob/main/tutorial/simple%20OU%20process%20-%20Neural%20SDE%20%28%2B%20KLD%29.ipynb) |
| Proposed Neural LSDE | $dz(t) = \gamma(\tilde{z}(t); \theta_\gamma) \, dt + \sigma(t; \theta_\sigma) \, dW_t$ | [example](https://github.com/yongkyung-oh/Stable-Neural-SDEs/blob/main/tutorial/simple%20OU%20process%20-%20Neural%20LSDE.ipynb) |
| Proposed Neural LSDE + KLD | $dz(t) = \gamma(\tilde{z}(t); \theta_\gamma) \, dt + \sigma(t; \theta_\sigma) \, dW_t$ | [example](https://github.com/yongkyung-oh/Stable-Neural-SDEs/blob/main/tutorial/simple%20OU%20process%20-%20Neural%20LSDE%20%28%2B%20KLD%29.ipynb) |
| Proposed Neural LNSDE (additive ablation) | $dz(t) = \gamma(t, \tilde{z}(t); \theta_\gamma) \, dt + \sigma(t; \theta_\sigma) \, dW_t$ | [example](https://github.com/yongkyung-oh/Stable-Neural-SDEs/blob/main/tutorial/simple%20OU%20process%20-%20Neural%20LNSDE%20%28additive%29.ipynb) |
| Proposed Neural LNSDE | $dz(t) = \gamma(t, \tilde{z}(t); \theta_\gamma) \, dt + \sigma(t; \theta_\sigma) \, z(t) \, dW_t$ | [example](https://github.com/yongkyung-oh/Stable-Neural-SDEs/blob/main/tutorial/simple%20OU%20process%20-%20Neural%20LNSDE.ipynb) |
| Proposed Neural GSDE | $\dfrac{dz(t)}{z(t)} = \gamma(t, \tilde{z}(t); \theta_\gamma) \, dt + \sigma(t; \theta_\sigma) \, dW_t$ | [example](https://github.com/yongkyung-oh/Stable-Neural-SDEs/blob/main/tutorial/simple%20OU%20process%20-%20Neural%20GSDE.ipynb) |

**Note**: The code is organized per experiment folder following the original reference implementations. For a cleaner interface, see the [torch-ists](https://github.com/yongkyung-oh/torch-ists) library.

---

**Robustness to Missingness Experiment**:

The experimental pipeline for this section has been moved to the independent library `torch-ists`. For reproducibility, use the updated version at the [torch-ists repository](https://github.com/yongkyung-oh/torch-ists).

---

## Reference
```bibtex
@inproceedings{oh_stable_2024,
	author       = {Oh, YongKyung and Lim, Dong-Young and Kim, Sungil},
	title        = {Stable {Neural} {Stochastic} {Differential} {Equations} in {Analyzing} {Irregular} {Time} {Series} {Data}},
	booktitle    = {The {Twelfth} {International} {Conference} on {Learning} {Representations}, {ICLR} 2024, {Vienna}, {Austria}, {May} 7-11, 2024},
	year         = 2024,
	url          = {https://openreview.net/forum?id=4VIgNuQ1pY}
}
```

```bibtex
@misc{oh_stable_2024_arxiv,
	author       = {Oh, YongKyung and Lim, Dong-Young and Kim, Sungil},
	title        = {Stable {Neural} {Stochastic} {Differential} {Equations} in {Analyzing} {Irregular} {Time} {Series} {Data}},
	publisher    = {arXiv},
	year         = 2024,
	doi          = {10.48550/arXiv.2402.14989},
	url          = {https://arxiv.org/abs/2402.14989}
}
```

Stable-Neural-SDEs

Stable-Neural-SDEs 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

Stable-Neural-SDEs

Stable-Neural-SDEs 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

Download README