STCast

KO | EN

gitlite — search

#javascript #react #typescript #go #android #machine-learning #deep-learning #java #windows #kubernetes #vue #awesome-list #mcp #openai

(★ 26)

The official repo for STCast (CVPR2026 Highlight).

STCast 최신버젼 다운로드

최종 버전 다운로드 (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# STCAST(Highlight paper in CVPR2026)

![HKUST](https://raw.githubusercontent.com/chenhao-zju/STCast/main/assets/HKUST-logo.png)

STCAST \[[paper](https://arxiv.org/abs/2509.25210)\] is a weather forecasting repository with two runnable pipelines:

- Global weather forecasting (ERA5-style global grid)
- Regional weather forecasting (high-resolution regional refinement with global guidance)

The current repository is streamlined for training, testing, and logging workflows.

## 1. What This Repository Supports

- End-to-end training and evaluation for global forecasting
- End-to-end training and evaluation for regional forecasting
- YAML-driven experiment configuration
- DDP multi-GPU training scripts
- Checkpointing and log output under `./logs/`

## 2. Repository Layout

## 3. Environment Setup

### 3.1 Python and CUDA

- Python: 3.8+
- OS: Linux recommended for multi-GPU training
- GPU: CUDA-capable GPU(s)

### 3.2 Core Dependencies

Install the commonly used dependencies first:

```bash
pip install numpy scipy einops ruamel.yaml boto3 botocore xarray tqdm pillow wandb timm mmengine
```

Install PyTorch matching your CUDA version (example shown for CUDA 12.1):

```bash
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121
```

### 3.3 Extra Dependencies Required by Imported Modules

Some models are imported at runtime by default, so these packages may also be required:

```bash
pip install dgl torch-scatter torch-geometric importlib_metadata
```

Optional but commonly needed in this codebase:

- `apex` (for `FusedAdam`)
- `flash-attn`
- `petrel_client` (if you use `s3://` style data path through Ceph/Petrel)

## 4. Data Preparation

### 4.1 Required Statistics Files

These files must exist and be valid:

- `./statistic/mean_std.json`
- `./statistic/mean_std_single.json`

### 4.2 Dataset Organization (From Data Loaders)

This section is aligned with:

- `utils/data_loader_npyfiles_global.py`
- `utils/data_loader_npyfiles_region.py`

Both loaders read data by timestamp, with fixed filename format:

```text
{HH}:00:00-{variable}.npy
{HH}:00:00-{variable}-{pressure_level}.npy
```

Time path format is:

```text
{YYYY}/{YYYY-MM-DD}/
```

Example timestamp prefix:

```text
2021/2021-01-09/06:00:00-
```

### 4.3 Global Dataset Layout (`data_loader_npyfiles_global.py`)

For `root_dir` in `config/STCAST.yaml`, the loader expects:

- Pressure-level variables:
  - `{root_dir}/{YYYY}/{YYYY-MM-DD}/{HH}:00:00-{var}-{level}.npy`
- Surface variables:
  - `{root_dir}/single/{YYYY}/{YYYY-MM-DD}/{HH}:00:00-{var}.npy`

Typical variable names used by default:

- Atmospheric: `z`, `q`, `u`, `v`, `t` with pressure levels
- Surface: `t2m`, `u10`, `v10`, `msl`

Default pressure levels used by the global loader:

```text
1000.0, 925.0, 850.0, 700.0, 600.0, 500.0, 400.0, 300.0, 250.0, 200.0, 150.0, 100.0, 50.0
```

Global directory example:

### 4.4 Regional Dataset Layout (`data_loader_npyfiles_region.py`)

Regional loader has two streams:

- Global/coarse stream (pressure + surface), read from `root_dir` and `root_dir/single`
- Regional/high-resolution stream, read by S3 client from fixed prefix logic in loader code

For the global/coarse stream in regional loader, the expected key layout is still:

- `{root_dir}/{YYYY}/{YYYY-MM-DD}/{HH}:00:00-{var}-{level}.npy`
- `{root_dir}/single/{YYYY}/{YYYY-MM-DD}/{HH}:00:00-{var}.npy`

Regional/high-resolution variables used by default:

- `t2m`, `u10`, `v10`, `msl`, `d2m`, `tp6h`

Regional/high-resolution key example (object storage):

```text
<bucket>/nwp_initial_fileds/analysis_MIR/np2001x4000/single/2024/2024-06-01/00:00:00-t2m.npy
<bucket>/nwp_initial_fileds/analysis_MIR/np2001x4000/single/2024/2024-06-01/00:00:00-u10.npy
...
```

Important notes from regional loader implementation:

- It uses S3 API reads via `utils/s3_client.py`.
- It expects bucket/prefix-style paths (for example `ai4earth/nwp_initial_fileds/...`).
- It performs internal resizing for some 2024/2025 inputs when path contains `np721x1440`.

### 4.5 Climate and Auxiliary Directory Convention

Both loaders define climate directories under `root_dir`:

- `climate_mean_day_128x256/1993-2016/` for `grid_resolution=1.4`
- `climate_mean_day/1993-2016/` for `grid_resolution=0.25`
- surface climate under corresponding `single/...` path

If your workflow does not use climate tensors directly in current train loop, you can still keep this layout for compatibility.

### 4.6 Regional Data Notes

Regional pipeline (`train_regional.py` / `test_regional.py`) uses:

- Global context from configured `root_dir`
- Regional high-resolution fields read through S3 client logic in `utils/s3_client.py`

Set these environment variables before regional runs:

```bash
export STCAST_S3_ACCESS_KEY="your_access_key"
export STCAST_S3_SECRET_KEY="your_secret_key"
export STCAST_S3_ENDPOINT="https://your-s3-endpoint"
```

### 4.7 Climate File for Global Test

Global evaluation code loads:

- `./climate_1.4_new.npy`

Place this file at repository root before running `test.py` / `test.sh`.

## 5. Configuration

### 5.1 Global

- File: `./config/STCAST.yaml`
- Entry used by scripts: `afno_backbone`

### 5.2 Regional

- File: `./config/STCAST_region.yaml`
- Entry used by scripts: `afno_backbone`

### 5.3 Key Fields You Will Usually Modify

- `root_dir`
- `train_period`, `valid_period`, `test_period`
- `batch_size`
- `grid_resolution`
- `feature_dims`, `in_channels`, `out_channels`
- `max_epochs`, `lr`, `scheduler`

## 6. Global Forecasting Workflow

### 6.1 Train

Script entry:

```bash
bash train_global.sh
```

Manual equivalent:

```bash
CUDA_VISIBLE_DEVICES=0,1,2,3 python -m torch.distributed.run --nproc_per_node=4 train_global.py \
  --enable_amp \
  --yaml_config=./config/STCAST.yaml \
  --config=afno_backbone \
  --run_num=1 \
  --exp_dir=./logs/your_global_exp/ \
  --checkpoint=''
```

### 6.2 Test

Script entry:

```bash
bash test.sh
```

Manual equivalent:

```bash
CUDA_VISIBLE_DEVICES=0 python test.py \
  --yaml_config=./config/STCAST.yaml \
  --config=afno_backbone \
  --run_num=1 \
  --override_dir=./logs/your_global_eval/ \
  --weights=./logs/your_global_exp/afno_backbone/1/training_checkpoints/ckpt.tar
```

## 7. Regional Forecasting Workflow

### 7.1 Important Prerequisite

Regional scripts depend on a global teacher checkpoint path hardcoded in:

- `train_regional.py`
- `test_regional.py`

Before running regional tasks, update `checkpoint_file` in those files to a valid global checkpoint in your environment.

### 7.2 Train

Script entry:

```bash
bash train_regional.sh
```

### 7.3 Test

Script entry:

```bash
bash test_regional.sh
```

Manual equivalent:

```bash
CUDA_VISIBLE_DEVICES=0 python test_regional.py \
  --yaml_config=./config/STCAST_region.yaml \
  --config=afno_backbone \
  --run_num=1 \
  --override_dir=./logs/your_regional_eval/ \
  --weights=./logs/your_regional_exp/afno_backbone/1/training_checkpoints/ckpt.tar
```

## 8. Outputs and Logs

By default, each experiment writes to:

- `./logs/<exp_name>/<config>/<run_num>/`

Common artifacts:

- `out.log` (training log)
- `training_checkpoints/ckpt.tar`
- `training_checkpoints/best_ckpt.tar`
- `hyperparams.yaml`

Test outputs and metric arrays are written to `--override_dir` or experiment directory, depending on script arguments.

## 9. Reproducibility Tips

- Keep `config/*.yaml` and shell scripts versioned together
- Fix random seeds in training code if strict reproducibility is required
- Record CUDA, PyTorch, and dependency versions in each run log
- Use explicit checkpoint paths when comparing experiments

## 10. Troubleshooting

- `ModuleNotFoundError` for graph packages:
  - Install `dgl`, `torch-scatter`, and `torch-geometric`.
- S3 credential errors:
  - Confirm `STCAST_S3_ACCESS_KEY`, `STCAST_S3_SECRET_KEY`, `STCAST_S3_ENDPOINT`.
- Missing climate file:
  - Add `climate_1.4_new.npy` to repository root for global test.
- DDP launch issues:
  - Verify `CUDA_VISIBLE_DEVICES`, `--nproc_per_node`, and NCCL environment compatibility.

## 11. License

This repository is distributed under the BSD 3-Clause License.
See `LICENSE` for details.

## References:

If you find this work useful, cite it using:
```
@inproceedings{stcast,
  title={STCast: Adaptive Boundary Alignment for Global and Regional Weather Forecasting},
  author={Chen, Hao and Han, Tao and Zhang, Jie and Guo, Song and Bai, Lei},
  booktitle={Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (CVPR)},
  year={2026}
}
```

STCast

STCast 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

STCast

STCast 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

Download README