preprompt

KO | EN

gitlite — search

#typescript #ai-agents #deepseek-harness #dsh-plugin #open-source #ai #claude-code #cli #windows #codex #developer-tools #dsh

★ 37 Open GitHub ↗

[KDD-2026, arXiv-2025.5.13] Official implementation of "PrePrompt: Predictive Prompting for Class-Incremental Learning"

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# 🌟 PrePrompt: Predictive Prompting for Class-Incremental Learning [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT) [![Framework](https://img.shields.io/badge/Framework-PyTorch-red)](https://pytorch.org) [![Python 3.8](https://img.shields.io/badge/Python-3.8-blue.svg)](https://www.python.org/)

[中文](README_zh.md) | **English**

<img src="docs/static/images/method.png" alt="PrePrompt Framework" width="52%" style="margin-right:15px;"/>
 <img src="docs/static/images/feature.png" alt="Feature Txtrapolation" width="45%"/>


 Figure 1: PrePrompt two-stage framework (left) and feature extrapolation mechanism (right).

🔥 Official PyTorch implementation of **PrePrompt**, a two-stage predictive prompting framework that enables pre-trained models to first predict task-specific prompts and then perform label prediction, effectively balancing stability and plasticity in class-incremental learning.

---

## 🧠 Overview

 <img src="docs/static/images/prompts.png" alt="PrePrompt Overview" width="70%"/>


 Figure 2: Main difference between conventional prompt-based CIL methods and PrePrompt.

**PrePrompt** introduces a *predictive prompting mechanism* that leverages pre-trained models' natural classification ability to predict task-specific prompts.

Unlike conventional prompt-based CIL methods that rely on correlation-based strategies, where an image's classification feature is used as a query to retrieve the most related key prompts and select the corresponding value prompts for training, PrePrompt circumvents the correlation-based limitations that fitting the entire feature space of all tasks with only a few trainable prompts - ensuring **robust knowledge retention**, **minimal forgetting**, and **efficient adaptation**.

## 🚀 Key Highlights

- 🧩 **Predictive Prompting**: Learns to anticipate task evolution, improving long-term adaptability.
- 📈 **State-of-the-Art Results**: Outperforms all prior prompt-based CIL methods across multiple benchmarks.
- ⚡ **Lightweight Integration**: Minimal computation overhead — plug-and-play for any ViT-based model.
- 🔁 **Stable & Scalable**: Balances plasticity (learning new tasks) and stability (preserving old knowledge).

## 📊 Benchmark Results
10 tasks with equal number of classes  of CIFAR-100, ImageNet-R, CUB-200 while 5 tasks of 5-Datasets:

| Dataset | Final Accuracy (%) ↑ | Average Incremental Accuracy (%) ↑ | Forgetting Rate (%) ↓|
|---------|-------------------|----------------------------------|---------------------|
| CIFAR-100 | 93.74 | 95.41 | 1.27 |
| ImageNet-R | 75.09 | 78.96 | 1.11 |
| CUB-200 | 88.27 | 88.29 | 1.81 |
| 5-Datasets | 94.54 | 95.78 | 0.21 |

## 🛠️ Installation

### Environment Setup
```bash
# Create and activate conda environment
conda create -n preprompt python=3.8 -y
conda activate preprompt

# Install dependencies (retry if network issues occur)
pip install -r requirements.txt
```

### Dependencies
``` text
timm==0.6.7
pillow==9.2.0
matplotlib==3.5.3
torchprofile==0.0.4
torch==1.13.1
torchvision==0.14.1
urllib3==2.0.3
scipy==1.7.3
scikit-learn==1.0.2
numpy==1.21.6
```

## 📁 Datasets
PrePrompt automatically handles downloading and preprocessing for the following datasets:
- 🖼️ [CIFAR-100](https://www.cs.toronto.edu/~kriz/cifar-100-python.tar.gz) — 100-class object recognition
- 🎨 [ImageNet-R](https://people.eecs.berkeley.edu/~hendrycks/imagenet-r.tar) — artistic renditions of ImageNetrenditions
- 🐦 [CUB-200](https://data.caltech.edu/records/65de6-vp158/files/CUB_200_2011.tgz) — fine-grained bird classification
- 🔢 5-Datasets — composite of SVHN, MNIST, CIFAR-10, notMNIST, and Fashion-MNIST

💡 Tip: If your network is unstable, pre-download datasets into `./datasets/`.

## 🎯 Quick Start
Run the corresponding training scripts for each benchmark:
```bash
# CIFAR-100 Experiments
bash training_scripts/train_cifar100_vit.sh

# ImageNet-R Experiments  
bash training_scripts/train_imr_vit.sh

# CUB-200 Fine-grained Classification
bash training_scripts/train_cub_vit.sh

# 5-Datasets Sequential Learning
bash training_scripts/train_5datasets_vit.sh
```
Logs and checkpoints will be stored in `./outputs/`.

## 🙏 Acknowledgments
This repository builds upon the following excellent open-source projects:
- [DualPrompt](https://github.com/JH-LEE-KR/dualprompt-pytorch) — continual prompting foundations.
- [HiDe-Prompt](https://github.com/thu-ml/HiDe-Prompt) — hierarchical prompt architecture

## 📜 License
This project is released under the MIT License. See the [LICENSE](./LICENSE) file for details.

preprompt

Explore Similar Repositories

Was this content helpful?

Recent Feedback

Download README