KO | EN

GitLite

Minimalist Open Source Repository Search Engine

gitlite — search

#typescript #ai-agents #cli #claude-code #codex #developer-tools #dsh-plugin #ai-agent #mcp #deepseek-harness #windows #open-source

Higgsfield-DS

★ 2 Open GitHub ↗

HackNU 2026 Higgsfield DS Challenge — churn prediction that separates voluntary from involuntary churn. 120+ engineered features, stacking, calibration; 0.573 macro F1.

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# HackNU 2026 — The Retention Architect

**Higgsfield DS Challenge** — predictive churn system that distinguishes **Voluntary Churn** (user chooses to cancel) from **Involuntary Churn** (payment failures).

## Final Results

| Metric | Score |
|---|---|
| Macro F1 (OOF) | **0.5734** |
| Macro PR-AUC | **0.6269** |
| Accuracy | ~0.60 |

3-class classification: `not_churned` (50%) / `vol_churn` (25%) / `invol_churn` (25%)

---

## Approach

### Pipeline Overview
```
Data → Feature Engineering → Models → Stacking → Calibration → Threshold Tuning → Submission
```

### Feature Engineering (120+ features across 6 tables)

**Properties** — subscription start date, country, temporal features (year 1067 synthetic dates parsed as strings)

**Purchases** — total spend, purchase types, purchase velocity

**Transactions** — dual approach for failed tx (no `user_id`):
- **Approach 1:** Fingerprint matching (card_brand + bank + country + funding) between failed and successful transactions, weighted 1/N for multi-user fingerprints
- **Approach 2:** Card risk metadata from successful transactions (is_prepaid, cvc_check, country_mismatch)

**Generations** (5.5GB, read in chunks) — temporal patterns per user:
- `day01_ratio`, `burst_flag` — captures "binge and leave" pattern for vol_churn
- 14 × `active_day_X` binary flags (daily retention)
- `first_quit_day`, `max_active_streak`
- Week1 vs Week2 trends (`gen_trend`, `credits_trend`)
- Recency features (`days_since_last_gen`)

**Quizzes** — onboarding signals (source, frustration, experience)

**Class-specific interactions:**
- `prepaid_x_failure`, `payment_risk_score` → invol_churn
- `burst_x_short_life`, `abrupt_decline` → vol_churn
- `sustained_engagement`, `exploration_score` → engagement

### Models (5-fold Stratified CV)

1. **CatBoost GPU** — native categorical handling, `TotalF1:Macro` eval metric
2. **LightGBM CPU** — with target encoding for high-cardinality `country_code`
3. **XGBoost GPU** — sample weighting for class balance
4. **Hierarchical 2-stage model:**
   - Stage 1: churned vs not_churned (binary)
   - Stage 2: vol vs invol (binary, trained on churned subset)
   - Combined: `P(vol) = P(churned) × P(vol|churned)`

### Ensemble
- **Isotonic calibration** per model before stacking
- **Weighted average search** across 4 models
- **Per-class threshold tuning** on OOF predictions (step=0.01)

---

## Key Insights (from EDA + SHAP)

### Voluntary Churn drivers
- **Burst pattern:** vol_churn users generate heavily on day 0-1, then quit (median lifespan: 1.1 days!)
- **High engagement → sudden drop:** `gen_trend` strongly negative
- **Japanese users:** 60% vol_churn rate (cultural signal?)
- **Explore feature, leave unsatisfied**

### Involuntary Churn drivers
- **Prepaid cards:** 2x higher invol rate (10.3% vs 4.9%)
- **Indian users:** 46% invol rate (payment infrastructure issues)
- **Turkey/Brazil:** 32%+ invol rate
- **card_declined:** 97% of all failed transactions
- **CVC not_provided** → risky card signal

### Churn-agnostic observations
- **Subscription plan = useless** (exactly 50/25/25 across all 4 plans)
- **MI max = 0.035** → weak individual signals, model finds patterns in combinations

---

## Business Strategy

### For Voluntary Churn (active users leaving)
1. **Day 2 re-engagement trigger** — push notification/email if `day01_ratio > 0.8` and silent on day 2
2. **Personal discounts** when `gen_trend < -0.3` (usage declining)
3. **Feature education** for users with `frustration = limited/high-cost`
4. **Premium content** for power users (`gen_total > median`)
5. **Loyalty rewards** on day 7 milestone

### For Involuntary Churn (payment failures)
1. **Smart retry logic** for `card_declined` (97% of failures)
2. **Pre-dunning notifications** before card expiry
3. **Prepaid card warning** — suggest credit alternatives at signup
4. **Grace period** (3-7 days) before deactivation on failure
5. **Country-specific:** Alternative payment methods for India/Turkey/Brazil
6. **Digital wallet promotion** to reduce decline rate

---

## Explainability

**SHAP TreeExplainer** on CatBoost models (averaged across 5 folds).

- **Global importance:** Top features per class (`shap_global.png`)
- **Per-user explanations:** Top-5 SHAP values for each prediction
- **Class-specific SHAP profiles:** Different drivers for vol vs invol

Per-user probability scores available in `submission_detailed.csv`.

---

## Repository Structure

```
.
├── hacknu_pipeline_kaggle.ipynb    # Main pipeline notebook
├── submission.csv                  # Test predictions (user_id, churn_status)
├── submission_detailed.csv         # Predictions + probabilities + confidence
├── shap_global.png                 # SHAP global importance by class
├── business_insights.png           # Behavioral analysis visualizations
└── README.md
```

---

## How to Run

1. Upload dataset to Kaggle (or Google Drive for Colab)
2. Update `DATA_ROOT` path in cell 3
3. Select GPU accelerator (T4 / H100)
4. Run all cells

**Runtime:** ~15-20 min on H100 / ~30-40 min on T4

**Dependencies:**
```
catboost, lightgbm, xgboost, optuna, shap, scikit-learn, pandas, numpy
```

---

## Tech Stack

- **ML:** CatBoost, LightGBM, XGBoost, Scikit-learn
- **Explainability:** SHAP
- **Infrastructure:** Kaggle Notebooks (H100 GPU)
- **Languages:** Python 3.10+

---

## Methodology Summary

A 3-class classifier combining gradient boosting models (CatBoost/LightGBM/XGBoost) with a hierarchical 2-stage binary classifier. Features engineered across 6 data tables with emphasis on temporal patterns (burst detection, daily retention) and payment risk signals. Isotonic calibration and per-class threshold tuning applied for final predictions. SHAP-based explainability provides per-user churn reason attribution, enabling targeted retention strategies distinguishing voluntary (engagement-based) from involuntary (payment-based) churn.

Higgsfield-DS

Explore Similar Repositories

Was this content helpful?

Recent Feedback

Download README