KO | EN

GitLite

Minimalist Open Source Repository Search Engine

gitlite — search

#dsh #dsh-plugin #ai-agent #developer-tools #codex #typescript #cli #deepseek-harness #claude-code #ai-agents #mcp #rust

xithsense

★ 24 Open GitHub ↗

XIthSense fuses human cricket expertise with AI to predict, optimize, and explain winning fantasy cricket teams in real time.

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# XithSense

> An AI-powered fantasy cricket platform that turns 22,062 matches of ball-by-ball data into winning Dream11 teams.

---

## Demo

> *Team prediction screen with captain confidence scores and differential pick highlights.*

---

## Table of Contents

- [Features](#features)
- [Installation](#installation)
  - [Prerequisites](#prerequisites)
  - [Quick Start](#quick-start)
  - [Docker Setup](#docker-setup)
- [Usage](#usage)
  - [Minimal Example](#minimal-example)
  - [Environment Variables](#environment-variables)
  - [Common Commands](#common-commands)
- [API Reference](#api-reference)
- [Architecture](#architecture)
  - [Tech Stack](#tech-stack)
  - [System Overview](#system-overview)
  - [Data Schema](#data-schema)
- [Testing](#testing)
- [Contributing](#contributing)
- [License](#license)
- [Acknowledgments](#acknowledgments)
- [Support](#support)

---

## Features

- 🏏 **Ball-by-ball intelligence** — Ingests and indexes 22,062+ Cricsheet JSON matches (T20, ODI, Test, IPL, BBL, PSL, CPL and more) spanning 2001–2026, covering every delivery, wicket, extra, DRS review, and powerplay phase.
- 🤖 **Multi-model ensemble predictions** — Combines XGBoost, LightGBM, and CatBoost models (weighted 40 / 30 / 20 / 10 across ML, human rules, recent form, and live context) to forecast runs, wickets, economy, and fantasy points per player.
- 🧠 **Human intelligence rule engine** — Stores analyst-curated conditional rules (e.g. player vs. left-arm swing, chasing vs. setting, pressure-match temperament) with confidence scores and impact weights that override or adjust raw ML outputs.
- ⚡ **Smart team optimizer** — Generates safe, grand-league, aggressive, and small-league squads simultaneously using linear programming (PuLP) and genetic algorithms (DEAP), respecting Dream11 credit limits, role constraints, and ownership distribution targets.
- 💬 **LLM-powered explainability** — Every player selection ships with a plain-English rationale (recent form, venue average, matchup score, confidence percentage) generated by a Claude / GPT / Gemini integration so users understand *why*, not just *who*.
- 📡 **Live match intelligence** — WebSocket feed updates win probabilities, per-player fantasy point projections, and captain success likelihood in real time as the match progresses.
- 🔁 **Backtesting harness** — Replay predictions against 10,000+ historical matches and measure captain accuracy, correct-player rate, average fantasy points error, and simulated ROI without touching the live system.

---

## Installation

### Prerequisites

| Tool | Minimum version | Install guide |
|------|----------------|---------------|
| Python | 3.11 | [python.org](https://www.python.org/downloads/) |
| Docker & Docker Compose | 24.x / 2.x | [docs.docker.com](https://docs.docker.com/get-docker/) |
| Redis | 7.x | [redis.io](https://redis.io/docs/install/install-redis/) |
| PostgreSQL (via Supabase) | 15.x | [supabase.com](https://supabase.com/docs/guides/getting-started) |
| Qdrant | 1.9+ | [qdrant.tech](https://qdrant.tech/documentation/quick-start/) |
| Git | 2.40+ | [git-scm.com](https://git-scm.com/) |

### Quick Start

```bash
# 1. Clone the repository
git clone https://github.com/your-org/xithsense.git
cd xithsense

# 2. Create and activate a virtual environment
python -m venv .venv
source .venv/bin/activate          # Windows: .venv\Scripts\activate

# 3. Install dependencies
pip install -r requirements.txt

# 4. Copy and configure environment variables
cp .env.example .env
# Edit .env with your Supabase, Redis, Qdrant, and LLM credentials

# 5. Run database migrations
python scripts/db_migrate.py

# 6. Ingest the Cricsheet ball-by-ball dataset
#    Download all_json.zip from https://cricsheet.org/downloads/all_json.zip
#    and place it in data/raw/
python scripts/ingest_cricsheet.py --source data/raw/all_json.zip

# 7. Run feature engineering pipeline
python scripts/build_features.py

# 8. Train the ensemble models
python training/train_ensemble.py

# 9. Start the API server
uvicorn backend.main:app --host 0.0.0.0 --port 8000 --reload
```

The API will be available at `http://localhost:8000`.  
Interactive Swagger docs: `http://localhost:8000/docs`.

### Docker Setup

```bash
# Build and start all services (API, Redis, Qdrant, background workers)
docker compose up --build

# Run only the ingestion pipeline inside Docker
docker compose run --rm ingest python scripts/ingest_cricsheet.py \
  --source /data/raw/all_json.zip

# Stop all services
docker compose down
```

```yaml
# docker-compose.yml (abbreviated)
services:
  api:
    build: .
    ports: ["8000:8000"]
    env_file: .env
    depends_on: [redis, qdrant]

worker:
    build: .
    command: python -m celery -A backend.worker worker --loglevel=info

redis:
    image: redis:7-alpine
    ports: ["6379:6379"]

qdrant:
    image: qdrant/qdrant:v1.9.0
    ports: ["6333:6333"]
    volumes: ["qdrant_data:/qdrant/storage"]

volumes:
  qdrant_data:
```

---

## Usage

### Minimal Example

**Predict the optimal team for an upcoming match:**

```python
import httpx

client = httpx.Client(base_url="http://localhost:8000", headers={"X-API-Key": "your_api_key"})

# Request a fantasy team for a T20 match
response = client.post("/api/v1/predict/team", json={
    "match_id": "1535465",
    "format": "T20",
    "team_a": "Gujarat Titans",
    "team_b": "Royal Challengers Bengaluru",
    "venue": "Narendra Modi Stadium, Ahmedabad",
    "toss": {"winner": "Royal Challengers Bengaluru", "decision": "field"},
    "mode": "grand_league"       # "safe" | "grand_league" | "aggressive" | "small_league"
})

team = response.json()
print("Selected XI:", [p["name"] for p in team["players"]])
print("Captain:", team["captain"]["name"], f"({team['captain']['confidence']}% confidence)")
print("VC:", team["vice_captain"]["name"])
print("Differential:", team["differential"]["name"], "—", team["differential"]["reason"])
```

**Sample response:**

```json
{
  "players": [
    {"name": "V Kohli", "role": "BAT", "credits": 10.5, "fantasy_ceiling": 118},
    {"name": "Shubman Gill", "role": "BAT", "credits": 10.0, "fantasy_ceiling": 95}
  ],
  "captain": {
    "name": "V Kohli",
    "confidence": 87,
    "reason": "Chasing specialist, venue avg 72, last-5 form: Excellent"
  },
  "vice_captain": {"name": "JR Hazlewood", "confidence": 71},
  "differential": {
    "name": "Arshad Khan",
    "ownership_estimate": "4%",
    "reason": "Death-overs specialist, dry pitch, tail-end liability for opposition"
  },
  "ensemble_weights": {"ml": 0.40, "human_rules": 0.30, "form": 0.20, "live": 0.10}
}
```

**Ask the AI assistant a question:**

```python
response = client.post("/api/v1/chat", json={
    "match_id": "1535465",
    "message": "Why not pick Rohit Sharma as captain today?"
})
print(response.json()["answer"])
# → "Rohit averages 28 while setting at this venue vs his 54 chasing average.
#    Today RCB are fielding first, making him a setting captain — low ceiling pick."
```

### Environment Variables

| Variable | Required | Description | Example |
|---|---|---|---|
| `SUPABASE_URL` | ✅ | Supabase project API URL | `https://xxxx.supabase.co` |
| `SUPABASE_SERVICE_KEY` | ✅ | Supabase service role key (server-side only) | `eyJ...` |
| `DATABASE_URL` | ✅ | PostgreSQL connection string | `postgresql://user:pass@host:5432/db` |
| `REDIS_URL` | ✅ | Redis connection URL | `redis://localhost:6379/0` |
| `QDRANT_URL` | ✅ | Qdrant vector DB URL | `http://localhost:6333` |
| `QDRANT_API_KEY` | ⬜ | Qdrant cloud API key (omit for local) | `qd-...` |
| `LLM_PROVIDER` | ✅ | LLM backend: `anthropic`, `openai`, or `google` | `anthropic` |
| `ANTHROPIC_API_KEY` | ⬜ | Anthropic API key (if `LLM_PROVIDER=anthropic`) | `sk-ant-...` |
| `OPENAI_API_KEY` | ⬜ | OpenAI API key (if `LLM_PROVIDER=openai`) | `sk-...` |
| `GOOGLE_API_KEY` | ⬜ | Google Gemini key (if `LLM_PROVIDER=google`) | `AIza...` |
| `SECRET_KEY` | ✅ | JWT signing secret (min 32 chars) | `change-me-in-production` |
| `XITHSENSE_API_KEY` | ✅ | Internal API key for service-to-service calls | `xs-live-...` |
| `CRICSHEET_DATA_PATH` | ⬜ | Path to extracted Cricsheet JSON files | `data/raw/all_json/` |
| `MODEL_ARTIFACTS_PATH` | ⬜ | Directory for trained model files | `models/artifacts/` |
| `ENV` | ✅ | Runtime environment | `development` / `production` |

### Common Commands

| Command | Description |
|---|---|
| `uvicorn backend.main:app --reload` | Start API in development mode |
| `python scripts/ingest_cricsheet.py --source data/raw/all_json.zip` | Ingest full Cricsheet dataset |
| `python scripts/build_features.py --from 2020-01-01` | Re-run feature engineering from a date |
| `python training/train_ensemble.py --format T20` | Train models for a specific match format |
| `python backtesting/run_backtest.py --n 10000` | Backtest on last 10,000 matches |
| `python human_rules/validate_rules.py` | Validate all human intelligence rules |
| `celery -A backend.worker beat` | Start the Celery scheduler (live jobs) |
| `pytest tests/ -v --cov=backend` | Run full test suite with coverage |
| `docker compose logs -f api` | Tail API container logs |
| `alembic upgrade head` | Apply latest database migrations |

---

## API Reference

All endpoints are prefixed `/api/v1`. Authentication uses an `X-API-Key` header.

### Team & Captain

| Method | Endpoint | Description |
|--------|----------|-------------|
| `POST` | `/predict/team` | Generate one or more fantasy squads for a match |
| `POST` | `/predict/captain` | Rank captain and vice-captain options with confidence scores |
| `GET` | `/predict/differentials/{match_id}` | List low-ownership high-ceiling picks for grand leagues |

### Players

| Method | Endpoint | Description |
|--------|----------|-------------|
| `GET` | `/players/{player_id}` | Full career and recent-form stats for a player |
| `GET` | `/players/{player_id}/matchups` | Batter vs. bowler type breakdown (spin/pace/left-arm) |
| `GET` | `/players/search` | Search players by name, team, or role |

### Matches

| Method | Endpoint | Description |
|--------|----------|-------------|
| `GET` | `/matches/{match_id}` | Match metadata, lineups, toss, and venue details |
| `GET` | `/matches/{match_id}/live` | WebSocket-compatible live intelligence feed |
| `GET` | `/matches/upcoming` | List of upcoming matches with prediction-ready status |

### Insights & Chat

| Method | Endpoint | Description |
|--------|----------|-------------|
| `POST` | `/chat` | AI chat assistant — ask any fantasy cricket question in natural language |
| `GET` | `/explain/{match_id}/{player_id}` | Player selection rationale (rule triggers + ML scores) |

### Admin

| Method | Endpoint | Description |
|--------|----------|-------------|
| `POST` | `/admin/ingest` | Trigger a manual Cricsheet data ingestion job |
| `POST` | `/admin/retrain` | Queue a model retraining job |
| `GET` | `/admin/metrics` | Prediction accuracy KPIs and system health |

Full OpenAPI specification is auto-generated at `/docs` (Swagger UI) and `/redoc`.

---

## Architecture

### Tech Stack

| Layer | Technology |
|---|---|
| API | FastAPI 0.111, Python 3.11, Uvicorn, Gunicorn |
| Task queue | Celery + Redis Broker |
| Database | Supabase (PostgreSQL 15) |
| Cache | Redis 7 |
| Vector store | Qdrant 1.9 |
| ML models | XGBoost, LightGBM, CatBoost, scikit-learn |
| Optimization | PuLP (linear programming), DEAP (genetic algorithms) |
| LLM | Anthropic Claude / OpenAI GPT / Google Gemini (configurable) |
| Real-time | WebSockets (FastAPI native) |
| Data source | Cricsheet JSON v1.2.0 (22,062 matches, 2001–2026) |
| Frontend | React + Vite (served separately) |
| Infrastructure | Docker, Docker Compose, Railway / Render / AWS |

### System Overview

```
┌─────────────────────────────────────────────────────────────────┐
│                        XithSense Platform                       │
│                                                                 │
│  ┌─────────────┐   ┌──────────────────┐   ┌─────────────────┐  │
│  │  Cricsheet  │──▶│  ETL Pipeline    │──▶│  Supabase (PG)  │  │
│  │  22k+ JSON  │   │  Feature Eng.    │   │  Redis Cache    │  │
│  └─────────────┘   └──────────────────┘   └────────┬────────┘  │
│                                                     │           │
│  ┌─────────────────────────────────────────────────▼────────┐  │
│  │                   Ensemble Engine                        │  │
│  │   XGBoost (40%) + Human Rules (30%) + Form (20%)        │  │
│  │   + Live Context (10%)  →  Player Score Array           │  │
│  └─────────────────────────────┬────────────────────────────┘  │
│                                │                               │
│  ┌─────────────────────────────▼────────────────────────────┐  │
│  │          Team Optimizer (PuLP / DEAP)                   │  │
│  │   Safe | Grand League | Aggressive | Small League        │  │
│  └─────────────────────────────┬────────────────────────────┘  │
│                                │                               │
│  ┌─────────────────────────────▼────────────────────────────┐  │
│  │          Explainability Engine (LLM)                    │  │
│  │   Claude / GPT / Gemini  →  Plain-English rationale     │  │
│  └─────────────────────────────┬────────────────────────────┘  │
│                                │                               │
│              FastAPI REST + WebSocket API                      │
│                    │                │                          │
│             React Frontend     Notification                    │
│           (Web / Mobile)      (Telegram / WA / Push)          │
└─────────────────────────────────────────────────────────────────┘
```

### Data Schema

Each Cricsheet JSON file (`<match_id>.json`) follows Cricsheet format v1.2.0:

```json
{
  "meta": { "data_version": "1.2.0", "created": "2026-05-31", "revision": 1 },
  "info": {
    "match_type": "T20",
    "teams": ["Gujarat Titans", "Royal Challengers Bengaluru"],
    "venue": "Narendra Modi Stadium, Ahmedabad",
    "toss": { "winner": "Royal Challengers Bengaluru", "decision": "field" },
    "season": "2026",
    "gender": "male",
    "team_type": "club",
    "event": { "name": "Indian Premier League", "stage": "Final" },
    "player_of_match": ["V Kohli"],
    "outcome": { "winner": "Royal Challengers Bengaluru", "by": { "wickets": 5 } }
  },
  "innings": [
    {
      "team": "Gujarat Titans",
      "powerplays": [{ "from": 0.1, "to": 5.6, "type": "mandatory" }],
      "overs": [
        {
          "over": 0,
          "deliveries": [
            {
              "actual_delivery": "0.1",
              "batter": "Shubman Gill",
              "bowler": "JR Hazlewood",
              "non_striker": "B Sai Sudharsan",
              "runs": { "batter": 0, "extras": 0, "total": 0 },
              "wickets": [
                { "player_out": "Shubman Gill", "kind": "caught",
                  "fielders": [{ "name": "RM Patidar" }] }
              ]
            }
          ]
        }
      ]
    }
  ]
}
```

The ingestion pipeline parses every delivery into relational tables (players, deliveries, innings, matches, venues) and pre-computes rolling features (last-3, last-5, last-10 fantasy points, venue averages, batter-vs-bowler-type records) stored in Supabase.

---

## Testing

```bash
# Run the full test suite
pytest tests/ -v

# Run with coverage report
pytest tests/ --cov=backend --cov=training --cov=optimizer --cov-report=term-missing

# Run only unit tests (fast, no DB)
pytest tests/unit/ -v

# Run integration tests (requires a running Supabase and Redis)
pytest tests/integration/ -v --tb=short

# Run backtesting regression checks
pytest tests/backtesting/ -v -k "accuracy"
```

**Coverage targets:**

| Module | Target |
|--------|--------|
| `backend/` (API routes, services) | ≥ 80% |
| `training/` (feature pipelines, model wrappers) | ≥ 75% |
| `optimizer/` (team generation) | ≥ 85% |
| `human_rules/` (rule engine) | ≥ 90% |

Tests use `pytest`, `httpx` (for async API testing), and `pytest-mock` for external service mocks. Integration tests spin up a dedicated test Supabase schema via `conftest.py` fixtures and tear it down after each session.

---

## Contributing

We welcome pull requests. Follow this workflow:

```bash
# 1. Fork the repository and clone your fork
git clone https://github.com/<your-username>/xithsense.git
cd xithsense

# 2. Create a feature branch off main
git checkout -b feat/your-feature-name

# 3. Install dev dependencies
pip install -r requirements-dev.txt
pre-commit install      # sets up ruff, black, isort hooks

# 4. Make your changes and write tests
# 5. Run the full test suite before pushing
pytest tests/ -v

# 6. Push your branch and open a pull request against main
git push origin feat/your-feature-name
```

**Guidelines:**

- Follow [PEP 8](https://peps.python.org/pep-0008/). `ruff` and `black` enforce this automatically via pre-commit.
- Every new feature must include tests. Bug fixes must include a regression test.
- Keep PRs focused. One logical change per PR.
- Add or update docstrings for any public function or class you touch.
- For human intelligence rules (`human_rules/`), include the source and a confidence estimate in the rule JSON.

Open an issue before starting large refactors or new subsystems so we can align on design.

---

## License

Distributed under the [MIT License](LICENSE).  
Ball-by-ball match data is sourced from [Cricsheet](https://cricsheet.org/) and is subject to Cricsheet's own [data licence](https://cricsheet.org/licence/).

---

## Acknowledgments

- **[Cricsheet](https://cricsheet.org/)** — The open ball-by-ball dataset that powers XithSense. 22,062 matches, meticulously compiled and maintained.
- **[FastAPI](https://fastapi.tiangolo.com/)** — Async Python API framework.
- **[Supabase](https://supabase.com/)** — Postgres hosting with real-time and auth built in.
- **[PuLP](https://coin-or.github.io/pulp/)** and **[DEAP](https://deap.readthedocs.io/)** — Linear programming and evolutionary algorithms for team optimization.
- **[XGBoost](https://xgboost.ai/)**, **[LightGBM](https://lightgbm.readthedocs.io/)**, **[CatBoost](https://catboost.ai/)** — The gradient-boosting trio behind the ensemble.
- **[Qdrant](https://qdrant.tech/)** — Vector similarity store for player embeddings and rule retrieval.

---

## Support

| Channel | Link |
|---------|------|
| 🐛 Bug reports | [GitHub Issues](https://github.com/your-org/xithsense/issues) |
| 💡 Feature requests | [GitHub Discussions](https://github.com/your-org/xithsense/discussions) |
| 📧 Email | support@xithsense.com |
| 💬 Telegram community | [@xithsense](https://t.me/xithsense) |

---

xithsense

Explore Similar Repositories

Was this content helpful?

Recent Feedback

Download README