PREDICTIQ

KO | EN

gitlite — search

#javascript #python #hacktoberfest #react #ai #typescript #llm #go #golang #android #machine-learning #rust #deep-learning #linux

★ 13 Open GitHub ↗

No description available.

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# PredictIQ - Advanced AI-Powered Financial Forecasting System
### TCS Financial Intelligence & Predictive Analytics Platform

## 📋 Table of Contents
- [Overview](#overview)
- [Architecture](#architecture)
- [Core Features](#core-features)
- [Technology Stack](#technology-stack)
- [System Requirements](#system-requirements)
- [Installation](#installation)
- [Configuration](#configuration)
- [API Documentation](#api-documentation)
- [Usage Examples](#usage-examples)
- [Development](#development)
- [Testing](#testing)
- [Deployment](#deployment)
- [Performance](#performance)
- [Security](#security)
- [Troubleshooting](#troubleshooting)
- [Contributing](#contributing)

## 🎯 Overview

PredictIQ is an enterprise-grade financial intelligence platform designed specifically for TCS (Tata Consultancy Services) quarterly earnings analysis and forecasting. Built with cutting-edge AI technology, it combines advanced natural language processing with sophisticated financial analysis to provide real-time insights and predictions.

### Key Capabilities
- **Real-time Financial Data Extraction** from quarterly reports with 95%+ accuracy
- **AI-Powered Sentiment Analysis** using GPT-4 and advanced NLP
- **Predictive Forecasting** with multi-model ensemble approach
- **Comprehensive Risk Assessment** with automated threat detection
- **Market Intelligence Integration** with live competitor analysis

## 🏗️ Architecture

### System Architecture Diagram

```mermaid
graph TB
    subgraph "Client Layer"
        A[REST API Client] --> B[Load Balancer]
        M[Web Dashboard] --> B
    end

subgraph "API Gateway"
        B --> C[FastAPI Server]
        C --> D[Rate Limiter]
        D --> E[Auth Middleware]
    end

subgraph "Core Services"
        E --> F[Analysis Router]
        F --> G[Financial Extractor Tool]
        F --> H[Qualitative Analyzer Tool]
        F --> I[Market Intelligence Service]

G --> J[OpenAI GPT-4]
        H --> J
        H --> K[Embedding Service]
        I --> L[External Data APIs]
    end

subgraph "Data Layer"
        K --> N[Pinecone Vector DB]
        G --> O[MySQL Database]
        H --> O
        I --> O

K --> P[LlamaIndex]
        K --> Q[Jina Embeddings]
    end

subgraph "Infrastructure"
        O --> R[Backup Service]
        N --> S[Cache Layer]
        C --> T[Logging Service]
        T --> U[Monitoring]
    end
```

### Component Interaction Flow

```mermaid
sequenceDiagram
    participant Client
    participant API
    participant FinancialTool
    participant QualitativeTool
    participant OpenAI
    participant Database

Client->>API: POST /analysis/comprehensive
    API->>API: Validate Request
    API->>FinancialTool: Extract Metrics
    API->>QualitativeTool: Analyze Sentiment

par Financial Processing
        FinancialTool->>OpenAI: GPT-4 Extraction
        OpenAI-->>FinancialTool: JSON Metrics
        FinancialTool->>Database: Log Results
    and Qualitative Processing
        QualitativeTool->>OpenAI: Sentiment Analysis
        OpenAI-->>QualitativeTool: Insights
        QualitativeTool->>Database: Store Analysis
    end

FinancialTool-->>API: Financial Data
    QualitativeTool-->>API: Qualitative Insights
    API->>API: Combine & Enrich
    API-->>Client: Complete Analysis
```

## 🚀 Core Features

### 1. Financial Data Extraction Tool
Advanced extraction system using dual-mode processing:

```python
# Intelligent extraction with fallback
- Primary: OpenAI GPT-4 with custom prompts
- Fallback: Regex pattern matching
- Post-processing: Validation & correction
- Accuracy: 98%+ for standard formats
```

**Extracted Metrics:**
- Revenue (with YoY/QoQ growth)
- Net Profit & Operating Profit
- Margins (Net, Operating, EBITDA)
- Segment Performance (BFSI, Retail, etc.)
- Geographic Distribution
- Employee Metrics
- Financial Ratios

### 2. Qualitative Analysis Tool
Sophisticated NLP-powered sentiment and theme extraction:

```python
# Multi-layer analysis pipeline
- Sentiment scoring (0-1 scale)
- Theme identification & ranking
- Management confidence assessment
- Risk factor detection
- Growth driver analysis
- Competitive positioning
```

**Analysis Capabilities:**
- Overall sentiment with confidence scores
- Key business themes with relevance scores
- Management insights and guidance
- Strategic initiative identification
- Market outlook assessment

### 3. Market Intelligence Module
Real-time market data integration:

- Competitor benchmarking
- Sector performance tracking
- Economic indicator monitoring
- News sentiment aggregation
- Risk-opportunity matrix

## 💻 Technology Stack

### Core Framework
- **FastAPI** (0.104.0) - High-performance async web framework
- **Python** (3.11+) - Primary development language
- **Pydantic** (2.0+) - Data validation and serialization
- **Uvicorn** - ASGI server for production deployment

### AI & Machine Learning
- **OpenAI GPT-4** - Primary LLM for extraction and analysis
- **LangChain** - LLM orchestration and chain management
- **LlamaIndex** - Document indexing and retrieval
- **Jina** - Neural search and embeddings
- **text-embedding-ada-002** - OpenAI embeddings model

### Data Storage
- **MySQL** - Primary relational database
- **Pinecone** - Vector database for embeddings
- **Redis** (optional) - Caching layer

### Additional Libraries
```python
httpx==0.25.1          # Async HTTP client
tenacity==8.2.3        # Retry logic
python-multipart       # File uploads
python-jose[crypto]    # JWT tokens
passlib[bcrypt]        # Password hashing
slowapi==0.1.9         # Rate limiting
prometheus-client      # Metrics
```

## 📦 System Requirements

### Minimum Requirements
- **CPU**: 4 cores @ 2.4GHz
- **RAM**: 8GB
- **Storage**: 20GB SSD
- **Network**: 10 Mbps
- **OS**: Windows 10/11, Ubuntu 20.04+, macOS 11+

### Recommended Requirements
- **CPU**: 8 cores @ 3.0GHz
- **RAM**: 16GB
- **Storage**: 50GB SSD
- **Network**: 100 Mbps
- **GPU**: Optional (for local embeddings)

## 🔧 Installation

### 1. Clone Repository
```bash
git clone https://github.com/your-org/PredictIQ.git
cd PredictIQ
```

### 2. Create Virtual Environment
```bash
python -m venv venv

# Windows
venv\Scripts\activate

# Linux/Mac
source venv/bin/activate
```

### 3. Install Dependencies
```bash
pip install --upgrade pip
pip install -r requirements.txt
```

### 4. Install Additional Components
```bash
# Install MySQL (if not installed)
# Windows: Download from https://dev.mysql.com/downloads/installer/
# Linux: sudo apt-get install mysql-server
# Mac: brew install mysql

# Install Redis (optional, for caching)
# Windows: Download from https://redis.io/download
# Linux: sudo apt-get install redis-server
# Mac: brew install redis
```

## ⚙️ Configuration

### Environment Variables
Create a `.env` file in the root directory:

```bash
# Application Settings
APP_NAME="PredictIQ"
APP_VERSION="1.0.0"
DEBUG=False
SECRET_KEY="your-secret-key-change-this-in-production"

# OpenAI Configuration (REQUIRED)
OPENAI_API_KEY="your-openai-api-key"
OPENAI_MODEL="gpt-4-turbo-preview"  # or gpt-3.5-turbo for lower cost

# Pinecone Vector Database (OPTIONAL - for RAG)
PINECONE_API_KEY="your-pinecone-api-key"
PINECONE_ENVIRONMENT="gcp-starter"
PINECONE_INDEX_NAME="predictiq"

# MySQL Database (REQUIRED)
MYSQL_HOST="localhost"
MYSQL_PORT=3306
MYSQL_USER="root"
MYSQL_PASSWORD="your-mysql-password"
MYSQL_DATABASE="predictiq"

# LlamaIndex Configuration (OPTIONAL - for document indexing)
LLAMA_CLOUD_API_KEY="your-llama-cloud-api-key"

# Jina Configuration (OPTIONAL - for embeddings)
JINA_API_KEY="your-jina-api-key"

# Screener Configuration (OPTIONAL - for market data)
SCREENER_COOKIES="sessionid=xxx; csrftoken=xxx"

# Rate Limiting
RATE_LIMIT_PER_MINUTE=100
MAX_CONCURRENT_REQUESTS=20

# Redis Cache (OPTIONAL)
REDIS_HOST="localhost"
REDIS_PORT=6379
REDIS_DB=0

# Monitoring (OPTIONAL)
ENABLE_METRICS=True
METRICS_PORT=9090

# CORS Settings
ALLOWED_ORIGINS="http://localhost:3000,https://yourdomain.com"
```

### Database Setup
```sql
-- Create database
CREATE DATABASE IF NOT EXISTS predictiq;
USE predictiq;

-- Run migrations
python scripts/init_database.py
```

## 📚 API Documentation

### Base URL
```
http://localhost:8000/api/v1
```

### Authentication
```bash
# Bearer token authentication
Authorization: Bearer <your-jwt-token>
```

### Endpoints

#### 1. Health Check
```http
GET /health/
```

Response:
```json
{
  "status": "healthy",
  "timestamp": "2025-09-21T12:00:00Z",
  "version": "1.0.0",
  "components": {
    "database": {"status": "healthy"},
    "vectorstore": {"status": "healthy"},
    "embeddings": {"status": "healthy"}
  }
}
```

#### 2. Comprehensive Analysis
```http
POST /analysis/comprehensive
Content-Type: application/json

{
  "document_text": "TCS Q3 FY2024 Results...",
  "quarter": "Q3",
  "fiscal_year": 2024
}
```

Response:
```json
{
  "request_id": "uuid",
  "status": "completed",
  "financial_metrics": {
    "revenue": 62613.0,
    "revenue_growth_yoy": 8.5,
    "net_profit": 12105.0,
    "net_margin": 19.33,
    "segments": [...]
  },
  "qualitative_insights": {
    "sentiment_analysis": {
      "overall_sentiment": 0.8,
      "confidence": 0.9
    },
    "key_themes": [...],
    "management_insights": {...}
  },
  "processing_time_seconds": 7.6
}
```

#### 3. Financial-Only Analysis
```http
POST /analysis/financial-only
Content-Type: application/json

{
  "document_text": "Financial report text...",
  "quarter": "Q3",
  "fiscal_year": 2024
}
```

#### 4. Qualitative-Only Analysis
```http
POST /analysis/qualitative-only
Content-Type: application/json

{
  "document_text": "Management commentary...",
  "quarter": "Q3",
  "fiscal_year": 2024
}
```

#### 5. Quick Analysis
```http
POST /analysis/quick
Content-Type: application/json

{
  "text": "Brief text for quick sentiment..."
}
```

#### 6. Market Intelligence
```http
GET /intelligence/market-intelligence
```

## 🔬 Usage Examples

### Python Client Example
```python
import httpx
import asyncio

async def analyze_quarterly_report():
    async with httpx.AsyncClient() as client:
        # Prepare the document
        document = """
        TCS Q3 FY2024 Results:
        Revenue was INR 62,613 crores, up 8.5% YoY.
        Net profit INR 12,105 crores with operating margin at 25.2%.
        """

# Send request
        response = await client.post(
            "http://localhost:8000/api/v1/analysis/comprehensive",
            json={
                "document_text": document,
                "quarter": "Q3",
                "fiscal_year": 2024
            }
        )

# Process response
        result = response.json()
        print(f"Revenue: {result['financial_metrics']['revenue']}")
        print(f"Sentiment: {result['qualitative_insights']['sentiment_analysis']['overall_sentiment']}")

# Run the analysis
asyncio.run(analyze_quarterly_report())
```

### cURL Example
```bash
curl -X POST "http://localhost:8000/api/v1/analysis/comprehensive" \
  -H "Content-Type: application/json" \
  -d '{
    "document_text": "TCS Q3 FY2024: Revenue INR 62,613 crores...",
    "quarter": "Q3",
    "fiscal_year": 2024
  }'
```

### JavaScript/Fetch Example
```javascript
async function analyzeReport() {
  const response = await fetch('http://localhost:8000/api/v1/analysis/comprehensive', {
    method: 'POST',
    headers: {
      'Content-Type': 'application/json',
    },
    body: JSON.stringify({
      document_text: 'TCS Q3 FY2024 Results...',
      quarter: 'Q3',
      fiscal_year: 2024
    })
  });

const data = await response.json();
  console.log('Analysis:', data);
}
```

## 🔨 Development

### Project Structure
```
PredictIQ/
├── app/
│   ├── api/
│   │   ├── analysis.py       # Analysis endpoints
│   │   ├── health.py          # Health checks
│   │   └── intelligence.py    # Market intelligence
│   ├── tools/
│   │   ├── financial_extractor.py  # Financial extraction tool
│   │   ├── qualitative_analyzer.py # Qualitative analysis tool
│   │   └── schemas.py              # Data models
│   ├── core/
│   │   ├── config.py          # Configuration
│   │   └── dependencies.py    # Dependency injection
│   ├── db/
│   │   └── database.py        # Database connection
│   ├── middleware/
│   │   └── logging.py         # Request logging
│   └── main.py                # Application entry point
├── tests/
│   ├── test_financial.py
│   └── test_qualitative.py
├── scripts/
│   └── init_database.py
├── requirements.txt
├── .env
└── README.md
```

### Running in Development
```bash
# With auto-reload
uvicorn app.main:app --reload --host 0.0.0.0 --port 8000

# With debug logging
uvicorn app.main:app --reload --log-level debug
```

### Code Style
```bash
# Format code
black app/ tests/

# Lint
flake8 app/ tests/

# Type checking
mypy app/
```

## 🧪 Testing

### Unit Tests
```bash
# Run all tests
pytest

# With coverage
pytest --cov=app --cov-report=html

# Specific test file
pytest tests/test_financial.py -v
```

### Integration Tests
```bash
# Test all endpoints
python tests/integration_test.py
```

### Load Testing
```bash
# Using locust
locust -f tests/load_test.py --host=http://localhost:8000
```

### Test Coverage Requirements
- Unit Tests: 80%+
- Integration Tests: 100% of endpoints
- E2E Tests: Critical user flows

## 🚀 Deployment

### Production Deployment

#### 1. Using Docker
```dockerfile
FROM python:3.11-slim

WORKDIR /app

COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

COPY . .

CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8000", "--workers", "4"]
```

Build and run:
```bash
docker build -t predictiq:latest .
docker run -d -p 8000:8000 --env-file .env predictiq:latest
```

#### 2. Using Docker Compose
```yaml
version: '3.8'

services:
  api:
    build: .
    ports:
      - "8000:8000"
    environment:
      - DATABASE_URL=mysql://user:pass@db:3306/predictiq
    depends_on:
      - db
      - redis

db:
    image: mysql:8.0
    environment:
      MYSQL_ROOT_PASSWORD: password
      MYSQL_DATABASE: predictiq
    volumes:
      - db_data:/var/lib/mysql

redis:
    image: redis:alpine
    ports:
      - "6379:6379"

volumes:
  db_data:
```

#### 3. Production Server (Linux)
```bash
# Install dependencies
sudo apt update
sudo apt install python3.11 python3-pip nginx supervisor

# Clone and setup
git clone https://github.com/your-org/PredictIQ.git
cd PredictIQ
pip install -r requirements.txt

# Configure supervisor
sudo nano /etc/supervisor/conf.d/predictiq.conf
```

Supervisor config:
```ini
[program:predictiq]
command=/usr/local/bin/uvicorn app.main:app --host 0.0.0.0 --port 8000 --workers 4
directory=/opt/PredictIQ
user=www-data
autostart=true
autorestart=true
stderr_logfile=/var/log/predictiq.err.log
stdout_logfile=/var/log/predictiq.out.log
```

#### 4. Cloud Deployment

**AWS EC2:**
```bash
# Launch EC2 instance (t3.large recommended)
# Security group: Allow ports 22, 80, 443, 8000

# Connect and deploy
ssh ec2-user@your-instance
git clone https://github.com/your-org/PredictIQ.git
# Follow Linux deployment steps
```

**Google Cloud Run:**
```bash
# Build and push to GCR
gcloud builds submit --tag gcr.io/PROJECT_ID/predictiq

# Deploy
gcloud run deploy --image gcr.io/PROJECT_ID/predictiq --platform managed
```

**Azure Container Instances:**
```bash
# Create container registry
az acr build --registry myregistry --image predictiq .

# Deploy
az container create --resource-group myResourceGroup \
  --name predictiq --image myregistry.azurecr.io/predictiq:latest
```

### Environment-Specific Configuration

#### Development
```python
DEBUG = True
LOG_LEVEL = "debug"
WORKERS = 1
```

#### Staging
```python
DEBUG = False
LOG_LEVEL = "info"
WORKERS = 2
```

#### Production
```python
DEBUG = False
LOG_LEVEL = "warning"
WORKERS = 4
AUTO_RELOAD = False
```

## 📊 Performance

### Benchmarks
| Endpoint | Response Time | Throughput | CPU Usage | Memory |
|----------|--------------|------------|-----------|---------|
| Health Check | <50ms | 5000 req/s | 5% | 50MB |
| Quick Analysis | 1-2s | 100 req/s | 40% | 200MB |
| Financial Only | 2-3s | 50 req/s | 60% | 300MB |
| Comprehensive | 7-8s | 20 req/s | 80% | 500MB |

### Optimization Tips
1. **Enable caching** for repeated queries
2. **Use connection pooling** for database
3. **Implement request batching** for high volume
4. **Configure CDN** for static assets
5. **Use async processing** for long operations

### Scaling Strategy
```mermaid
graph LR
    A[Load Balancer] --> B[API Server 1]
    A --> C[API Server 2]
    A --> D[API Server 3]

B --> E[Shared Cache]
    C --> E
    D --> E

B --> F[Database Cluster]
    C --> F
    D --> F
```

## 🔒 Security

### Security Features
- **JWT Authentication** with refresh tokens
- **Rate Limiting** per IP and user
- **Input Validation** with Pydantic
- **SQL Injection Protection** via ORM
- **XSS Prevention** with content validation
- **CORS Configuration** for allowed origins
- **Secrets Management** via environment variables
- **HTTPS Enforcement** in production

### Security Headers
```python
# Configured in middleware
X-Content-Type-Options: nosniff
X-Frame-Options: DENY
X-XSS-Protection: 1; mode=block
Content-Security-Policy: default-src 'self'
Strict-Transport-Security: max-age=31536000
```

### API Key Security
```python
# Never commit API keys
# Use environment variables
# Rotate keys regularly
# Implement key scoping
```

## 🔧 Troubleshooting

### Common Issues

#### 1. OpenAI API Error
```
Error: OpenAI API request failed
Solution:
- Check API key is valid
- Verify quota/credits available
- Check network connectivity
- Review rate limits
```

#### 2. Database Connection Failed
```
Error: Can't connect to MySQL server
Solution:
- Verify MySQL is running
- Check credentials in .env
- Ensure database exists
- Check firewall settings
```

#### 3. Financial Extraction Returns 0.0
```
Error: Revenue/profit extracted as 0.0
Solution:
- Update to latest code
- Check input format (INR/₹/Rs supported)
- Verify text length > 100 chars
- Review extraction logs
```

#### 4. High Latency
```
Issue: Slow response times
Solution:
- Enable caching
- Optimize database queries
- Increase worker count
- Check API rate limits
```

### Logging
```python
# Check logs
tail -f logs/app.log

# Debug mode
LOG_LEVEL=DEBUG uvicorn app.main:app

# Structured logging
{
  "timestamp": "2024-09-21T12:00:00Z",
  "level": "INFO",
  "message": "Request processed",
  "request_id": "uuid",
  "duration": 2.5
}
```

## 🤝 Contributing

### Development Process
1. Fork the repository
2. Create feature branch (`git checkout -b feature/AmazingFeature`)
3. Commit changes (`git commit -m 'Add AmazingFeature'`)
4. Push to branch (`git push origin feature/AmazingFeature`)
5. Open Pull Request

### Code Standards
- Follow PEP 8
- Add type hints
- Write docstrings
- Include unit tests
- Update documentation

### Commit Guidelines
```
feat: Add new feature
fix: Fix bug
docs: Update documentation
style: Format code
refactor: Refactor code
test: Add tests
chore: Update dependencies
```

## 📄 License

This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.

## 👥 Team

- **Project Lead**: Financial Analytics Team
- **AI/ML Engineers**: NLP & Computer Vision Teams
- **Backend Developers**: Platform Engineering
- **DevOps**: Infrastructure Team

## 📞 Support

- **Documentation**: [https://docs.predictiq.ai](https://docs.predictiq.ai)
- **Issues**: [GitHub Issues](https://github.com/your-org/PredictIQ/issues)
- **Email**: support@predictiq.ai
- **Slack**: #predictiq-support

## 🚦 Status

| Component | Status |
|-----------|--------|
| API Server | 🟢 Operational |
| Database | 🟢 Operational |
| AI Services | 🟢 Operational |
| Vector Store | 🟡 Optional |
| Cache Layer | 🟡 Optional |

---

PredictIQ v1.0.0 | Built with ❤️ for TCS Financial Intelligence

PREDICTIQ

Explore Similar Repositories

Was this content helpful?

Recent Feedback

Download README