KO | EN

GitLite

Minimalist Open Source Repository Search Engine

gitlite — search

#dsh #dsh-plugin #ai-agents #developer-tools #typescript #codex #claude-code #mcp #rust #deepseek-harness #ai-agent #cli

easyLM

★ 2 Open GitHub ↗

Simplified LLM Creation Framework

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# easyLM - Simplified LLM Creation Framework

## Overview
easyLM is an early alpha-stage framework designed to simplify the creation and training of your own language models. The project is currently in active development and has successfully completed its first CPU training run. While not production-ready, easyLM provides a foundation for experimenting with transformer-based language models with minimal setup.

**Current Status:** Early Alpha - Not production ready. Breaking changes may occur.

## Features

- **Simplified Architecture**: Clean implementation of modern transformer components
- **Training Pipeline**: End-to-end data processing and training pipeline
- **Conversational AI Ready**: Built-in support for chat-style interactions
- **Customizable**: Easy to modify model architecture and training parameters
- **CPU/GPU Compatible**: Runs on both CPU and GPU (with TensorFlow)

## Quick Start

### Installation

```
# Clone the repository
git clone https://github.com/csoftware-arigpt/easyLM.git
cd easyLM

# Install dependencies
pip install -r requirements.txt
```

### Basic Usage

1. **Prepare Training Data**:
```
# Create a datasets directory
mkdir datasets

# Add your training data (CSV or JSON format)
# Example CSV should have columns: instruction, output
# Currently tested on Alpaca
```

2. **Generate Training Data**:
```
python make_data.py
```

3. **Train the Model**:
```
python train.py
```

4. **Chat with the Model**:
```
python chat.py
```

## Architecture

easyLM implements a GPT-style architecture with modern improvements:

- **Grouped Query Attention (GQA)**: More efficient attention mechanism
- **RMSNorm**: Faster alternative to LayerNorm
- **SwiGLU Activation**: Improved feed-forward networks
- **Rotary Positional Encoding (RoPE)**: Better sequence length handling

## Configuration

The framework uses a centralized configuration file (`config.py`):

```
# Model Architecture
EMBED_DIM = 128
NUM_HEADS = 4
NUM_LAYERS = 2
VOCAB_SIZE = 10000
MAX_LEN = 64

# Training
BATCH_SIZE = 32
EPOCHS = 10
LEARNING_RATE = 2e-3
```

## Data Format

Training data should follow this JSONL format:

```
{"messages": [
  {"role": "system", "content": "You are a helpful assistant."},
  {"role": "user", "content": "What is 2+2?"},
  {"role": "assistant", "content": "2+2 equals 4."}
]}
```

## Project Structure

```
easyLM/
├── make_data.py          # Data preparation and generation
├── train.py             # Model training script
├── chat.py              # Interactive chat interface
├── model.py             # Model architecture definitions
├── data_loader.py       # Data loading and preprocessing
├── config.py            # Configuration settings
└── datasets/            # Training data directory
```

## Current Limitations

- **Early Alpha**: Many features are incomplete
- **Performance**: Not optimized for production use
- **Documentation**: Limited examples and guides
- **Model Size**: Currently supports smaller models only
- **No Fine-tuning**: Only basic training supported

## Roadmap

- [x] Basic transformer implementation
- [x] CPU training pipeline
- [ ] GPU optimization
- [ ] Model checkpointing
- [ ] Evaluation metrics
- [ ] Web interface
- [ ] Pre-trained model releases
- [ ] Fine-tuning support
- [ ] Quantization support

## Requirements

- Python 3.8+
- TensorFlow 2.x
- NumPy
- Pandas

## Contributing

Since easyLM is in early alpha, we welcome feedback and suggestions. Please note that the codebase is rapidly evolving.

1. Fork the repository
2. Create a feature branch
3. Submit a pull request with detailed description

## License

This project is available under the MIT License. See LICENSE file for details.

## Disclaimer

This is an experimental framework. The models created with easyLM may produce inaccurate, biased, or inappropriate content. Use at your own risk.

## Getting Help

For questions and discussions:
- Open an issue on GitHub
- Check the examples directory
- Review the source code

---

**Note:** easyLM is in active development. APIs may change without notice. Not recommended for production use.

easyLM

Explore Similar Repositories

Was this content helpful?

Recent Feedback

Download README