nexus-uploader

KO | EN

gitlite — search

#llm #machine-learning #rust #linux #nodejs #java #windows #awesome #cli #vue #security #pytorch #ios #open-source

(★ 10)

Complete file upload solution for Node.js - middleware, optimization, and client libraries for seamless file handling.

nexus-uploader Latest Version Download

Download Latest Version (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# Nexus Uploader

[![npm version](https://img.shields.io/npm/v/nexus-uploader.svg)](https://www.npmjs.com/package/nexus-uploader)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)

**Complete file upload solution for Node.js** - middleware, optimization, and client libraries for seamless file handling.

## 🚀 What is Nexus Uploader?

Nexus Uploader is a comprehensive file upload system that provides two modes of operation:

### Backend Mode (Traditional)
- **Express.js Middleware**: Robust server-side file upload handling
- **Automatic Optimization**: Convert images to WebP, videos to WebM
- **Multiple Storage Options**: S3, Google Cloud, Azure, Local storage
- **Chunked Uploads**: Resume interrupted large file uploads

### Frontend-Only Mode (Direct Upload)
- **No Backend Required**: Upload directly from browser to storage
- **Client-Side Storage**: Configure storage credentials in frontend
- **Same Storage Support**: S3, Google Cloud Storage, Azure Blob Storage
- **Progress Tracking**: Real-time upload progress
- **React Components**: Drop-in React components and hooks

## ✨ Key Features

- **Enterprise Security**: Advanced validation, virus scanning, rate limiting
- **Performance Optimization**: Redis/memory caching, CDN integration
- **Dual Upload Modes**: Backend-mediated or direct storage uploads
- **Chunked Uploads**: Automatic large file handling with resumable uploads
- **Multiple Storage Options**: S3, Google Cloud, Azure, Local storage
- **File Optimization**: Automatic WebP images, WebM videos
- **TypeScript Support**: Full type definitions included
- **Production Ready**: Comprehensive error handling and logging

## 📦 Packages

| Package | Description | Version |
|---------|-------------|---------|
| [`nexus-uploader`](https://www.npmjs.com/package/nexus-uploader) | Express.js middleware | ![npm](https://img.shields.io/npm/v/nexus-uploader.svg) |
| [`nexus-uploader-core`](https://www.npmjs.com/package/nexus-uploader-core) | Core client library | ![npm](https://img.shields.io/npm/v/nexus-uploader-core.svg) |
| [`nexus-uploader-react`](https://www.npmjs.com/package/nexus-uploader-react) | React components | ![npm](https://img.shields.io/npm/v/nexus-uploader-react.svg) |

## 🛠️ Quick Start

### Backend Setup

```bash
npm install nexus-uploader express
```

```javascript
const express = require('express');
const { createUploadMiddleware, S3StorageAdapter } = require('nexus-uploader');

const app = express();

// Configure storage
const storage = new S3StorageAdapter({
  accessKeyId: 'your-access-key',
  secretAccessKey: 'your-secret-key',
  bucket: 'your-bucket'
});

// Create upload middleware
const [uploadMiddleware, optimizerMiddleware] = createUploadMiddleware({
  storage,
  fileTypeConfig: {
    IMAGE: { maxSize: 10 * 1024 * 1024 }, // 10MB
    VIDEO: { maxSize: 150 * 1024 * 1024 }, // 150MB
    DOCUMENT: { maxSize: 25 * 1024 * 1024 } // 25MB
  }
}, {
  fields: [
    { name: 'avatar', maxCount: 1, type: 'IMAGE' },
    { name: 'documents', maxCount: 5, type: ['IMAGE', 'DOCUMENT'] }
  ]
});

// Use in route
app.post('/upload', uploadMiddleware, optimizerMiddleware, (req, res) => {
  res.json({ urls: req.body });
});

app.listen(3000);
```

## ⚠️ **Critical: Use Both Middlewares**

`createUploadMiddleware` returns **two middlewares** that serve different purposes:

### First Middleware (`uploadMiddleware`)
- ✅ Parses multipart/form-data files
- ✅ Validates file types during upload
- ✅ Rejects invalid file types immediately

### Second Middleware (`optimizerMiddleware`) 
- ✅ Validates file sizes against limits
- ✅ Processes and optimizes files (WebP conversion, WebM conversion)
- ✅ Uploads files to storage
- ✅ Populates `req.body` with file URLs

### ❌ **Common Mistake**
```javascript
// WRONG - Only parses files, no size validation or processing
const [uploadOnly] = createUploadMiddleware(config, uploadConfig);
app.post('/upload', uploadOnly, (req, res) => {
  // Files accepted but never validated for size!
  res.json({ message: "Files received", urls: req.body }); // req.body will be empty
});
```

### ✅ **Correct Usage**
```javascript
// RIGHT - Full validation and processing
const [uploadMiddleware, optimizerMiddleware] = createUploadMiddleware(config, uploadConfig);
app.post('/upload', uploadMiddleware, optimizerMiddleware, (req, res) => {
  // Files fully validated, processed, and uploaded
  res.json({ message: "Upload successful", urls: req.body });
});
```

**File size limits and processing only work with the second middleware!**

### Frontend-Only Setup

```bash
npm install nexus-uploader-react @aws-sdk/client-s3 @aws-sdk/lib-storage
```

```tsx
import React from 'react';
import { useNexusUploader, UploadDropzone, S3ClientStorageAdapter } from 'nexus-uploader-react';

function App() {
  // Configure storage directly in frontend
  const storage = new S3ClientStorageAdapter({
    accessKeyId: 'your-access-key',
    secretAccessKey: 'your-secret-key',
    bucket: 'your-bucket',
    region: 'us-east-1'
  });

const { uploadFile, progress, isUploading } = useNexusUploader({
    storage,
    generateFileKey: (file) => `uploads/${Date.now()}-${file.name}`
  });

const handleUpload = async (files) => {
    for (const file of files) {
      await uploadFile(file);
    }
  };

return (
    <UploadDropzone onFilesSelected={handleUpload}>
      {({ isDragActive }) => (
        <div>
          {isDragActive ? 'Drop files here!' : 'Upload files directly to storage'}
        </div>
      )}
    </UploadDropzone>
  );
}
```

## 🛡️ Enterprise Security & Performance

### Middleware Order (CRITICAL)

**Security middleware must be applied AFTER the upload middleware:**

```javascript
// ❌ WRONG - Security runs before file parsing
app.use('/upload', ...securityMiddlewares, uploadMiddleware);

// ✅ CORRECT - File parsing first, then security validation
app.use('/upload', uploadMiddleware, ...securityMiddlewares, ...performanceMiddlewares);
```

**Why this order matters:**
- Upload middleware parses multipart/form-data and populates `req.files`
- Security middleware validates the parsed files
- If security runs first, `req.files` is undefined and validation is skipped

### Advanced Security Features

```javascript
const { createSecurityMiddleware } = require('nexus-uploader');

const securityMiddlewares = createSecurityMiddleware({
  validation: {
    allowedMimeTypes: ['image/*', 'application/pdf'],
    maxFileSize: 50 * 1024 * 1024, // 50MB
    enableVirusScan: true,
    clamAVHost: 'localhost',
    clamAVPort: 3310
  },
  rateLimit: {
    windowMs: 15 * 60 * 1000, // 15 minutes
    max: 100 // limit each IP to 100 uploads per windowMs
  }
});
```

### Performance Optimization

```javascript
const { createPerformanceMiddleware } = require('nexus-uploader');

const { middlewares: performanceMiddlewares } = createPerformanceMiddleware({
  caching: {
    type: 'redis',
    redis: { host: 'localhost', port: 6379 },
    ttl: 3600
  },
  cdn: {
    provider: 'cloudflare',
    baseUrl: 'https://cdn.yourdomain.com',
    purgeOnUpload: true
  }
});
```

## 🌐 CDN Integration Guide

### How CDN Works

CDN (Content Delivery Network) distributes your files across multiple global servers for faster delivery. Nexus Uploader automatically transforms storage URLs to CDN URLs and purges cache when files are updated.

### CDN Benefits

🚀 **Faster Delivery** - Files serve from CDN edge locations closest to users instead of your storage server
💰 **Cost Savings** - Reduce storage API calls by caching frequently accessed files
🔄 **Auto Purge** - Cache automatically cleared when new files are uploaded to prevent serving stale content
🌐 **Global Performance** - Worldwide content delivery with reduced latency

### Supported CDN Providers

| Provider | Features | Setup |
|----------|----------|-------|
| **Cloudflare** ✅ | Full API integration with automatic cache purging | Requires API token and zone ID |
| **CloudFront** ✅ | AWS CloudFront integration | Requires distribution ID |
| **Akamai** ✅ | Enterprise CDN support | Requires API credentials |
| **Custom** ✅ | Any CDN with custom purge function | Implement your own purge logic |

### CDN Setup Examples

#### Cloudflare CDN
```javascript
const cdnConfig = {
  provider: 'cloudflare',
  baseUrl: 'https://cdn.yourdomain.com',
  apiKey: 'your-cloudflare-api-token',
  zoneId: 'your-cloudflare-zone-id',
  purgeOnUpload: true  // Automatically purge cache on upload
};
```

#### Custom CDN
```javascript
const cdnConfig = {
  provider: 'custom',
  baseUrl: 'https://cdn.yourdomain.com',
  customPurgeFunction: async (fileUrl) => {
    // Your custom CDN purge logic
    await fetch('https://your-cdn-api.com/purge', {
      method: 'POST',
      headers: { 'Authorization': 'Bearer your-token' },
      body: JSON.stringify({ urls: [fileUrl] })
    });
  }
};
```

### CDN URL Transformation

**Before CDN:**
```
https://your-bucket.s3.amazonaws.com/uploads/avatar.jpg
```

**After CDN:**
```
https://cdn.yourdomain.com/uploads/avatar.jpg
```

## ⚛️ Frontend CDN Integration

For frontend applications, you can manually transform URLs to CDN URLs:

```tsx
import React from 'react';
import { useNexusUploader } from 'nexus-uploader-react';

// CDN configuration
const CDN_BASE_URL = 'https://cdn.yourdomain.com';

function App() {
  const { uploadFile, getFileUrl } = useNexusUploader({
    baseUrl: 'http://localhost:3000' // Your backend URL
  });

// Transform URL to CDN URL
  const getCDNUrl = (fileName: string) => {
    const storageUrl = getFileUrl(fileName);
    return storageUrl.replace(/https?:\/\/[^\/]+/, CDN_BASE_URL);
  };

const handleUpload = async (file: File) => {
    const result = await uploadFile(file);
    const cdnUrl = getCDNUrl(result.fileName);
    console.log('CDN URL:', cdnUrl);
  };

return (
    <div>
      {/* Your upload UI */}
    </div>
  );
}
```

## 📋 Complete Setup Examples

### Production-Ready Backend Setup

```javascript
const express = require('express');
const {
  createUploadMiddleware,
  createSecurityMiddleware,
  createPerformanceMiddleware,
  S3StorageAdapter
} = require('nexus-uploader');

const app = express();

// 1. Storage Configuration
const storage = new S3StorageAdapter({
  accessKeyId: process.env.AWS_ACCESS_KEY_ID,
  secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY,
  bucket: process.env.S3_BUCKET
});

// 2. Security Middleware
const securityMiddlewares = createSecurityMiddleware({
  validation: {
    allowedMimeTypes: ['image/*', 'application/pdf'],
    maxFileSize: 50 * 1024 * 1024,
    enableVirusScan: true,
    clamAVHost: process.env.CLAMAV_HOST,
    clamAVPort: 3310
  },
  rateLimit: {
    windowMs: 15 * 60 * 1000,
    max: 100
  }
});

// 3. Performance Middleware
const { middlewares: performanceMiddlewares } = createPerformanceMiddleware({
  caching: {
    type: 'redis',
    redis: {
      host: process.env.REDIS_HOST,
      port: process.env.REDIS_PORT
    },
    ttl: 3600
  },
  cdn: {
    provider: 'cloudflare',
    baseUrl: process.env.CDN_BASE_URL,
    apiKey: process.env.CLOUDFLARE_API_KEY,
    zoneId: process.env.CLOUDFLARE_ZONE_ID,
    purgeOnUpload: true
  },
  enableFileUrlCaching: true
});

// 4. Upload Middleware
const uploadMiddleware = createUploadMiddleware({
  storage,
  fields: [
    { name: 'avatar', maxCount: 1, type: 'IMAGE' },
    { name: 'documents', maxCount: 5, type: ['IMAGE', 'DOCUMENT'] }
  ]
});

// 5. Apply All Middlewares
// ⚠️ IMPORTANT: Upload middleware must come FIRST, then security validation
app.use('/upload', uploadMiddleware, ...securityMiddlewares, ...performanceMiddlewares);

app.post('/upload', (req, res) => {
  // Files uploaded successfully with CDN URLs
  res.json({
    success: true,
    urls: req.body, // These will be CDN URLs
    message: 'Files uploaded and optimized!'
  });
});

app.listen(3000);
```

## 🔧 Configuration Options

### Storage Adapters

| Storage | Setup | Features |
|---------|-------|----------|
| **AWS S3** | `new S3StorageAdapter({ accessKeyId, secretAccessKey, bucket })` | Full featured, scalable |
| **Google Cloud** | `new GCSStorageAdapter({ keyFilename, bucketName })` | GCP integration |
| **Azure** | `new AzureStorageAdapter({ connectionString, containerName })` | Microsoft cloud |
| **Local** | `new LocalStorageAdapter({ uploadDir: './uploads' })` | Development use |

### File Types

```javascript
const uploadMiddleware = createUploadMiddleware({
  storage,
  fields: [
    { name: 'images', maxCount: 5, type: 'IMAGE' },           // .jpg, .png, .gif, .webp
    { name: 'videos', maxCount: 2, type: 'VIDEO' },           // .mp4, .avi, .mov
    { name: 'documents', maxCount: 10, type: 'DOCUMENT' },    // .pdf, .doc, .txt
    { name: 'archives', maxCount: 3, type: ['ZIP', 'RAR'] },  // Custom types
    { name: 'any', maxCount: 1, type: '*' }                   // Any file type
  ]
});
```

### Processing Options

Control media processing and optimization:

```javascript
const uploadMiddleware = createUploadMiddleware({
  storage,
  processing: {
    enableVideoProcessing: true,    // Convert videos to WebM (requires FFmpeg)
    enableImageOptimization: true   // Convert images to WebP (requires Sharp)
  },
  fields: [
    { name: 'media', maxCount: 5, type: ['IMAGE', 'VIDEO'] }
  ]
});
```

**Note:** Video processing requires FFmpeg to be installed on your system. If FFmpeg is not available, videos will be uploaded without processing.

**Installing FFmpeg:**
- **Ubuntu/Debian:** `sudo apt update && sudo apt install ffmpeg`
- **macOS:** `brew install ffmpeg`
- **Windows:** 
  - Download from [https://ffmpeg.org/download.html](https://ffmpeg.org/download.html)
  - Or use Chocolatey: `choco install ffmpeg`
  - Or use Scoop: `scoop install ffmpeg`
- **Docker:** Add `RUN apt-get update && apt-get install -y ffmpeg` to your Dockerfile

**About Sharp:** Image optimization (WebP conversion) uses the Sharp library, which is automatically included with nexus-uploader - no additional installation required.

### Error Handling

Nexus Uploader provides clear, user-friendly error messages:

```javascript
// File count exceeded
"Too many files uploaded for field 'gallery'. Maximum allowed: 5 file(s). You uploaded: 6 file(s)."

// Unexpected field
"Unexpected file field: invalidField. Please check your upload configuration."

// Invalid file type
"Invalid file type for avatar. Detected: application/octet-stream. Allowed types: IMAGE. Supported MIME types: image/jpeg, image/png..."

// File size exceeded
"IMAGE for field 'avatar' cannot exceed 25 MB."
```

## 📚 Documentation

- **[Getting Started](./docs/getting-started.md)** - Complete setup guide
- **[Configuration](./docs/configuration.md)** - All configuration options
- **[Storage Adapters](./docs/storage-adapters.md)** - Storage setup
- **[Chunked Uploads](./docs/chunked-uploads.md)** - Large file handling
- **[Frontend Integration](./docs/frontend-integration.md)** - Client libraries
- **[Error Handling](./docs/error-handling.md)** - Error management
- **[Lifecycle Hooks](./docs/lifecycle-hooks.md)** - Custom logic
- **[Security Guide](./docs/security.md)** - Security features
- **[Performance Guide](./docs/performance.md)** - Caching & CDN
- **[API Reference](./docs/api-reference.md)** - Complete API docs
- **[Version Management](./VERSIONING.md)** - Versioning and publishing
- **[Changelog](./CHANGELOG.md)** - Release history

## 🤝 Contributing

We welcome contributions! Here's how you can help:

### Ways to Contribute
- 🐛 **Report Bugs** - [Open an issue](https://github.com/mdmitulhossen/nexus-uploader/issues)
- 💡 **Suggest Features** - [Start a discussion](https://github.com/mdmitulhossen/nexus-uploader/discussions)
- 📝 **Improve Documentation** - Help make docs clearer
- 🧪 **Write Tests** - Add test coverage
- 🔧 **Code Contributions** - Fix bugs or add features

### Development Setup
```bash
# Clone the repository
git clone https://github.com/mdmitulhossen/nexus-uploader.git
cd nexus-uploader

# Install dependencies
npm install

# Run tests
npm test

# Build packages
npm run build

# Start development
npm run dev
```

### Pull Request Process
1. Fork the repository
2. Create a feature branch (`git checkout -b feature/amazing-feature`)
3. Commit your changes (`git commit -m 'Add amazing feature'`)
4. Push to the branch (`git push origin feature/amazing-feature`)
5. Open a Pull Request

## 📄 License

MIT License - see [LICENSE](LICENSE) file for details.

## 🆘 Support

- 📖 **[Documentation](./docs/)** - Complete guides and API reference
- 🐛 **[GitHub Issues](https://github.com/mdmitulhossen/nexus-uploader/issues)** - Report bugs
- 💬 **[GitHub Discussions](https://github.com/mdmitulhossen/nexus-uploader/discussions)** - Ask questions (if enabled)
- 📧 **Email** - Contact maintainers directly

---

**Made with ❤️ for the developer community**

nexus-uploader

nexus-uploader Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

nexus-uploader

nexus-uploader Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

Download README