distributed-map-reduce

KO | EN

gitlite — search

#python #react #ai #typescript #llm #android #deep-learning #cli #vue #security #pytorch #ios #openai #open-source

distributed-map-reduce

(★ 10)

An experimental distributed map reduce system based on Google's MapReduce, written in Rust!

distributed-map-reduce 최신버젼 다운로드

최종 버전 다운로드 (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# Distributed Map Reduce
An experimental implementation of Map Reduce based on Google's Map Reduce [paper](https://static.googleusercontent.com/media/research.google.com/en//archive/mapreduce-osdi04.pdf) written in Rust.

```
        ┌─────────────────┐                           ┌─────────────────────┐
        │ Coordinator     │                           │      Worker         │
        │                 │                           │                     │
        └─────────┬───────┘                           └──────────┬──────────┘
                  │                                              │
                  │                                              │
                  │Provisions                                    │Provisions
                  │                                              │
                  │                                              │
┌─────────────────┼─────────────────┐     ┌──────────────────────▼────────────────────┐
│                 │                 │     │                                           │
│                 │                 │     │                                           │
│                 │                 │     │              ┌───────────────────┐        │
│                 │                 │     │              │   Worker Node 1   │        │
│                 │                 │     │      ┌◄──────┤                   │        │
│                 │                 │     │      │       └───────────────────┘        │
│                 │                 │     │      │                                    │
│         ┌───────▼──────────┐      │     │      │       ┌───────────────────┐        │
│         │                  │      │     │      │       │  Worker Node 2    │        │
│         │ Coordinator Node ◄──────┼─────┼──────┤◄──────┤                   │        │
│         └──────────────────┘      │     │      │       └───────────────────┘        │
│                                   │     │      │                                    │
│                                   │     │      │       ┌───────────────────┐        │
│                                   │     │      │       │   Worker Node 3   │        │
│                                   │     │      └◄──────┤                   │        │
│                                   │     │              └───────────────────┘        │
│                                   │     │                                           │
│                                   │     │                                           │
│                                   │     │                                           │
└───────────────────────────────────┘     └───────────────────────────────────────────┘
         Coordiantor Plane                              Worker Plane
```

## Instructions to Run
First start the coordinator
```
cargo run --bin coordinator -- <n_reduce> <filenames>
```

Example: `cargo run --bin coordinator -- 5 *.txt`

Then start the workers
```
cargo run --bin worker -- 127.0.0.1:12345 <number_of_workers>
```

Example: `cargo run --bin worker -- 127.0.0.1:12345 10`

distributed-map-reduce

distributed-map-reduce 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

distributed-map-reduce

distributed-map-reduce 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

Download README