DIY-A-Cluster

KO | EN

gitlite — search

#ai #golang #android #machine-learning #rust #docker #java #awesome #cli #kubernetes #vue #pytorch #security #open-source

(★ 11)

How to Do-It-Yourself A Cluster for Spark & Hadoop

DIY-A-Cluster Latest Version Download

Download Latest Version (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# How to Do-It-Yourself A Cluster for Spark & Hadoop

It's unusual to have opportunity working on server farm, but it's not that difficult to build a cluster by yourself, with some commodity machines, like your old PC, not-use-anymore laptop, or even Raspberry Pi. It's not a bad idea to build such a cluster and try Apache Spark together with Hadoop on it.

In this document, I'm going to introduce the process to set up a cluster by yourself, including the hardware needed, network configuration, software installation, as well as some issues I encountered. It may be helpful for you too.

- [Hardware Requirement](https://github.com/XD-DENG/DIY-A-Cluster/blob/master/chapters/hardware_requirement.md)
- [Network Basics](https://github.com/XD-DENG/DIY-A-Cluster/blob/master/chapters/network_basics.md)
- [Set Up Hadoop Distributed File System](https://github.com/XD-DENG/DIY-A-Cluster/blob/master/chapters/set_up_hadoop_distributed_file_system.md)
- [Set Up Apache Spark](https://github.com/XD-DENG/DIY-A-Cluster/blob/master/chapters/set_up_apache_spark.md)
- [Running Example](https://github.com/XD-DENG/DIY-A-Cluster/blob/master/chapters/running_example.md)
- [Common Issues](https://github.com/XD-DENG/DIY-A-Cluster/blob/master/chapters/common_issues.md)
- [References](https://github.com/XD-DENG/DIY-A-Cluster/blob/master/chapters/reference.md)

![title_pic](https://github.com/XD-DENG/DIY-A-Cluster/raw/master/images/cluster_title_pic.jpg)

DIY-A-Cluster

DIY-A-Cluster Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

DIY-A-Cluster

DIY-A-Cluster Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

Download README