KO | EN

GitLite

Minimalist Open Source Repository Search Engine

gitlite — search

#dsh #ai-agents #dsh-plugin #codex #developer-tools #claude-code #windows #desktop-app #typescript #rust #ai-agent #cli

awesome-big-data

★ 29 Open GitHub ↗

大数据&&分布式系统学习过程中一些经验总结

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# awesome-big-data
大数据&&分布式系统学习过程中一些经验总结

## 计算/查询/存储引擎相关

### RemoteShuffleService(RSS)
- [各大厂RSS实现](https://zhuanlan.zhihu.com/p/462338206)

### flink
- [flink实战总结](./docs/engine/Flink实战总结.pdf)
- [flink学习总结](./docs/engine/Flink学习.pdf)

### spark
- [spark实战总结](./docs/engine/Spark学习笔记.md)
- [【spark-tips】spark2-4-0触发的executor内存溢出排查.md](./docs/engine/spark2-4-0触发的executor内存溢出排查.md)
- [【Spark源码分析】Dynamic-Resource-Allocation设计的思考](./docs/engine/Dynamic-Resource-Allocation设计的思考.md)
- [【Spark源码分析】Broadcast](./docs/engine/Broadcast.md)
- [【Spark源码分析】Job提交执行过程详解](./docs/engine/spark-job-submit.md)

### impala
- [impala集群搭建](./docs/engine/impala集群搭建.md)

## 任务&资源调度相关

- [Airflow 实战总结](./docs/scheduler/airflow实战总结.md)

### Hadoop
- [Yarn架构实现解析](./docs/scheduler/Yarn架构解析.pdf)
- [Yarn-Federation源码串读](./docs/scheduler/Yarn-Federation源码串读.md)
- [Hadoop&Yarn Rpc源码剖析](./docs/scheduler/Hadoop-Rpc源码分析.md)
- [MR任务在Hadoop子系统中状态流转](./docs/scheduler/MR任务在Hadoop子系统中状态流转.md)
- [Hadoop Pipes Ping Timeout问题排查](https://zhuanlan.zhihu.com/p/358167020)
- [动态调整MR运行时object property方案](https://zhuanlan.zhihu.com/p/349907241)

### kubernetes
- [kubernetes federation深度解析](./docs/engine/kubernetes-federation深度解析.md)
- [kubernetes shared Informer 源码解析](https://zhuanlan.zhihu.com/p/255078405)

## OLAP 相关

### kylin
- [Kylin学习笔记](./docs/olap/Kylin学习笔记.md)
- [Kylin二次开发——测试环境搭建](./docs/olap/Kylin学习笔记.md)
- [kylin-master-slave同步原理及问题排查.md](./docs/olap/kylin-master-slave同步原理及问题排查.md)
- [kylin-query原理剖析.md](./docs/olap/kylin-query原理剖析.md)

### Doris
- [okhttp-support-100-continue-for-palo.md](./docs/olap/okhttp-support-100-continue-for-palo.md)

## 分布式系统论文

- [Howard大佬维护的共识算法知识库](https://github.com/heidihoward/distributed-consensus-reading-list)

- [Raft](./docs/learning/raft.pdf)
  - [Raft学习总结](./docs/learning/Raft论文学习.md)
- Flink分布式数据流的异步快照核心实现
	- [chandy-lamport-algorithm](./docs/learning/chandy-lamport-algorithm.pdf)
	- [Lightweight Asynchronous Snapshots for Distributed Dataflows](./docs/learning/Snapshots-for-Distributed-Dataflows.pdf)
- Massively Parallel Processing (MPP Engine)
  - [Massively Parallel Databases and MapReduce
    Systems](./docs/learning/massive-parallel-processing.pdf)
- 火山模型
	- [Volcano an Extensible and Parallel Query Evaluation System](./docs/learning/Volcano.pdf)

- spark论文
	- [Resilient Distributed Datasets: A Fault-Tolerant Abstraction for In-Memory Cluster Computing](./docs/learning/spark设计论文.pdf)
	- [Structured Streaming: A Declarative API for Real-Time Applications in Apache Spark](./docs/learning/sigmod_structured_streaming.pdf)

- [Google Borg](./docs/learning/Google-With-Borg.pdf)

## 优秀博文汇总
- [大数据&&后端优秀博文汇总](./docs/learning/优秀博文汇总.pdf)

awesome-big-data

Explore Similar Repositories

Was this content helpful?

Recent Feedback

Download README