BDM

KO | EN

gitlite — search

#javascript #react #llm #android #machine-learning #linux #nodejs #windows #awesome #kubernetes #awesome-list #mcp #openai #open-source

(★ 58)

Course covers big data fundamentals, processes, technologies, platform ecosystem, and management for practical application development.

BDM 최신버젼 다운로드

최종 버전 다운로드 (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

<a href="https://github.com/drshahizan/BDM/stargazers"><img src="https://img.shields.io/github/stars/drshahizan/BDM" alt="Stars Badge"/></a>
<a href="https://github.com/drshahizan/BDM/network/members"><img src="https://img.shields.io/github/forks/drshahizan/BDM" alt="Forks Badge"/></a>
<a href="https://github.com/drshahizan/BDM/pulls"><img src="https://img.shields.io/github/issues-pr/drshahizan/BDM" alt="Pull Requests Badge"/></a>
<a href="https://github.com/drshahizan/BDM"><img src="https://img.shields.io/github/issues/drshahizan/BDM" alt="Issues Badge"/></a>
<a href="https://github.com/drshahizan/BDM/graphs/contributors"><img alt="GitHub contributors" src="https://img.shields.io/github/contributors/drshahizan/BDM?color=2b9348"></a>
![Visitors](https://api.visitorbadge.io/api/visitors?path=https%3A%2F%2Fgithub.com%2Fdrshahizan%2BDM&labelColor=%23d9e3f0&countColor=%23697689&style=flat)

# Big Data Management

## Course Synopsis
This course provides a basic fundamental of big data architecture and management.  Students will learn the big data processes and the current big data technologies that are available. Further, students will be exposed to the big data platform ecosystem for big data manipulation. The big data management will be explored for the best practice in managing and manipulating large amount of data. At the end of the course, students should be able to understand the architecture and management of big data and also can develop simple application of big data handling using particular platform in assignment.

## Course Learning Outcomes
1. Understand the technology for managing, processing and manipulating large amount of data.
2. Design big data platform demonstrating the implementation of big data applications.
3. Discuss current technology that support for sustainability of the big data platform ecosystem.

## 🔥 Important things
1. [How to Become a Data Engineer in 2024](./materials/roadmap.md)
2. [Essential Preparations for a Successful Start in BDM Class](./materials/essentials.md)
3. [Student Information](student.md)
4. [Course Information](./images/L1_BigDataManagement.pdf)
5. [Assignment](./assignment)
6. [Exercise](./exercise)
   
## Notes

### 1. Data

| No. | Content | File | 
|----: |----------------------------------------------------|------|
| 1. | Charting Your Path in Data and Machine Learning | <a href="./materials/de_ds.md" ><img src="./images/rfp.png" width="24px" height="24px" ></a> |
| 2. | Navigating the Data Science Landscape | <a href="./materials/ds_landscape.md" ><img src="./images/rfp.png" width="24px" height="24px" ></a> |
| 3. | Database Types | <a href="./materials/db_type.md" ><img src="./images/rfp.png" width="24px" height="24px" ></a> |
| 3a. | [NoSQL Database](https://github.com/drshahizan/special-topic-data-engineering/tree/main/materials/mongodb)|<a href="https://github.com/drshahizan/special-topic-data-engineering/blob/main/materials/notes/mod3.md" ><img src="./images/rfp.png" width="24px" height="24px" ></a>| 
| 4. | Navigating the Database Landscape | <a href="./materials/choose_db.md" ><img src="./images/rfp.png" width="24px" height="24px" ></a> |
| 5. | Data Management | <a href="./materials/data_mng.md" ><img src="./images/rfp.png" width="24px" height="24px" ></a> |
| 6. | The Data Journey: From Raw to Refined | <a href="./materials/data_journey.md" ><img src="./images/rfp.png" width="24px" height="24px" ></a> |
| 7. | The Data to MLOps Journey: An End-to-End Process | <a href="./materials/data_mlops.md" ><img src="./images/rfp.png" width="24px" height="24px" ></a> |
| 8. | Data Platforms Architecture: Governance and Operations | <a href="./materials/data_platform.md" ><img src="./images/rfp.png" width="24px" height="24px" ></a>|
| 9. | Creating Data Products to Monetize Data | <a href="./materials/data_product.md" ><img src="./images/rfp.png" width="24px" height="24px" ></a> |
| 10. | Revolutionizing Data and Machine Learning with DataOps and MLOps | <a href="./materials/dataops.md" ><img src="./images/rfp.png" width="24px" height="24px" ></a> |
| 11. | Data Science for Beginners - A Curriculum | <a href="https://github.com/microsoft/Data-Science-For-Beginners/tree/main" ><img src="./images/rfp.png" width="24px" height="24px" ></a> |

### 2. Case Study
| No. | Content | File |
|----: |----------------------------------------------------|------|
| 1. | Unveiling Instagram's Engagement Magic through Machine Learning | <a href="./materials/cs_insta.md" ><img src="./images/rfp.png" width="24px" height="24px" ></a> |
| 2. | Unlocking Spotify's Musical Enchantment with Machine Learning | <a href="./materials/cs_spotify.md" ><img src="./images/rfp.png" width="24px" height="24px" ></a> |
| 3. | Netflix | <a href="./materials/cs_netflix.md" ><img src="./images/rfp.png" width="24px" height="24px" ></a> |
| 4. | Big data in healthcare: management, analysis and future prospects | <a href="https://github.com/drshahizan/BDM/blob/main/images/art_bdata.pdf" ><img src="./images/rfp.png" width="24px" height="24px" ></a> |
| 5. | Introduction to Big Data Computing for Geospatial Applications | <a href="https://www.mdpi.com/2220-9964/9/8/487" ><img src="./images/rfp.png" width="24px" height="24px" ></a> |
| 6. | 40 Stats and Real-Life Examples of How Companies Use Big Data | <a href="https://www.scnsoft.com/blog/big-data-use-cases-stats-and-examples" ><img src="./images/rfp.png" width="24px" height="24px" ></a> |

### 3. Big Data Management

| **No.** | **Section** | **Content** |
|--------:|---------------------------------------------|--------------------------------------------------------|
| **1** | [**Introduction to Big Data Management**](./materials/1intro_bdm.md) | A. What is Big Data? B. The Importance of Managing Big Data C. Why Big Data Management is Important? D. The History of Big Data |
| **2** | [**Understanding Big Data**](./materials/2understand_bd.md) | A. Defining Big Data B. Characteristics of Big Data C. Sources of Big Data D. Challenges in Dealing with Big Data E. The workflow of Big Data Management |
| **3** | [**The Role of Big Data Management**](./materials/3role_bdm.md) | A. Managing big data B. Benefits of Effective Big Data Management C. Risks of Ignoring Big Data Management D. Industries Benefiting from Big Data Management |
| **4** | [**Data Collection and Storage**](./materials/4data_collection.md) | A. Data Collection Methods &nbsp;&nbsp;&nbsp;1. Traditional Data Sources &nbsp;&nbsp;&nbsp;2. Emerging Data Sources B. Data Storage and Warehousing C. Data Security and Privacy Concerns |
| **5** | [**Data Processing and Analysis**](./materials/5data_process.md) | A. Data Preprocessing B. Data Analysis Tools and Techniques C. Real-time Data Processing D. Machine Learning in Big Data Analysis |
| **6** | **Data Integration and Governance** | A. Data Integration Strategies B. Data Governance Best Practices C. Ensuring Data Quality |
| **7** | **Data Visualization** | A. Importance of Data Visualization B. Data Visualization Tools C. Creating Effective Data Visualizations |
| **8** | **Case Studies** | A. Successful Big Data Management Implementations B. Lessons Learned from Failed Big Data Projects C. Real-world Examples |
| **9** | **Challenges in Big Data Management** | A. Scalability Issues B. Security Concerns C. Compliance and Regulatory Challenges |
| **10** | **Future Trends in Big Data Management** | A. The Evolving Landscape of Big Data B. Predictions for the Future C. Preparing for the Next Generation of Big Data |
| **11** | **Best Practices in Big Data Management** | A. Key Takeaways B. Actionable Strategies C. Tips for Effective Big Data Management |
| **12** | **Other References** | |

## Weekly Schedule

| Week | Topic |
|:----------:|---------------------------------------------------------|
| 1 | **Introduction to Big Data and Big Data Analytics** - Fundamentals and concepts of big data |
| 2 | **Big Data Processing and Technology** - Batch, real-time, and streaming processing. - Scalability, storage, sourcing challenges. |
| 3-4 | - ACID, BASE, and CAP theorem - Distributed File Processing & Map Reduce Processing |
| 5 | - Lambda Architecture |
| 6-7 | **Relational Database (RDBMS)** - Relational Data Modelling - Database design phases |
| 8 | **Relational Database (RDBMS)** - SQL programming (DDL, DML, CRUD Operation) |
| 9 | **Relational Database (RDBMS)** - SQL programming (Subqueries, Join Tables, Aggregate) |
| 10 | **No SQL Database** - Introduction to No SQL database - Semi-structured data Modelling (Key Value, Column Family, Document, and Graph) |
| 11-12| **No SQL database (Document-based Database)** - Document-based data modelling - MongoDB query language |
| 13-14| **[Cloud Technology](./materials/cloud.md)** - Introduction to Cloud - AWS Cloud (via AWS Learning Management System) |
| 15 | Project Presentation |

## Contribution 🛠️
Please create an [Issue](https://github.com/drshahizan/BDM/issues) for any improvements, suggestions or errors in the content.

You can also contact me using [Linkedin](https://www.linkedin.com/in/drshahizan/) for any other queries or feedback.

[![Visitors](https://api.visitorbadge.io/api/visitors?path=https%3A%2F%2Fgithub.com%2Fdrshahizan&labelColor=%23697689&countColor=%23555555&style=plastic)](https://visitorbadge.io/status?path=https%3A%2F%2Fgithub.com%2Fdrshahizan)
![](https://hit.yhype.me/github/profile?user_id=81284918)

BDM

BDM 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

BDM

BDM 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

Download README