KO
|
EN
gitlite — search
Search
#typescript
#ai-agents
#ai
#dsh-plugin
#deepseek-harness
#open-source
#cli
#claude-code
#codex
#developer-tools
#react
#windows
Cookbook
★ 15,300
Open GitHub ↗
The Data Engineering Cookbook
Download README (.md)
Explore Similar Repositories
DeDRM_tools
:
DeDRM tools for ebooks
headless-recorder
:
Chrome extension that records your browser interactions and generates a Playwright or Puppeteer script.
ChuanhuChatGPT
:
GUI for ChatGPT API and many LLMs. Supports agents, file-based QA, GPT finetuning and query with web search. All with a neat UI.
xiaohongshu-mcp
:
MCP for xiaohongshu.com
lime
:
Open source API-compatible alternative to the text editor Sublime Text
// repository documentation
Was this content helpful?
★ 0
(0 ratings)
Select Rating:
★
★
★
★
★
Submit Feedback
Recent Feedback
×
Download README
Do you want to download the
README.md
file for
Cookbook
?
Download (.md)
<!--- # The Data Engineering Cookbook --> <div align="center"> <img width="341" height="426" src="images/CookbookCover.jpg" alt="Data Engineering Cookbook"> <br> <br> <br> </div> <p align="center"> <a href="sections/01-Introduction.md">What is this Book?</a> <a href="#how-to-contribute">How to Contribute</a> <a href="https://www.youtube.com/channel/UCY8mzqqGwl5_bTpBY9qLMAA">YouTube</a> <a <a href="https://twitter.com/andreaskayy">Twitter</a> <a href="https://www.amazon.com/shop/plumbersofdatascience">Amazon Shop</a> </p> <br> ## If You Like This Book & Need More Help Check out my Data Engineering Academy at LearnDataEngineering.com trusted by almost 2,000 students! **Visit learndataengineering.com:** [Click Here](https://learndataengineering.com) - Learn Data Engineering with our online Academy - Perfect for becoming a Data Engineer or add Data Engineering to your skillset - Proven process based on years of experience and hundreds of hours of personal coaching - Over 30 prepared courses on the most important techniques, fundamental tools and platforms plus our - Associate Data Engineer Certification - Academy Discord server with over 1,000 members ## Support This Book For Free! - **Amazon:** [Click Here](https://www.amazon.com/shop/plumbersofdatascience) buy whatever you like from Amazon using this link* (Also check out my complete podcast gear and books) <!--- I get asked super often how to become a Data Engineer. That's why I decided to start this cookbook with all the topics you need to look into. It's not only useful for beginners, professionals will definitely like the case study section. If you look for the old PDF version it's [here](https://github.com/andkret/Cookbook/raw/LaTex-Version-Deprecated/Data%20Engineering%20Cookbook.pdf) --> ## Here's what's new: Find the change log with all recent updates here: [SEE UPDATES](sections/10-Updates.md) ## Building a Portfolio Project? Use my project README template to document it like a professional would: [Get the template](Code%20Examples/template-readme.md) # Contents: - [Introduction](sections/01-Introduction.md) - [Basic Engineering Skills](sections/02-BasicSkills.md) - [Advanced Engineering Skills](sections/03-AdvancedSkills.md) - [Free Hands On Courses / Tutorials](sections/04-HandsOnCourse.md)‚ - [Case Studies](sections/05-CaseStudies.md) - [Best Practices Cloud Platforms](sections/06-BestPracticesCloud.md) - [130+ Data Sources for Portfolio Projects](sections/07-DataSources.md) - [Project README Template](Code%20Examples/template-readme.md) - [1001 Interview Questions](sections/08-InterviewQuestions.md) - [Recommended Books, Courses, and Podcasts](sections/09-BooksAndCourses.md) - [Updates](sections/10-Updates.md) <!-- test --> - [How To Contribute](#how-to-contribute) - [Support What You Like](#support) - [Important Links](#important-links) # Full Table Of Contents: ## Introduction - [What is this Cookbook](sections/01-Introduction.md#what-is-this-cookbook) - [Data Engineers](sections/01-Introduction.md#data-engineers) - [My Data Science Platform Blueprint](sections/01-Introduction.md#my-data-science-platform-blueprint) - [Connect](sections/01-Introduction.md#connect) - [Buffer](sections/01-Introduction.md#buffer) - [Processing Framework](sections/01-Introduction.md#processing-framework) - [Store](sections/01-Introduction.md#store) - [Visualize](sections/01-Introduction.md#visualize) - [Who Companies Need](sections/01-Introduction.md#who-companies-need) - [How to Learn Data Engineering](sections/01-Introduction.md#how-to-learn-data-engineering) - [Andreas on the Super Data Science Podcast](sections/01-Introduction.md#Interview-with-Andreas-on-the-Super-Data-Science-Podcast) - [Building Blocks to Learn Data Engineering](sections/01-Introduction.md#building-blocks-to-learn-data-engineering) - [Roadmap for Beginners](sections/01-Introduction.md#roadmap-for-beginners) - [Roadmap for Data Analysts](sections/01-Introduction.md#roadmap-for-data-analysts) - [Roadmap for Data Scientists](sections/01-Introduction.md#roadmap-for-data-scientists) - [Roadmap for Software Engineers](sections/01-Introduction.md#roadmap-for-software-engineers) - [Data Engineers Skills Matrix](sections/01-Introduction.md#data-engineers-skills-matrix) - [How to Become a Senior Data Engineer](sections/01-Introduction.md#how-to-become-a-senior-data-engineer) ## Basic Engineering Skills - [Learn To Code](sections/02-BasicSkills.md#learn-to-code) - [Get Familiar With Git](sections/02-BasicSkills.md#get-familiar-with-git) - [Agile Development](sections/02-BasicSkills.md#agile-development) - [Why is agile so important?](sections/02-BasicSkills.md#Why-is-agile-so-important) - [Agile rules I learned over the years](sections/02-BasicSkills.md#agile-rules-i-learned-over-the-years) - [Agile Frameworks](sections/02-BasicSkills.md#agile-frameworks) - [Scrum](sections/02-BasicSkills.md#scrum) - [OKR](sections/02-BasicSkills.md#okr) - [Software Engineering Culture](sections/02-BasicSkills.md#software-engineering-culture) - [Learn how a Computer Works](sections/02-BasicSkills.md#learn-how-a-computer-works) - [Data Network Transmission](sections/02-BasicSkills.md#data-network-transmission) - [Security and Privacy](sections/02-BasicSkills.md#security-and-privacy) - [SSL Public and Private Key Certificates](sections/02-BasicSkills.md#ssl-public-and-private-key-Certificates) - [JSON Web Tokens](sections/02-BasicSkills.md#json-web-tokens) - [GDPR regulations](sections/02-BasicSkills.md#gdpr-regulations) - [Linux](sections/02-BasicSkills.md#linux) - [OS Basics](sections/02-BasicSkills.md#os-basics) - [Shell scripting](sections/02-BasicSkills.md#shell-scripting) - [Cron Jobs](sections/02-BasicSkills.md#cron-jobs) - [Packet Management](sections/02-BasicSkills.md#packet-management) - [Docker](sections/02-BasicSkills.md#docker) - [What is Docker and How it Works](sections/02-BasicSkills.md#what-is-docker-and-what-do-you-use-it-for) - [Don't Mess Up Your System](sections/02-BasicSkills.md#dont-mess-up-your-system) - [Preconfigured Images](sections/02-BasicSkills.md#preconfigured-images) - [Take it With You](sections/02-BasicSkills.md#take-it-with-you) - [Kubernetes Container Deployment](sections/02-BasicSkills.md#kubernetes-container-deployment) - [How to Create Start and Stop a Container](sections/02-BasicSkills.md#how-to-create-start-stop-a-container) - [Docker Micro Services](sections/02-BasicSkills.md#docker-micro-services) - [Kubernetes](sections/02-BasicSkills.md#kubernetes) - [Why and How To Do Docker Container Orchestration](sections/02-BasicSkills.md#why-and-how-to-do-docker-container-orchestration) - [Userful Docker Commands](sections/02-BasicSkills.md#useful-docker-commands) - [The Cloud](sections/02-BasicSkills.md#the-cloud) - [IaaS vs PaaS vs SaaS](sections/02-BasicSkills.md#iaas-vs-paas-vs-saas) - [AWS Azure IBM Google IBM](sections/02-BasicSkills.md#aws-azure-ibm-google) - [Cloud vs On-Premises](sections/02-BasicSkills.md#cloud-vs-on-premises) - [Security](sections/02-BasicSkills.md#security) - [Hybrid Clouds](sections/02-BasicSkills.md#hybrid-clouds) - [Security Zone Design](sections/02-BasicSkills.md#security-zone-design) - [How to secure a multi layered application](sections/02-BasicSkills.md#how-to-secure-a-multi-layered-application) - [Cluster security with Kerberos](sections/02-BasicSkills.md#cluster-security-with-kerberos) ## Advanced Engineering Skills - [Data Science Platform](sections/03-AdvancedSkills.md#data-science-platform) - [Why a Good Data Platform Is Important](sections/03-AdvancedSkills.md#why-a-good-data-platform-is-important) - [Big Data vs Data Science and Analytics](sections/03-AdvancedSkills.md#Big-Data-vs-Data-Science-and-Analytics) - [The 4 Vs of Big Data](sections/03-AdvancedSkills.md#the-4-vs-of-big-data) - [Why Big Data](sections/03-AdvancedSkills.md#why-big-data) - [Planning is Everything](sections/03-AdvancedSkills.md#planning-is-everything) - [The Problem with ETL](sections/03-AdvancedSkills.md#the-problem-with-etl) - [Scaling Up](sections/03-AdvancedSkills.md#scaling-up) - [Scaling Out](sections/03-AdvancedSkills.md#scaling-out) - [When not to Do Big Data](sections/03-AdvancedSkills.md#please-dont-go-big-data) - [81 Platform & Pipeline Design Questions](sections/03-AdvancedSkills.md#81-platform-and-pipeline-design-questions) - [Data Source Questions](sections/03-AdvancedSkills.md#data-source-questions) - [Goals and Destination Questions](sections/03-AdvancedSkills.md#goals-and-destination-questions) - [Connect](sections/03-AdvancedSkills.md#connect) - [REST APIs](sections/03-AdvancedSkills.md#rest-apis) - [API Design](sections/03-AdvancedSkills.md#api-design) - [Implemenation Frameworks](sections/03-AdvancedSkills.md#implementation-frameworks) - [Security](sections/03-AdvancedSkills.md#security) - [Apache Nifi](sections/03-AdvancedSkills.md#apache-nifi) - [Logstash](sections/03-AdvancedSkills.md#logstash) - [Buffer](sections/03-AdvancedSkills.md#buffer) - [Apache Kafka](sections/03-AdvancedSkills.md#apache-kafka) - [Why a Message Queue Tool?](sections/03-AdvancedSkills.md#why-a-message-queue-tool) - [Kafka Architecture](sections/03-AdvancedSkills.md#kafka-architecture) - [Kafka Topics](sections/03-AdvancedSkills.md#what-are-topics) - [Kafka and Zookeeper](sections/03-AdvancedSkills.md#what-does-zookeeper-have-to-do-with-kafka) - [How to Produce and Consume Messages](sections/03-AdvancedSkills.md#how-to-produce-and-consume-messages) - [Kafka Commands](sections/03-AdvancedSkills.md#kafka-commands) - [Apache Redis Pub-Sub](sections/03-AdvancedSkills.md#redis-pub-sub) - [AWS Kinesis](sections/03-AdvancedSkills.md#apache-kafka) - [Google Cloud PubSub](sections/03-AdvancedSkills.md#google-cloud-pubsub) - [Processing Frameworks](sections/03-AdvancedSkills.md#processing-frameworks) - [Lambda and Kappa Architecture](sections/03-AdvancedSkills.md#lambda-and-kappa-architecture) - [Batch Processing](sections/03-AdvancedSkills.md#batch-processing) - [Stream Processing](sections/03-AdvancedSkills.md#stream-processing) - [Three Methods of Streaming](sections/03-AdvancedSkills.md#three-methods-of-streaming) - [At Least Once](sections/03-AdvancedSkills.md#at-least-once) - [At Most Once](sections/03-AdvancedSkills.md#at-most-once) - [Exactly Once](sections/03-AdvancedSkills.md#exactly-once) - [Check The Tools](sections/03-AdvancedSkills.md#check-the-tools) - [Should You do Stream or Batch Processing](sections/03-AdvancedSkills.md#should-you-do-stream-or-batch-processing) - [Is ETL still relevant for Analytics?](sections/03-AdvancedSkills.md#is-etl-still-relevant-for-analytics) - [MapReduce](sections/03-AdvancedSkills.md#mapreduce) - [How Does MapReduce Work](sections/03-AdvancedSkills.md#How-does-mapreduce-work) - [MapReduce](sections/03-AdvancedSkills.md#mapreduce) - [MapReduce Example](sections/03-AdvancedSkills.md#example) - [MapReduce Limitations](sections/03-AdvancedSkills.md#What-is-the-limitation-of-mapreduce) - [Apache Spark](sections/03-AdvancedSkills.md#apache-spark) - [What is the Difference to MapReduce?](sections/03-AdvancedSkills.md#what-is-the-difference-to-MapReduce) - [How Spark Fits to Hadoop](sections/03-AdvancedSkills.md#how-does-spark-fit-to-hadoop) - [Spark vs Hadoop](sections/03-AdvancedSkills.md#wheres-the-difference) - [Spark and Hadoop a Perfect Fit](sections/03-AdvancedSkills.md#spark-and-hadoop-is-a-perfect-fit) - [Spark on YARn](sections/03-AdvancedSkills.md#spark-on-yarn) - [My Simple Rule of Thumb](sections/03-AdvancedSkills.md#my-simple-rule-of-thumb) - [Available Languages](sections/03-AdvancedSkills.md#available-languages) - [Spark Driver Executor and SparkContext](sections/03-AdvancedSkills.md#how-spark-works-driver-executor-sparkcontext) - [Spark Batch vs Stream processing](sections/03-AdvancedSkills.md#spark-batch-vs-stream-processing) - [How Spark uses Data From Hadoop](sections/03-AdvancedSkills.md#How-does-spark-use-data-from-hadoop) - [What are RDDs and How to Use Them](sections/03-AdvancedSkills.md#what-are-rdds-and-how-to-use-them) - [SparkSQL How and Why to Use It](sections/03-AdvancedSkills.md#available-languages) - [What are Dataframes and How to Use Them](sections/03-AdvancedSkills.md#what-are-dataframes-how-to-use-them) - [Machine Learning on Spark (TensorFlow)](sections/03-AdvancedSkills.md#machine-learning-on-spark-tensor-flow) - [MLlib](sections/03-AdvancedSkills.md#mllib) - [Spark Setup](sections/03-AdvancedSkills.md#spark-setup) - [Spark Resource Management](sections/03-AdvancedSkills.md#spark-resource-management) - [AWS Lambda](sections/03-AdvancedSkills.md#apache-flink) - [Apache Flink](sections/03-AdvancedSkills.md#apache-flink) - [Elasticsearch](sections/03-AdvancedSkills.md#elasticsearch) - [Apache Drill](sections/03-AdvancedSkills.md#apache-drill) - [StreamSets](sections/03-AdvancedSkills.md#streamsets) - [Store](sections/03-AdvancedSkills.md#store) - [Analytical Data Stores](03-AdvancedSkills.md#analytical-data-stores) - [Data Warehouse vs Data Lake](sections/03-AdvancedSkills.md#data-warehouse-vs-data-lake) - [Snowflake and dbt](sections/03-AdvancedSkills.md#snowflake-and-dbt) - [Transactional Data Stores](sections/03-AdvancedSkills.md#transactional-data-stores) - [SQL Databases](sections/03-AdvancedSkills.md#sql-databases) - [PostgreSQL DB](sections/03-AdvancedSkills.md#postgresql-db) - [Database Design](sections/03-AdvancedSkills.md#database-design) - [SQL Queries](sections/03-AdvancedSkills.md#sql-queries) - [Stored Procedures](sections/03-AdvancedSkills.md#stored-procedures) - [ODBC/JDBC Server Connections](sections/03-AdvancedSkills.md#odbc-jdbc-server-connections) - [NoSQL Stores](sections/03-AdvancedSkills.md#nosql-stores) - [HBase KeyValue Store](sections/03-AdvancedSkills.md#keyvalue-stores-hbase) - [HDFS Document Store](sections/03-AdvancedSkills.md#document-stores-hdfs) - [MongoDB Document Store](sections/03-AdvancedSkills.md#document-stores-mongodb) - [Elasticsearch Document Store](sections/03-AdvancedSkills.md#Elasticsearch-search-engine-and-document-store) - [Hive Warehouse](sections/03-AdvancedSkills.md#hive-warehouse) - [Impala](sections/03-AdvancedSkills.md#impala) - [Kudu](sections/03-AdvancedSkills.md#kudu) - [Apache Druid](sections/03-AdvancedSkills.md#apache-druid) - [InfluxDB Time Series Database](sections/03-AdvancedSkills.md#influxdb-time-series-database) - [Greenplum MPP Database](sections/03-AdvancedSkills.md#mpp-databases-greenplum) - [Visualize](sections/03-AdvancedSkills.md#visualize) - [Android and IOS](sections/03-AdvancedSkills.md#android-and-ios) - [API Design for Mobile Apps](sections/03-AdvancedSkills.md#how-to-design-apis-for-mobile-apps) - [Dashboards](sections/03-AdvancedSkills.md#dashboards) - [Grafana](sections/03-AdvancedSkills.md#grafana) - [Kibana](sections/03-AdvancedSkills.md#kibana) - [Webservers](sections/03-AdvancedSkills.md#how-to-use-webservers-to-display-content) - [Tomcat](sections/03-AdvancedSkills.md#tomcat) - [Jetty](sections/03-AdvancedSkills.md#jetty) - [NodeRED](sections/03-AdvancedSkills.md#nodered) - [React](sections/03-AdvancedSkills.md#react) - [Business Intelligence Tools](sections/03-AdvancedSkills.md#business-intelligence-tools) - [Tableau](sections/03-AdvancedSkills.md#tableau) - [Power BI](sections/03-AdvancedSkills.md#power-bi) - [Quliksense](sections/03-AdvancedSkills.md#quliksense) - [Identity & Device Management](sections/03-AdvancedSkills.md#Identity-and-device-management) - [What Is A Digital Twin](sections/03-AdvancedSkills.md#what-is-a-digital-twin) - [Active Directory](sections/03-AdvancedSkills.md#active-directory) - [Machine Learning](sections/03-AdvancedSkills.md#machine-learning) - [How to do Machine Learning in production](sections/03-AdvancedSkills.md#how-to-domachine-learning-in-production) - [Why machine learning in production is harder then you think](sections/03-AdvancedSkills.md#why-machine-learning-in-production-is-harder-then-you-think) - [Models Do Not Work Forever](sections/03-AdvancedSkills.md#models-do-not-work-forever) - [Where are The Platforms That Support Machine Learning](sections/03-AdvancedSkills.md#where-are-the-platforms-that-support-this) - [Training Parameter Management](sections/03-AdvancedSkills.md#training-parameter-management) - [How to Convince People That Machine Learning Works](sections/03-AdvancedSkills.md#how-to-convince-people-machine-learning-works) - [No Rules No Physical Models](sections/03-AdvancedSkills.md#no-rules-no-physical-models) - [You Have The Data. Use It!](sections/03-AdvancedSkills.md#you-have-the-data-use-it) - [Data is Stronger Than Opinions](sections/03-AdvancedSkills.md#data-is-stronger-than-opinions) - [AWS Sagemaker](sections/03-AdvancedSkills.md#aws-sagemaker) ## Hands On Course - [Free Data Engineering Course with AWS, TDengine, Docker and Grafana](sections/04-HandsOnCourse.md#free-data-engineering-course-with-aws-tdengine-docker-and-grafana) - [Monitor your data in dbt & detect quality issues with Elementary](sections/04-HandsOnCourse.md#monitor-your-data-in-dbt-and-detect-quality-issues-with-elementary) - [Solving Engineers 4 Biggest Airflow Problems](sections/04-HandsOnCourse.md#solving-engineers-4-biggest-airflow-problems) - [The best alternative to Airlfow? Mage.ai](sections/04-HandsOnCourse.md#the-best-alternative-to-airlfow?-mage.ai) ## Case Studies - [Data Science @Airbnb](sections/05-CaseStudies.md#data-science-at-Airbnb) - [Data Science @Amazon](sections/05-CaseStudies.md#data-science-at-Amazon) - [Data Science @Baidu](sections/05-CaseStudies.md#data-science-at-Baidu) - [Data Science @Blackrock](sections/05-CaseStudies.md#data-science-at-Blackrock) - [Data Science @BMW](sections/05-CaseStudies.md#data-science-at-BMW) - [Data Science @Booking.com](sections/05-CaseStudies.md#data-science-at-Booking.com) - [Data Science @CERN](sections/05-CaseStudies.md#data-science-at-CERN) - [Data Science @Disney](sections/05-CaseStudies.md#data-science-at-Disney) - [Data Science @DLR](sections/05-CaseStudies.md#data-science-at-DLR) - [Data Science @Drivetribe](sections/05-CaseStudies.md#data-science-at-Drivetribe) - [Data Science @Dropbox](sections/05-CaseStudies.md#data-science-at-Dropbox) - [Data Science @Ebay](sections/05-CaseStudies.md#data-science-at-Ebay) - [Data Science @Expedia](sections/05-CaseStudies.md#data-science-at-Expedia) - [Data Science @Facebook](sections/05-CaseStudies.md#data-science-at-Facebook) - [Data Science @Google](sections/05-CaseStudies.md#data-science-at-Google) - [Data Science @Grammarly](sections/05-CaseStudies.md#data-science-at-Grammarly) - [Data Science @ING Fraud](sections/05-CaseStudies.md#data-science-at-ING-Fraud) - [Data Science @Instagram](sections/05-CaseStudies.md#data-science-at-Instagram) - [Data Science @LinkedIn](sections/05-CaseStudies.md#data-science-at-LinkedIn) - [Data Science @Lyft](sections/05-CaseStudies.md#data-science-at-Lyft) - [Data Science @NASA](sections/05-CaseStudies.md#data-science-at-NASA) - [Data Science @Netflix](sections/05-CaseStudies.md#data-science-at-Netflix) - [Data Science @OLX](sections/05-CaseStudies.md#data-science-at-OLX) - [Data Science @OTTO](sections/05-CaseStudies.md#data-science-at-OTTO) - [Data Science @Paypal](sections/05-CaseStudies.md#data-science-at-Paypal) - [Data Science @Pinterest](sections/05-CaseStudies.md#data-science-at-Pinterest) - [Data Science @Salesforce](sections/05-CaseStudies.md#data-science-at-Salesforce) - [Data Science @Siemens Mindsphere](sections/05-CaseStudies.md#data-science-at-Siemens-Mindsphere) - [Data Science @Slack](sections/05-CaseStudies.md#data-science-at-Slack) - [Data Science @Spotify](sections/05-CaseStudies.md#data-science-at-Spotify) - [Data Science @Symantec](sections/05-CaseStudies.md#data-science-at-Symantec) - [Data Science @Tinder](sections/05-CaseStudies.md#data-science-at-Tinder) - [Data Science @Twitter](sections/05-CaseStudies.md#data-science-at-Twitter) - [Data Science @Uber](sections/05-CaseStudies.md#data-science-at-Uber) - [Data Science @Upwork](sections/05-CaseStudies.md#data-science-at-Upwork) - [Data Science @Woot](sections/05-CaseStudies.md#data-science-at-Woot) - [Data Science @Zalando](sections/05-CaseStudies.md#data-science-at-Zalando) ## Best Practices Cloud Platforms - [Amazon Web Services (AWS)](sections/06-BestPracticesCloud.md#aws) - [Connect](sections/06-BestPracticesCloud.md#Connect) - [Buffer](sections/06-BestPracticesCloud.md#Buffer) - [Processing](sections/06-BestPracticesCloud.md#Processing) - [Store](sections/06-BestPracticesCloud.md#Store) - [Visualize](sections/06-BestPracticesCloud.md#Visualize) - [Containerization](sections/06-BestPracticesCloud.md#Containerization) - [Best Practices](sections/06-BestPracticesCloud.md#Best-Practices) - [More Details](sections/06-BestPracticesCloud.md#More-Details) - [Microsoft Azure](sections/06-BestPracticesCloud.md#azure) - [Connect](sections/06-BestPracticesCloud.md#Connect-1) - [Buffer](sections/06-BestPracticesCloud.md#Buffer-1) - [Processing](sections/06-BestPracticesCloud.md#Processing-1) - [Store](sections/06-BestPracticesCloud.md#Store-1) - [Visualize](sections/06-BestPracticesCloud.md#Visualize-1) - [Containerization](sections/06-BestPracticesCloud.md#Containerization-1) - [Best Practices](sections/06-BestPracticesCloud.md#Best-Practices-1) - [Google Cloud Platform (GCP)](sections/06-BestPracticesCloud.md#gcp) - [Connect](sections/06-BestPracticesCloud.md#Connect-2) - [Buffer](sections/06-BestPracticesCloud.md#Buffer-2) - [Processing](sections/06-BestPracticesCloud.md#Processing-2) - [Store](sections/06-BestPracticesCloud.md#Store-2) - [Visualize](sections/06-BestPracticesCloud.md#Visualize-2) - [Containerization](sections/06-BestPracticesCloud.md#Containerization-2) - [Best Practices](sections/06-BestPracticesCloud.md#Best-Practices-2) ## 130+ Data Sources for Portfolio Projects - [Student Favorites](sections/07-DataSources.md#Student-Favorites) - [General And Academic](sections/07-DataSources.md#General-And-Academic) - [Content Marketing](sections/07-DataSources.md#Content-Marketing) - [Crime](sections/07-DataSources.md#Crime) - [Drugs](sections/07-DataSources.md#Drugs) - [Education](sections/07-DataSources.md#Education) - [Entertainment](sections/07-DataSources.md#Entertainment) - [Environmental And Weather Data](sections/07-DataSources.md#Environmental-And-Weather-Data) - [Financial And Economic Data](sections/07-DataSources.md#Financial-And-Economic-Data]) - [Government And World](sections/07-DataSources.md#Government-And-World) - [Health](sections/07-DataSources.md#Health) - [Human Rights](sections/07-DataSources.md#Human-Rights) - [Labor And Employment Data](sections/07-DataSources.md#Labor-And-Employment-Data) - [Politics](sections/07-DataSources.md#Politics) - [Retail](sections/07-DataSources.md#Retail) - [Social](sections/07-DataSources.md#Social) - [Travel And Transportation](sections/07-DataSources.md#Travel-And-Transportation) - [Various Portals](sections/07-DataSources.md#Various-Portals) - [Source Articles and Blog Posts](sections/07-DataSources.md#Source-Articles-and-Blog-Posts) - [Free Data Sources Data Science](sections/07-DataSources.md) ## 1001 Interview Questions - [Interview Questions](sections/08-InterviewQuestions.md) ## Recommended Books, Courses, and Podcasts - [About Books and Courses](sections/09-BooksAndCourses.md#about-books-and-courses) - [Books](sections/09-BooksAndCourses.md#books) - [Languages](sections/09-BooksAndCourses.md#books-languages) - [Data Tools & Platforms](sections/09-BooksAndCourses.md#books-data-science-tools) - [Business](sections/09-BooksAndCourses.md#Books-Business) - [Community Recommendations](sections/09-BooksAndCourses.md#Community-Recommendations) - [Online Courses](sections/09-BooksAndCourses.md#online-courses) - [Preparation courses](sections/09-BooksAndCourses.md#Preparation-courses) - [Data engineering courses](sections/09-BooksAndCourses.md#Data-engineering-courses) - [Certifications](sections/09-BooksAndCourses.md#Certifications) - [Podcasts](sections/09-BooksAndCourses.md#Podcasts) - [Super Data Science](sections/09-BooksAndCourses.md#Super-Data-Science) - [Data Skeptic](sections/09-BooksAndCourses.md#Data-Skeptic) - [Data Engineering Podcast](sections/09-BooksAndCourses.md#Data-Engineering-Podcast) - [Roaring Elephant BiteSized Big Tech](sections/09-BooksAndCourses.md#Roaring-Elephant-BiteSized-Big-Tech) - [SQL Data Partners Podcast](sections/09-BooksAndCourses.md#SQL-Data-Partners-Podcast) ## How To Contribute If you have some cool links or topics for the cookbook, please become a contributor. Simply pull the repo, add your ideas and create a pull request. You can also open an issue and put your thoughts there. Please use the "Issues" function for comments. ## Important Links Subscribe to my YouTube channel for regular updates: [Link to YouTube](https://www.youtube.com/channel/UCY8mzqqGwl5_bTpBY9qLMAA) I have a Medium publication where you can publish your data engineer articles to reach more people: [Medium publication](https://link.medium.com/9oi1VDrhPW) <br> *(As an Amazon Associate I earn from qualifying purchases from Amazon This is free of charge for you, but super helpful for supporting this channel)
