Agile_Data_Code_2
Code for Agile Data Science 2.0, O'Reilly 2017, Second Edition
File Explorer
Download Latest Version (.zip)- block.device.mappings.json
- ec2_bootstrap.sh
- download_list.txt
- get_files_from_ec2.sh
- get_student_work.sh
- send_files_to_ec2.sh
- start_notebook.sh
- stop_flask.sh
- stop_notebook.sh
- upgrade.sh
- test.jsonl
- ads2_0201.png
- ads2_0202.png
- ads2_0209.png
- ads2_0211.png
- ads2_0212.png
- ads2_0215.png
- ads2_0217.png
- ads2_0219.png
- ads2_0220.png
- ads2_0401.png
- ads2_0402.png
- ads2_0403.png
- ads2_0405.png
- ads2_0406.png
- ads2_0408.png
- ads_bootstrap.png
- faa_table.png
- flask_terminal.png
- json.png
- bootstrap-theme.min.css
- bootstrap.min.css
- bootstrap.min.js
- d3.v3.min.js
- table.html
- flask_pymongo.py
- test_flask.py
- test_flask_bootstrap.py
- test_flask_pymongo.py
- Agile_Tools.ipynb
- airflow_test.py
- elasticsearch.sh
- flatmap.py
- groupby.py
- histogram.py
- Introduction_to_PySpark.ipynb
- load_on_time_performance.py
- mongo.js
- pyspark_elasticsearch.py
- pyspark_mongodb.py
- pyspark_streaming.py
- pyspark_task_one.py
- pyspark_task_two.py
- python_kafka.py
- setup_airflow_test.sh
- spark.py
- sql.py
- test_elasticsearch.py
- test_elasticsearch.sh
- test_json.py
- test_pymongo.py
- test_pymongo_2.py
- ads2_0201.png
- ads2_0202.png
- ads2_0209.png
- ads2_0211.png
- ads2_0212.png
- ads2_0215.png
- ads2_0217.png
- ads2_0219.png
- ads2_0220.png
- ads2_0401.png
- ads2_0402.png
- ads2_0403.png
- ads2_0405.png
- ads2_0406.png
- ads2_0408.png
- ads2_0409.png
- ads_bootstrap.png
- ags2_0402.png
- airline_data_fields.png
- faa_table.png
- flask_terminal.png
- json.png
- parquet_logo.jpg
- row_format_column_format.png
- bootstrap-theme.min.css
- bootstrap.min.css
- bootstrap.min.js
- d3.v3.min.js
- flight.html
- flights.html
- layout.html
- macros.jnj
- search.html
- config.py
- on_time_flask.py
- on_time_flask_template.py
- Collecting_and_Displaying_Records.ipynb
- convert_data.py
- download.sh
- load_on_time_pyspark.py
- mongo.js
- pyspark_to_elasticsearch.py
- pyspark_to_mongo.py
- ads2_0501.png
- ads2_0502.png
- ads2_0503.png
- ads2_0504.png
- ads2_0505.png
- ads2_0507.png
- ads2_0508.png
- ads2_0509.png
- ads2_0510.png
- ads2_0511.png
- ads2_0512.png
- first_order_form.png
- flight_search_with_tail_num_link.png
- mapreduce.png
- total_flights_2.png
- .exist
- app.js
- app2.js
- app3.js
- bootstrap-theme.min.css
- bootstrap.min.css
- bootstrap.min.js
- d3.v3.min.js
- jquery-1.12.2.min.js
- nv.d3.css
- nv.d3.min.js
- flight.html
- flights.html
- flights_per_airplane.html
- flights_per_airplane_2.html
- layout.html
- macros.jnj
- search.html
- top_routes.html
- top_routes_chart.html
- total_flights.html
- total_flights_chart.html
- total_flights_chart_2.html
- chart_flask.py
- config.py
- flights_per_airplane.html
- assess_airplanes.py
- assess_faa.py
- extract_airplanes.py
- install.sh
- mongo.js
- save_tail_numbers.py
- total_flights.py
- Visualizing_Data_with_Charts_and_Tables.ipynb
- ads2_0601.png
- ads2_0602.png
- ads2_0603.png
- ads2_0604.png
- ads2_0605.png
- ads2_0606.png
- ads2_0607.png
- ads2_0608.png
- ads2_0609.png
- airplanes.js
- app.js
- bootstrap-theme.min.css
- bootstrap.min.css
- bootstrap.min.js
- d3.v3.min.js
- jquery-1.12.2.min.js
- nv.d3.css
- nv.d3.min.js
- airlines.html
- airlines2.html
- airport.html
- all_airlines.html
- all_airplanes.html
- flight.html
- flights.html
- flights_per_airplane.html
- layout.html
- macros.jnj
- search.html
- total_flights.html
- total_flights_chart.html
- __init__.py
- config.py
- report_flask.py
- search_helpers.py
- add_name_to_airlines.py
- airplanes_mapping.json
- airplanes_to_elasticsearch.py
- analyze_airplanes.py
- analyze_airplanes_again.py
- create_airplanes_index.sh
- enrich_airlines_wikipedia.py
- Exploring_Data_with_Reports.ipynb
- extract_airlines.py
- extract_airports.py
- import_airlines.sh
- prepare_airplanes.py
- resolve_airplane_manufacturers.py
- scrape_faa.py
- test_elastic_airplanes.sh
- ads2_0701.png
- ads2_0702.png
- explore_delays.py
- extract_features.py
- Making_Predictions.ipynb
- Predicting flight delays with sklearn.ipynb
- train_sklearn_model.py
- train_spark_mllib_model.py
- setup.py
- ads2_0807.png
- ads2_0808.png
- ads2_0809.png
- ads2_0810.png
- ads2_0811.png
- airplanes.js
- app.js
- bar.css
- barchart.js
- bootstrap-theme.min.css
- bootstrap.min.css
- bootstrap.min.js
- d3.v3.min.js
- flight_delay_predict_polling.js
- jquery-1.12.2.min.js
- nv.d3.css
- nv.d3.min.js
- airlines.html
- all_airlines.html
- all_airplanes.html
- delays.html
- flight.html
- flight_delays_predict.html
- flight_delays_predict_batch.html
- flight_delays_predict_batch_results.html
- flight_delays_predict_kafka.html
- flights.html
- flights_per_airplane.html
- layout.html
- macros.jnj
- search.html
- total_flights.html
- total_flights_chart.html
- __init__.py
- config.py
- predict_flask.py
- predict_utils.py
- Deploying_Predictive_Systems.ipynb
- download_data.sh
- extract_features.py
- fetch_prediction_requests.py
- import_distances.sh
- kafka_test.py
- links.txt
- load_prediction_results.py
- make_predictions.py
- make_predictions_streaming.py
- origin_dest_distances.py
- python_kafka_consumer.py
- python_kafka_producer.py
- streaming_test.py
- test_airflow.sh
- test_classification_api.sh
- test_regression_api.sh
- train_spark_mllib_model.py
- baseline_spark_mllib_model.py
- Debugging Prediction Problems.ipynb
- explore_delays.py
- extract_features.py
- extract_features_with_airplanes.py
- extract_features_with_flight_time.py
- improve_sklearn_model.py
- improved_spark_mllib_model.py
- Improving flight delay predictions with sklearn.ipynb
- Improving_Predictions.ipynb
- make_predictions_final.py
- make_predictions_streaming_final.py
- spark_model_with_airplanes.py
- spark_model_with_flight_time.py
- train_spark_mllib_model.py
- airplanes.js
- app.js
- bar.css
- barchart.js
- bootstrap-theme.min.css
- bootstrap.min.css
- bootstrap.min.js
- calendar.js
- calendar.min.css
- d3.v3.min.js
- flight_delay_predict_polling.js
- jquery-1.12.2.min.js
- nv.d3.css
- nv.d3.min.js
- airlines.html
- all_airlines.html
- all_airplanes.html
- daily_weather_station.html
- delays.html
- flight.html
- flight_delays_predict.html
- flight_delays_predict_batch.html
- flight_delays_predict_batch_results.html
- flight_delays_predict_kafka.html
- flights.html
- flights_per_airplane.html
- layout.html
- macros.jnj
- search.html
- total_flights.html
- total_flights_chart.html
- weather_station.html
- __init__.py
- config.py
- predict_flask.py
- predict_utils.py
- convert_observations.py
- explore_weather.py
- load_weather.py
- match_airport_with_weather_station.py
- match_reports_with_flights.py
- spark_model_with_weather.py
- .exists
- create.sh
- drop.sh
- query.sh
- airline_page_enriched_wikipedia.png
- airplanes_page_chart_v1_v2.png
- back_end_realtime_architecture.png
- climbing_the_pyramid_chapter_intro.png
- data_syndrome_logo.png
- DeepDiscoveryTechnicalLogo.png
- flight_delay_chart_2.0.png
- front_end_realtime_architecture.png
- predicting_flight_kafka_waiting.png
- ubuntu_images.png
- video_course_cover.png
- phantomjs.sh
- example.csv
- faa_tail_number_inquiry.jsonl
- pyspark_csv.py
- setup_spark.py
- utils.py
- .exists
- .exists
- .bashrc
- .dockerignore
- .gitignore
- airflow.env
- docker-compose.yml
- Dockerfile
- download.sh
- download_weather.sh
- intro_download.sh
- jupyter_notebook_config.py
- LICENSE
- old.Dockerfile
- Part_II_-_Climbing_the_Pyramid.ipynb
- poetry.lock
- pyproject.toml
- README.md
- requirements.txt
- Welcome.ipynb
# Installation Guide
1. Get the code
git clone https://github.com/rjurney/Agile_Data_Code_2
Downloads the entire project code from GitHub to your computer.
cd Agile_Data_Code_2
Moves into the project folder you just downloaded.
2. Docker
Easy RecommendedPrerequisites
- Git Needed to download the project code from GitHub.
- Docker Desktop Needed to build and run containers. Install it and keep it running in the background.
docker-compose build agile
Runs the command against the services defined in the compose file.
docker-compose up -d
Runs the command against the services defined in the compose file.
Run docker compose ps to check the containers are Up. If the README mentions a port, open http://localhost:PORT in your browser.
Pulled directly from this repo's README.
3. Python
EasyPrerequisites
pip install -r requirements.txt
Installs the Python libraries listed in requirements.txt (or similar).
jupyter notebook
Launches Jupyter in your browser so you can open and run the notebook (.ipynb) files.
If it runs without errors and prints output in the terminal, it worked.
// repository documentation
Was this content helpful?
(0 ratings)