KO | EN

GitLite

Minimalist Open Source Repository Search Engine

gitlite — search

#dsh #dsh-plugin #ai-agents #developer-tools #ai-agent #claude-code #mcp #typescript #codex #cli #rust #deepseek-harness

ai_summer

★ 69 Open GitHub ↗

Summary repository for AI Summer 2024

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/vanderbilt-data-science/ai_summer)

# AI Summer
> Summary repository for AI Summer 2024. Introduction to generative AI, with practical applications to inferencing and training

Presented by Vanderbilt Data Science Institute data scientists:
* Dr. Jesse Spencer-Smith, Chief Data Scientist
* Dr. Charreau Bell, Senior Data Scientist
* Myranda Shirk, Senior Data Scientist
* Umang Chaudhry, Data Scientist
* Dr. Abigail Petulante, DSI Postdoctoral Fellow
* Dr. Joshua Su, DSI Postdoctoral Fellow

## Overview
The objective of these workshops is to develop foundational skills in understanding, inferencing and training generative AI models and other transformer models.

## Getting Ready for AI Summer

### Learn or Improve Your Python

Practice your Python skills using the below documents. Choose either a Google Colab for interactive programming environment, or alternatively read through the Google Doc.

- [Colab Notebook Format](https://colab.research.google.com/drive/1PRpyEJQyQc-h3mdKuY2awCEhsHcP8AZ0?usp=sharing)

- [Google Doc Format](https://docs.google.com/document/d/1CUjOkKSZ-YY4R_562IoOQxTZvDZ2MGn-w_lURC8acBo/edit?usp=sharing)

### Getting Accounts

You’ll want to use the most advanced AI chat model that you can get access to. Microsoft just opened access to BingGPT through Bing Chat, which is based on an early version of GPT4, currently the most advanced AI chat model available to the public. You’ll need to install the Edge browser (https://www.microsoft.com › edge › download) and go to bing. com. Click on “Chat”.

### Decide on a Project (recommended)

Think about any data you might want to bring to the workshop. Also begin thinking about any projects you might want to accomplish during our month. We’ll have office hours for you to work with us to get your first project off the ground!

## Workshop Schedule

Session will run live from 9am-11am, with an office hour from 11am to noon (all times Central).

### Week 1, 5/6 - 5/8: Introduction to Generative AI Models, Transformers, and Prompt Engineering

Monday: https://vanderbilt.zoom.us/rec/share/17V0GyZJoD9PPmenE2stE9AAPGgT0x8KQcEis-VHMn4n8GdsUrjbmu61Tb8r4iv3.FZdNC_03OR0tiDSI?startTime=1715004136000

Wednesday: https://vanderbilt.zoom.us/rec/share/jmUNeAlIF-vBLDWmFuV_HWlI8c944oM3YB94Q_9H36pgKtwTvQlzQAyGASEM3gaw.OKGh4pXYucHTLKAv

No class Friday (Vanderbilt Commencement)

### Weeks 2, 5/13 - 5/17: Retrieval-Augmented Generation (RAG), Assistants, Agents, and Intro to Diffusion Models

Monday Part 1: https://vanderbilt.zoom.us/rec/share/DFJ_bDybROtlo0qUseTFRo8QvntugqIsAlSVud1WKoEVlnO17ksxpz7uPgGgYN1r.1o_oPGd_07POUKux?startTime=1715608503000

Monday Part 2: https://vanderbilt.zoom.us/rec/share/DFJ_bDybROtlo0qUseTFRo8QvntugqIsAlSVud1WKoEVlnO17ksxpz7uPgGgYN1r.1o_oPGd_07POUKux?startTime=1715613842000

Wednesday: https://vanderbilt.zoom.us/rec/share/D_asoLsFb3tHZ4AB6qHaODUJvnwNyGQA6bQsG4pSMqvI1zQ-cQkNXsxcuPi1wqYv.vcKQy17Jc8Y0xGbM?startTime=1715781305000

Friday: https://vanderbilt.zoom.us/rec/share/y8paBkSmKLcUJRnHtwo8ZYSmNWCCG8nX9mtxfaOWBq_HbGSdMyq_VLi_E6wQZG2l.09Mt6t9VgpsR_ruj?startTime=1715954101000

### Week 3, 5/20 - 5/24: Building AI Solutions, Running AI Securely Locally or in the Cloud, Introduction to Training Models

Monday:
- Recording: https://vanderbilt.zoom.us/rec/share/LGj-kG8kW22LzCacO6P2GdAtHRL0Tq3cn0O2NV36TlNQzeQKHDuZJkVqHraKcqaR.ScCzM5Gqp82li_6q
  
- Homework: Watch the following videos: [General Backprop](https://youtu.be/Ilg3gGewQ5U?si=kyrsyNj-DghEWnxU) and
(math-centric backprop](https://youtu.be/tIeHLnjs5U8?si=mnT36GTL7YqU8qBO)

Wednesday: Recording: (https://vanderbilt.zoom.us/rec/share/fswTlpFMlqAVgxRDDBza920i9brAuxaSiteHpDNUwpm9YQzedJa5g_2oZSSr2Eq1.wF73yKYGD5eY3cyY?startTime=1716392393000)

Friday: Recording: (https://vanderbilt.zoom.us/rec/share/plozihJcLFBIfjPxQ8Bsv9IdqHh39qFinkVUChsYtuiuiGAc8O2TcvTEbTE5cAUW.3XYBPJfbdZJ1GzAS?startTime=1716558902000)

### Week 4, 5/29 - 5/31: Fine-tuning Models, Mulit-modal Models, Applied AI

No class Monday (Memorial Day)

Wednesday:

Papers/Blogs discussed:

https://arxiv.org/pdf/2405.17247

https://proceedings.mlr.press/v139/radford21a/radford21a.pdf

https://openaccess.thecvf.com/content/CVPR2022/papers/Singh_FLAVA_A_Foundational_Language_and_Vision_Alignment_Model_CVPR_2022_paper.pdf

https://arxiv.org/pdf/2405.09818

https://sh-tsang.medium.com/review-flamingo-a-visual-language-model-for-few-shot-learning-ec477d47e7bf

https://arxiv.org/pdf/2304.10592

https://arxiv.org/pdf/2310.03744

https://huggingface.co/papers/2311.05437

https://arxiv.org/pdf/2311.05437

https://llava-vl.github.io/blog/2024-01-30-llava-next/

https://llava-vl.github.io/blog/2024-05-10-llava-next-stronger-llms/

https://llava-vl.github.io/blog/2024-04-30-llava-next-video/

https://arxiv.org/abs/2310.02239

https://wandb.ai/byyoung3/ml-news/reports/How-to-Fine-Tune-LLaVA-on-a-Custom-Dataset--Vmlldzo2NjUwNTc1

Recording: https://vanderbilt.zoom.us/rec/share/YV3qbB-MzAuBeF3dnBnaZdif3LjD1uaObgw9Yg3OWEb_UYwLi0befTYMz57MWKVo.OrWLyaDkkCugELz8?startTime=1716990223000

Friday: Recording https://vanderbilt.zoom.us/rec/share/g2Q9l9ZAxfjPuPKSY9W9W9aZF_oq7a9rTcAQk2ppNzDTsZBCnpWdN3Dktd2THXY.G5FUcMWQPRFaenje

## Breakout Rooms

### How to Breakout

Remember we are all learning and exploring
- Please share your video upon entering the room and unmute
- Share your screens--someone volunteer to share their screen upon entering, and everyone be ready to share your screen to show what you’ve found
- Make notes of what you’ve discussed in the Response Reports below
- Everyone be ready to report out (random)
- Make some friends
- Breakout Rooms Worksheets

### Report Documents
Google Docs has a limit of 100 people viewing/editing a document at one time.

### Special Breakout Room Groups

Please be sure your display name is set in Zoom. If you are in one of the following special groups, please pre-pend your name with one of the following qualifiers. 
- Data Science for Social Good: DSSG
- Center for AI in Protein Dynamics: Protein
- If you are in a lab and would like your own breakout room: Labname (keep it short, please!)
- If you are faculty and would like to be in a breakout room with other faculty: Faculty

For example, I might be *DSSG-Jesse Spencer-Smith*

## Workshop Video Recordings
Video recordings of these workshops can be found on our YouTube channel [AI Summer playlist](https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.youtube.com%2Fplaylist%3Flist%3DPL6KxUvysa-7yV8T4qcoLaDH3ZzUFwYx8u&data=05%7C01%7Cjesse.spencer-smith%40vanderbilt.edu%7C05c7b9c0dbf942c3186408db558cf221%7Cba5a7f39e3be4ab3b45067fa80faecad%7C0%7C0%7C638197836249753010%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=u5Q84sUPC7K1VOB9hyspSqduBhHBd8SvGXIXlNkjACM%3D&reserved=0)

Looking for the code resources for Summer 2023? View the 2023 repo version [here.](https://github.com/vanderbilt-data-science/ai_summer/tree/v2023)

## Course Resources

- Prompt Engineering paper https://arxiv.org/abs/2302.11382
- Prompt Engineering Courserea Course: https://www.coursera.org/learn/prompt-engineering
- Visual overview of Generative AI from 3Blue1Brown: https://www.youtube.com/watch?v=wjZofJX0v4M 
- Semester-long course on transformer models, DS 5690. Graduate students and advanced undergraduates can register by contacting me. I welcome auditing by a select number of postdoctoral fellows, and drop-ins from faculty!

## Other Resources

### Compute Grants for Vanderbilt Faculty and Students

DGX A100 Compute Grant: https://forms.gle/2mGfEy9DB4JU2GpZ8

### Python

### Transformers
-  Natural Language Processing with Transformers by Lewis Tunstall, Leandro von Werra and Thomas Wolf. If you are affiliated with Vanderbilt University, you can access this pre-print book (and any book by O’Reilly) free by logging into O'Reilly Media using your Vanderbilt email address. Vanderbilt licenses all content from O’Reilly. The book covers Transformers for purposes beyond text.

### Getting the Most out of this Course
To get the most out of this workshop:
* Open Colab (workbook) notebooks and actively write code along with the instructor
* Actively participate in discussions
* Actively participate in breakout rooms
* Work on homework assignments before coming to class
* Relax your mind and ask questions

### Accessing Bing Chat
1. Open the Edge browser (yes, Edge) and navigate to www.bing.com
2. Select "chat". A new window should open saying you need the new Bing.
3. Select "Start chatting" at the bottom of this window. This should prompt you to sign in to a Microsoft account. **Do not use an organizational/school email** (such as Vanderbilt). Instead, select "No account? Create a new one" and create one with your **personal** email. 
  Note: if you get stuck in the "use the new Bing" window, go back to Bing.com and select "Sign in" instead. Follow instructions for Step 3.
  
  ![](https://github.com/vanderbilt-data-science/ai_summer/blob/main/img/Bing-chat.png)
  ![](https://github.com/vanderbilt-data-science/ai_summer/blob/main/img/bing-1.png)

5. Sign in to your personal Microsoft account.
6. Select "chat" - it should work now!
  ![](https://github.com/vanderbilt-data-science/ai_summer/blob/main/img/bing-2.png)

ai_summer

Explore Similar Repositories

Was this content helpful?

Recent Feedback

Download README