Trilha-de-Estudos-de-Data-Science

KO | EN

gitlite — search

#javascript #hacktoberfest #golang #android #deep-learning #linux #docker #java #windows #awesome #kubernetes #awesome-list #openai #open-source

Trilha-de-Estudos-de-Data-Science

(★ 72)

Técnicas e recursos para estudar ciência de dados.

Trilha-de-Estudos-de-Data-Science 최신버젼 다운로드

최종 버전 다운로드 (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# Trilha de Estudos de Ciência de Dados

Este é um repositório em constante construção e atualização. Adiciono aqui técnicas de estudo e fontes que considero boas para o aprendizado de ciência de dados, com o objetivo de manter recursos organizados para consulta e ajudar quem se interessa pelo tema. O conteúdo aqui compilado vai do básico ao avançado.

Todas as fontes aqui referenciadas foram vistas e curadas por mim, e este repositório também me serve como fácil retorno a bons conteúdos.

# Sumário

- [Como estudar](https://github.com/HinePo/Trilha-de-Estudos-de-Data-Science/blob/main/README.md#como-estudar)
- [Coding with AI](https://github.com/HinePo/Trilha-de-Estudos-de-Data-Science/blob/main/README.md#coding-with-ai)
- [Python e Git](https://github.com/HinePo/Trilha-de-Estudos-de-Data-Science/blob/main/README.md#python-e-git)
- [Data Analysis basics](https://github.com/HinePo/Trilha-de-Estudos-de-Data-Science/blob/main/README.md#data-analysis-basics)
- [Data Engineering](https://github.com/HinePo/Trilha-de-Estudos-de-Data-Science/blob/main/README.md#data-engineering)
- [Data Visualization](https://github.com/HinePo/Trilha-de-Estudos-de-Data-Science/blob/main/README.md#data-visualization)
- [Machine Learning - Teoria](https://github.com/HinePo/Trilha-de-Estudos-de-Data-Science/blob/main/README.md#machine-learning---teoria)
- [Machine Learning - Prática](https://github.com/HinePo/Trilha-de-Estudos-de-Data-Science/blob/main/README.md#machine-learning---prática)
- [Time Series](https://github.com/HinePo/Trilha-de-Estudos-de-Data-Science/blob/main/README.md#time-series)
- [Deep Learning - Neural Networks](https://github.com/HinePo/Trilha-de-Estudos-de-Data-Science/blob/main/README.md#deep-learning---neural-networks)
- [Transformers](https://github.com/HinePo/Trilha-de-Estudos-de-Data-Science/blob/main/README.md#transformers)
- [NLP - Natural Language Processing](https://github.com/HinePo/Trilha-de-Estudos-de-Data-Science/blob/main/README.md#nlp---natural-language-processing)
- [LLMs](https://github.com/HinePo/Trilha-de-Estudos-de-Data-Science/blob/main/README.md#llms)
- [Computer Vision](https://github.com/HinePo/Trilha-de-Estudos-de-Data-Science/blob/main/README.md#computer-vision)
- [RecSys](https://github.com/HinePo/Trilha-de-Estudos-de-Data-Science/blob/main/README.md#recsys)
- [Marketing Mix Modeling](https://github.com/HinePo/Trilha-de-Estudos-de-Data-Science/blob/main/README.md#marketing-mix-modeling)
- [Survival Analysis](https://github.com/HinePo/Trilha-de-Estudos-de-Data-Science/blob/main/README.md#survival-analysis)
- [Deploy](https://github.com/HinePo/Trilha-de-Estudos-de-Data-Science/blob/main/README.md#deploy)
- [MLOps](https://github.com/HinePo/Trilha-de-Estudos-de-Data-Science/blob/main/README.md#mlops)
- [Tech Management](https://github.com/HinePo/Trilha-de-Estudos-de-Data-Science/blob/main/README.md#tech-management)
- [Vibe Coding](https://github.com/HinePo/Trilha-de-Estudos-de-Data-Science/blob/main/README.md#vibe-coding)
- [Youtube channels](https://github.com/HinePo/Trilha-de-Estudos-de-Data-Science/blob/main/README.md#youtube-channels)
- [Perfis no twitter](https://github.com/HinePo/Trilha-de-Estudos-de-Data-Science/blob/main/README.md#perfis-no-twitter)

# Como estudar

- [AI with Karpathy](https://www.youtube.com/watch?v=tsTeEkzO9xc) - vídeo: minuto 3 ao 23 - Karpathy
- [Never memorize code](https://www.youtube.com/watch?app=desktop&v=AavXBoxTCIA)
- [3 Tips to Build a Career in Machine Learning](https://www.youtube.com/watch?v=plXM4pNSYXs)
- [How to learn data science smartly](https://www.youtube.com/watch?app=desktop&v=csG_qfOTvxw)
- Criar um documento (Word, Notion, Evernote etc) pessoal com a sua organização do que vc já aprendeu/estudou e o que planeja aprender/estudar, de preferência organizado por mês ou bimestre. Procurar manter este doc atualizado
- Instalar a extensão [video speed controller](https://chrome.google.com/webstore/detail/video-speed-controller/nffaoalbilbmmfgbnbgppjihopabppdk) no google chrome (funciona em qualquer vídeo tocado pelo chrome browser), e aprender a usar
- Sempre estude do geral para o específico: [top-down learning](https://en.wikipedia.org/wiki/Top-down_and_bottom-up_design)
- Para cada dose de teoria, uma dose de prática: [problem-based learning](https://en.wikipedia.org/wiki/Problem-based_learning)
-	Ao entrar em um assunto novo, ver um ou dois vídeos de ~10 min no youtube, pesquisar sobre o tema focando em material escrito, e estudar aplicações
-	Evitar ficar muito tempo na parte teórica: Qualquer assunto novo tem suas aplicações via bibliotecas específicas. Se familiarizar com a documentação é o primeiro passo para aplicar o que aprendeu
-	O segundo passo é a aplicação e uso, parte prática: Resolver problemas usando IA: Pesquisar aplicações no Kaggle (notebooks), fazer o fork (Copy and Edit), adicionar ideias.
-	Evitar tentar reinventar a roda: aproveitar os códigos que já existem
- Adicionar aplicação ao seu repositório pessoal (público ou privado - kaggle ou github) de forma organizada para que você possa facilmente consultá-la no futuro

### ML twitter

- Desde ~2020, nem os papers do arxiv e nem blogposts conseguem acompanhar a velocidade no avanço no ML/DL. As publicações acontecem de forma muito mais rápida e dinâmica no twitter, que é absolutamente fundamental pra quem quer acompanhar o estado da arte. [Exemplo](https://twitter.com/bneyshabur/status/1570843225585504256)
- O twitter deve ser utilizado como ferramenta de estudo e atualização. Funciona muito bem como dose diária de aprendizado, e ajuda muito a acompanhar o trabalho de outros cientistas de dados e pesquisadores.
- Ferramenta essencial não só para o acompanhamento dos avanços na área de ciência de dados e papers publicados, mas também para a absorção de dicas e experiências compartilhadas sobre casos reais de DS na indústria e área de negócios.
- Nenhuma outra plataforma te entrega a informação de forma tão rápida e curada.
- Ver sugestões de perfis a seguir no final deste documento.

### Ferramentas
- Focar em Google Colab e Kaggle notebooks.
- No futuro, é interessante conhecer IDEs como VS Code, PyCharm e Spyder.

### Panorama de DS no mercado

- [Data Scientist: The Sexiest Job of the 21st Century - artigo out/2012](https://hbr.org/2012/10/data-scientist-the-sexiest-job-of-the-21st-century)
- [Is Data Scientist Still the Sexiest Job of the 21st Century? - artigo jul/2022](https://hbr.org/2022/07/is-data-scientist-still-the-sexiest-job-of-the-21st-century)

### Áreas de dados

![Data fields](https://github.com/HinePo/Trilha-de-Estudos-de-Data-Science/blob/main/imgs/data_fields.jpg)

### Data Sequence

![Data sequence](https://github.com/HinePo/Trilha-de-Estudos-de-Data-Science/blob/main/imgs/data_sequence.jpg)

### DE vs ML pipelines

![DE vs ML pipelines](https://github.com/HinePo/Trilha-de-Estudos-de-Data-Science/blob/main/imgs/DE_vs_ML_pipelines.jpg)

- [Data Engineering vs ML pipelines](https://seattledataguy.substack.com/p/data-engineering-vs-machine-learning)

# Coding with AI

É claro que IA ajuda muito a programar, e a nossa produtividade depende muito da forma que usamos IA, principalmente via técnicas Prompt Engineering e escolha das ferramentas.

- [História da Engenharia de Software](https://www.youtube.com/watch?v=oDXDYjksMds)
- [Karpathy post](https://x.com/karpathy/status/1827143768459637073)
- [Cursor - AI Code Editor](https://www.youtube.com/watch?v=ocMOZpuAMw4) - 15 min tutorial
- [How I use LLMs](https://www.youtube.com/watch?v=EWvNQjAaOHw) - Karpathy 2h

# Python e Git

- [uv - Working on projects](https://docs.astral.sh/uv/guides/projects/)
- [Python Installation - Local](https://github.com/rasbt/LLMs-from-scratch/tree/main/setup/01_optional-python-setup-preferences)
- [Git in 2 minutes](https://www.youtube.com/watch?v=hwP7WQkmECE)
- [Git cheat sheet](https://www.linkedin.com/posts/paoloperrone_git-master-it-or-regret-it-when-ai-writes-activity-7320100374508380160-1Acn/?utm_source=share&utm_medium=member_android&rcm=ACoAADGdJpoBgFh12xUvPZJ_GJ3LDaC8Pz05JWQ)
- [Git tips](https://x.com/vikasrajputin/status/1595272684291489792)

# Data Analysis basics

- [Pandas tutorial - kaggle learn](https://www.kaggle.com/learn/pandas) - Fazer o fork dos notebooks e aprender praticando
- [Use pandas like a pro](https://www.youtube.com/watch?v=_gaAoJBMJ_Q)
- [Dataframe efficiency with dtypes - intro](https://www.youtube.com/watch?v=u4_c2LDi4b8) - Rob Mulla
- [Example of "reduce_memory_usage" function](https://www.kaggle.com/code/werus23/sleep-critical-point-prepare-data)
- [pandas cheat sheet](https://www.linkedin.com/posts/jagdish-saini-8098b1138_dataanalysis-dataengineering-datascience-activity-7079772863175532544-pZhh/?utm_source=share&utm_medium=member_android)
- [Learn Pandas with pokemons](https://www.kaggle.com/ash316/learn-pandas-with-pokemons)
- [Handling Missing Data](https://www.kaggle.com/robikscube/handling-with-missing-data-youtube-stream)
- [Statistics: most important concepts](https://www.youtube.com/watch?v=ACTr64lA_rI)

### Data Analysis workflow - entender e praticar as etapas básicas:

- Importar e ler csv, criar dataframe
- Checar tipos de variáveis (data types): numéricas e categóricas
- Plots básicos
- Analisar missing values (valores faltantes), tomar decisões sobre o que fazer com eles
- Analisar outliers, decidir o que fazer com eles
- Análise univariada, bivariada, multivariada (variáveis categóricas e numéricas)
- Feature Engineering (criação de variáveis)
- Deixar dados prontos para eventual modelagem de IA

### Machine Learning workflow - entender e praticar as etapas básicas:

- Definir Features and Target (if it is a supervised problem)
- Preprocess: Scaling and categorical encoders
- Check Target distributions
- Check features distributions, normalize them if needed
- Split train/validation datasets: Definir estratégia confiável de validação dos modelos (cross-validation strategy)
- Definir métricas de avaliação dos modelos
- Criar um baseline simples ***sem*** usar machine learning, e avaliar usando as métricas definidas
- Choose algorithm, train and validate model: create simple baseline ***using*** a ML model: [Lei de Gall](https://twitter.com/radekosmulski/status/1614385689424187392)
- Evaluate model (fora da etapa de modelagem: cruzamento com business KPIs)
- Melhorar baseline: criar variáveis melhores (feature engineering), tunar hiperparâmetros, testar outros algoritmos, treinar e avaliar de novo
- [Missing values & XGBoost](https://twitter.com/svpino/status/1559510300428156929)
- [OOF analysis & Error analysis](https://www.kaggle.com/competitions/siim-isic-melanoma-classification/discussion/175614)
- Experimentos de Feature selection e preprocessing: diferentes sets de features e diferentes etapas de pré-processamento
- Melhorar explicabilidade
- Ensemble: combinar modelos para aumentar performance, estabilidade e poder de generalização

# Data Engineering

Um pouco de Engenharia de Dados (para cientistas de dados):

- [Data Warehouse, Data Lake, Data Lake House, Data Mesh](https://www.linkedin.com/posts/rajesh-kr-mehra_datamanagement-datawarehouse-datalake-activity-7112461700250931200-RIlw?utm_source=share&utm_medium=member_desktop)
- Pesquisar no Youtube: "End to End Azure Data Engineering Project"
- [Pyspark playlist](https://www.youtube.com/watch?v=KnUXztKueMU&list=PLWAuYt0wgRcLCtWzUxNg4BjnYlCZNEVth)
- [Luan Moreno - Youtube channel](https://www.youtube.com/@LuanMorenoMMaciel)

# Data Visualization

- [Python Graph Gallery](https://www.python-graph-gallery.com/)
- [Visual Reference](https://www.sqlbi.com/wp-content/uploads/videotrainings/dashboarddesign/visuals-reference-may2017-A3.pdf)
- [Séries de notebooks de visualização](https://www.kaggle.com/residentmario/univariate-plotting-with-pandas) - ao final de cada notebook tem um link para o próximo
- [Data Analysis - Brazilian Society (PNAD)](https://www.kaggle.com/hinepo/pnad-data-analysis) - @hinepo
- [Rio Temperature Analysis](https://www.kaggle.com/hinepo/is-rio-getting-hotter-seasons-analysis/) - @hinepo
- [Power BI playlists](https://www.youtube.com/c/HashtagTreinamentos/playlists?app=desktop)
- [Power BI - Leonardo Karpinski](https://www.youtube.com/c/AprendaPowerBI)
- [Power BI - Karine Lago](https://www.youtube.com/c/KarineLago/playlists?app=desktop)
- [Power BI + DAX + Projetos na prática - Curso Udemy](https://www.udemy.com/course/curso-de-powerbi-desktop-dax/)
- [graphviz - exemplos](https://graphviz.readthedocs.io/en/stable/examples.html)

# Machine Learning - Teoria

- [Supervised x Unsupervised Learning](https://www.youtube.com/watch?app=desktop&v=1FZ0A1QCMWc)
- [Supervised x Unsupervised Learning: applications](https://www.youtube.com/watch?app=desktop&v=rHeaoaiBM6Y)
- Pesquisar sobre Overfitting e Underfitting, ver vídeos e gráficos
- [Cross Validation](https://www.youtube.com/watch?app=desktop&v=fSytzGwwBVw)
- [Cross Validation - scikit docs](https://scikit-learn.org/stable/modules/cross_validation.html)
- Pesquisar sobre Cross Validation para Time Series (como evitar contaminação de dados do futuro pro passado, data leakage, train/test contamination...)
- [Kaggle courses](https://www.kaggle.com/learn)
- [pdf do livro do Abhishek Thakur](https://github.com/abhishekkrthakur/approachingalmost/blob/master/AAAMLP.pdf) - disponível na Amazon tb
- [Statquest - Vídeos sobre conceitos, teoria e matemática de algoritmos e ML](https://www.youtube.com/c/joshstarmer/playlists?app=desktop)
- [Scikit-learn User Guide](https://scikit-learn.org/stable/user_guide.html) - Muito importante ler todo o item 1
- [Scikit-learn Pre-processing](https://scikit-learn.org/stable/modules/preprocessing.html)
- Pesquisar sobre "Feature Engineering" (criação de variáveis)
- Pesquisar sobre métricas e como avaliar modelos:
  - Classificação: Accuracy, ROC AUC, f1-score, recall, precision, [MCC](https://towardsdatascience.com/the-best-classification-metric-youve-never-heard-of-the-matthews-correlation-coefficient-3bf50a2f3e9a), [Log Loss e Balanced Log Loss](https://www.kaggle.com/competitions/icr-identify-age-related-conditions/discussion/422442)
  - Regressão: RMSE, NRMSE, MSE, MAE, MAPE, [Tip about MAPE](https://twitter.com/a_erdem4/status/1562321014243164161), R²
    - [Transforming skewed data](https://medium.com/@ODSC/transforming-skewed-data-for-machine-learning-90e6cc364b0) - como tratar o viés nos dados
- Outros conceitos importantes: Pesquisar sobre Boosting (XGBoost, LGBM, Catboost, GBM), Bagging, Split train/test, data leakage, time series, feature importances, ensemble...
- Imbalanced learning:
  - [4 ways to balance classes](https://www.kaggle.com/competitions/icr-identify-age-related-conditions/discussion/412507)
  - downsampling/upsampling
  - [imblearn](https://opendatascience.com/strategies-for-addressing-class-imbalance/)
  - [Oversampling x Undersampling](https://www.kdnuggets.com/2020/01/5-most-useful-techniques-handle-imbalanced-datasets.html)
  - [Resampling example](https://www.kaggle.com/rafjaa/resampling-strategies-for-imbalanced-datasets)
  - [SMOTE for classification example](https://www.kaggle.com/shrutimechlearn/pokemon-classification-and-smote-for-imbalance/notebook)
  - [Stop Using SMOTE to Treat Class Imbalance](https://towardsdatascience.com/stop-using-smote-to-treat-class-imbalance-take-this-intuitive-approach-instead-9cb822b8dc45)

# Machine Learning - Prática
- [Kaggle's 30 Days of ML](https://www.youtube.com/playlist?app=desktop&list=PL98nY_tJQXZnP-k3qCDd1hljVSciDV9_N) - Abhishek Thakur
- [Feature Engineering](11055170822145) - Mark Tenenholtz
- [SHAP (SHapley Additive exPlanations)](https://github.com/helenaEH/SHAP_tutorial)
- [Target encoding - kaggle Learn](https://www.kaggle.com/code/ryanholbrook/target-encoding)
- [Target encoding - blog post](https://towardsdatascience.com/why-you-should-try-mean-encoding-17057262cd0)
- [11 Categorical Encoders and Benchmark - kaggle](https://www.kaggle.com/code/subinium/11-categorical-encoders-and-benchmark)
- [Adversarial Validation](https://twitter.com/svpino/status/1775154270708396215)
- Browse kaggle, ver notebooks e datasets dos assuntos que te interessam
- Fazer forks de notebooks do kaggle (Copy and Edit), testar hipóteses e técnicas
- Falar com as pessoas do kaggle, comentar e postar, fazer parte da comunidade
- Competições 'Getting Started': estudar notebooks com bom score, e usar técnicas e conceitos aprendidos para criar o seu próprio. Estudar notebooks com score médio, comparar com os de score bom, e entender o que causou a melhora na pontuação. Recomendo no mínimo uns 10 dias de estudo para cada uma das competições abaixo:
  - [Titanic Classification](https://www.kaggle.com/c/titanic)
  - [House Prices Regression](https://www.kaggle.com/c/house-prices-advanced-regression-techniques)
  - [Predict Future Sales](https://www.kaggle.com/c/competitive-data-science-predict-future-sales)
  - [Tabular Playground Series](https://www.kaggle.com/c/tabular-playground-series-sep-2021)
  - Nível avançado: competições reais (valendo prêmios)

### Algorithm Optimization & Tuning techniques

- [Optuna library](https://optuna.org/)
- [Optuna example - notebook](https://www.kaggle.com/hinepo/extra-trees-optimization-with-optuna) - @hinepo
- [Optuna example](https://www.youtube.com/watch?v=m5YSKPMjkrk&list=PL98nY_tJQXZnP-k3qCDd1hljVSciDV9_N&index=23) - Abhishek Thakur
- [Optuna official tutorial](https://optuna.readthedocs.io/en/stable/tutorial/index.html)
- [Tuning techniques](https://www.youtube.com/watch?v=5nYqK-HaoKY) - Abhishek Thakur

# Time Series

- [Rob Mulla tutorial](https://www.youtube.com/watch?v=vV12dGe_Fho)
- [3 methods for Time Series validation](https://forecastegy.com/posts/3-essential-methods-to-do-time-series-validation-in-machine-learning/)
- [Error Analysis for Time Series - Mark Tanenholtz](https://twitter.com/marktenenholtz/status/1509500787189190658)
- [More time series tips from Mark](https://twitter.com/marktenenholtz/status/1633090131514241026?t=j_GHtqW4tGlL-7mYP7xLrQ&s=08)
- [Even more time series tips from Mark](https://twitter.com/marktenenholtz/status/1661372273888993285?t=zzPy9Meia-gtr2kK_tFJsg&s=08)
- [Classic features for time series models](https://twitter.com/marktenenholtz/status/1667670055897927681)
- [Techniques for working with time series data](https://towardsdatascience.com/a-collection-of-must-know-techniques-for-working-with-time-series-data-in-python-7c01d199b184)
- [Time Series basic concepts](https://medium.com/swlh/time-series-analysis-7006ea1c3326)
- [Time Series - Youtube playlist](https://www.youtube.com/playlist?list=PL98nY_tJQXZmT9ZB59T0lsx0ZzzLrYdX4)
- [pmdarima](http://alkaline-ml.com/pmdarima/), [statsmodels](https://www.statsmodels.org/stable/index.html), ARIMA, SARIMA, [prophet](https://facebook.github.io/prophet/docs/quick_start.html#python-api), theta model
- [Case Zillow, Prophet, Time Series, & Prices](https://ryxcommar.com/2021/11/06/zillow-prophet-time-series-and-prices/)
- [Time-series prediction with XGBoost](https://www.linkedin.com/feed/update/urn:li:activity:7069236572063186944/)
- [XGBoost for time series example](https://mlwhiz.com/blog/2019/12/28/timeseries/?utm_campaign=using-gradient-boosting-for-time-series-prediction-tasks&utm_medium=social_link&utm_source=missinglettr)
- **Tabular Machine learning approach para múltiplas séries temporais:**
  - Create Lag features
  - Group by série, tail(1)
  - Validation strategy, Types of leakage, Evaluation
  - Exemplos, notebooks e datasets bons:
    - [Riiid](https://www.kaggle.com/competitions/riiid-test-answer-prediction/code?competitionId=21651&sortBy=voteCount)
    - [Predict Future Sales](https://www.kaggle.com/competitions/competitive-data-science-predict-future-sales/code?competitionId=8587&sortBy=voteCount)
    - [AMEX](https://www.kaggle.com/code/cdeotte/xgboost-starter-0-793#Process-and-Feature-Engineer-Train-Data)

# Deep Learning - Neural Networks

Principais conceitos e keywords a pesquisar e aprender: tensors, gradient descent, automatic differentiation, forward pass, backpropagation, layers, batch, epoch, iteration, optimizer step, scheduler step, vanishing gradients, exploding gradients, transfer learning (fine-tuning & feature extraction)...

Basics:
- [Neural Networks - 3Blue1Brown Playlist (~1h)](https://www.youtube.com/playlist?list=PLZHQObOWTQDNU6R1_67000Dx_ZCJB-3pi)
- [Aula Intro de DL - Lex Friedman](https://www.youtube.com/watch?app=desktop&v=O5xeyoRL95U)
- [Deep Learning & Bugs - discussion](https://twitter.com/SebastienBubeck/status/1571887980046065667)
- [Exploding and Vanishing gradients](https://www.analyticsvidhya.com/blog/2021/06/the-challenge-of-vanishing-exploding-gradients-in-deep-neural-networks/)
- [Regression Networks - The Magic of No Dropout](https://www.kaggle.com/competitions/commonlitreadabilityprize/discussion/260729)
- [Kaggle Grandmasters Tips to train deep learning models](https://www.youtube.com/watch?v=NCGkBseUSdM)
- [Deep Learning Tuning Playbook](https://github.com/google-research/tuning_playbook)
- [How To Tune Learning Rate Scheduler](https://www.kaggle.com/competitions/hms-harmful-brain-activity-classification/discussion/488083)
- [LSTM & GRU in 11 minutes](https://www.youtube.com/watch?v=8HyCNIVRbSU)

Frameworks:
- [Pytorch vs Tensorflow in 2022](https://www.assemblyai.com/blog/pytorch-vs-tensorflow-in-2022/)

Keras:
- [Keras Sequential class](https://keras.io/api/models/sequential/)
- [Finetuning - Keras tutorial](https://keras.io/guides/transfer_learning/)
- [Keras - code examples](https://keras.io/examples/)

Tensorflow:
- [Tensorflow guide - kaggle learn](https://www.kaggle.com/learn-guide/tensorflow)

Pytorch:
- [Pytorch Model In Minutes - Rob Mulla](https://www.youtube.com/watch?v=tHL5STNJKag)
- [Pytorch DataLoader animation](https://twitter.com/_ScottCondron/status/1363494433715552259?t=whYsVwzXnMxZ-pHuQn8FMQ&s=08)
- [Finetuning x Feature Extraction - pytorch docs and examples](https://pytorch.org/tutorials/beginner/finetuning_torchvision_models_tutorial.html)
- [Pytorch - Abhishek Thakur playlist and tutorials](https://www.youtube.com/playlist?app=desktop&list=PL98nY_tJQXZln8spB5uTZdKN08mYGkOf2)
- [Pytorch - torch.nn](https://pytorch.org/docs/stable/nn.html)
- [Vídeo aulas com code examples (pytorch)](https://sebastianraschka.com/blog/2021/dl-course.html)

Um estudo muito útil e proveitoso é comparar e olhar em paralelo as documentações de Quick Start do Keras, do Tensorflow e do Pytorch. A lógica é bem parecida e existem muitas analogias:
- [Keras Quick Start](https://www.tensorflow.org/tutorials/quickstart/beginner)
- [Tensorflow Quick Start](https://www.tensorflow.org/tutorials/quickstart/advanced)
- [Pytorch Quick Start](https://pytorch.org/tutorials/beginner/basics/data_tutorial.html)

Principais tipos de camadas (layers):
- Dense & Linear (fully connected)
- Activation functions (ReLU, LeakyReLU, SELU, PReLU, Tanh, Softmax, Sigmoid....)
- Conv (Convolutional)
- Flatten
- BatchNorm, LayerNorm, RMSNorm
- LSTM (Long Short Term Memory), BiLSTM
- GRU (Gated Recurrent Unit - Short Term Memory), BiGRU
- Dropout
- Pooling (Max, Mean, Average, Generalized Mean etc)
- Concatenate

### Papers - Why and When Deep Learning?

- [Do We Really Need Deep Learning Models for Time Series Forecasting? - paper oct/2021](https://arxiv.org/pdf/2101.02118.pdf)
- [Tabular Data: Deep Learning Is Not All You Need - paper nov/2021](https://arxiv.org/pdf/2106.03253.pdf)

### Extra:

- [JAX docs](https://jax.readthedocs.io/en/latest/)

"JAX is Autograd and XLA, brought together for high-performance numerical computing and machine learning research. It provides composable transformations of Python+NumPy programs: differentiate, vectorize, parallelize, Just-In-Time compile to GPU/TPU, and more."

- [JAX - Quick Start](https://jax.readthedocs.io/en/latest/notebooks/quickstart.html)

"JAX is NumPy on the CPU, GPU, and TPU, with great automatic differentiation for high-performance machine learning research."

[JAX é um projeto open source do Google](https://github.com/google/jax) com o objetivo de criar uma API simples e backend eficiente para cálculos de deep learning. Tem crescido em popularidade e sido considerada muito promissora por pesquisadores. Imagina-se que em alguns anos será um concorrente direto do Pytorch (na área de pesquisa), e também deverá substituir o backend do tensorflow (na área de aplicações). Há quem chame o JAX de  "tensorflow 3", e já existem planos para criação de uma API high level para JAX, adaptando a biblioteca Keras para usar JAX como backend. Portanto, é interessante conhecer.

- [JAX guide - kaggle learn](https://www.kaggle.com/learn-guide/jax)

# Transformers

Os Transformers e o Attention Mechanism, propostos em 2017 por Vaswani - Google Brain no paper Attention Is All You Need, são, até hoje, a maior revolução que o mundo do Deep Learning já passou. Vale a pena estudá-los com atenção (pun intended 😆), pois eles são o estado da arte em redes neurais hoje em dia para a maioria dos tasks, e pelo visto continuarão sendo por bastante tempo.

Transformers mostraram que não é preciso usar camadas LSTM para fazer tasks de NLP no estado da arte, e também não precisamos de camadas de Convolução para fazer CV (Computer Vision) no estado da arte. Attention Is All You Need.

### Papers

- [How to read papers - twitter thread](https://twitter.com/marktenenholtz/status/1498644231149142025)
- [Attention Is All You Need - paper dec/2017](https://arxiv.org/pdf/1706.03762.pdf)
- [BERT - paper may/2019](https://arxiv.org/pdf/1810.04805.pdf)
- [RoBERTa - paper jul/2019](https://arxiv.org/pdf/1907.11692.pdf)
- [SBERT - paper aug/2019](https://arxiv.org/pdf/1908.10084.pdf) - Sentence Transformers
- [TaBERT - paper may/2020](https://arxiv.org/pdf/2005.08314v1.pdf) - Learning Joint Representations over Textual and Tabular Data
- [T5: Text-To-Text Transfer Transformer - paper jul/2020](https://arxiv.org/pdf/1910.10683.pdf)
- [Longformer - paper dec/2020](https://arxiv.org/pdf/2004.05150.pdf) - Local Attention
- [ViT - paper jun/2021](https://arxiv.org/pdf/2010.11929.pdf) - Vision Transformers
- [Swin Transformer - paper aug/2021](https://arxiv.org/pdf/2103.14030.pdf) - Shifted Window based Self-Attention
- [DeBERTa - paper oct/2021](https://arxiv.org/pdf/2006.03654.pdf) - Disentangled Attention

### Outras fontes

- [Attention explained - Lightning AI - 4 min](https://twitter.com/rasbt/status/1693635148569354688?t=dVVuj6gv0m9gGlqBySUUrA&s=08)
- [BERT Attention Mechanism](https://peltarion.com/blog/data-science/self-attention-video)
- [Illustrated Guide to Transformers](https://www.youtube.com/watch?app=desktop&v=4Bdc55j80l8)
- [The Illustrated Transformer](http://jalammar.github.io/illustrated-transformer/)
- [The Annotated Transformer](http://nlp.seas.harvard.edu/2018/04/03/attention.html)
- [Attention implementation in torch from scratch - twitter thread](https://twitter.com/abhi1thakur/status/1470406419786698761) - Abhishek Thakur
- [Attention implementation in torch from scratch - twitter thread 2](https://twitter.com/abhi1thakur/status/1471126502112698368) - Abhishek Thakur
- [Transformers from Scratch](https://e2eml.school/transformers.html) - explicação visual e detalhada
- [Awesome Self-Supervised Learning - github repo](https://github.com/jason718/awesome-self-supervised-learning)

# NLP - Natural Language Processing

Principais conceitos e keywords a conhecer: n-grams, CountVectorizer, TF-IDF, BOW (Bag of Words), CBOW (Continuous Bag of Words), Word2vec, FastText (facebook model), GloVe (Global Vectors), BERT, RoBERTa, Hugging Face....

- [A brief timeline of NLP from Bag of Words to the Transformer family](https://medium.com/nlplanet/a-brief-timeline-of-nlp-from-bag-of-words-to-the-transformer-family-7caad8bbba56)
  - Bag of words & ML models
  - Word vectors & Deep Learning
  - Context vectors & Transformers
- [The Illustrated Word2vec - A Gentle Intro to Word Embeddings in Machine Learning](https://www.youtube.com/watch?v=ISPId9Lhc1g)
- [Resumo Hugging face library - 15 min video](https://www.youtube.com/watch?v=QEaBAZQCtwE)
- [Hugging face - finetune a pretrained model: Trainer, native Pytorch, native Tensorflow](https://huggingface.co/docs/transformers/training)
- [Hugging Face course](https://huggingface.co/course/chapter1/1) - excelente curso. HF é o melhor ecossistema de NLP e continuará sendo por muitos anos
- [10 Things You Need to Know About BERT and Transformer Architecture](https://neptune.ai/blog/bert-and-the-transformer-architecture-reshaping-the-ai-landscape)
- [A Survey of Transformers - paper jun/2021](https://arxiv.org/pdf/2106.04554.pdf)

### Aplicações:
- [Question Answering & Chatbot on private pdf (without LangChain) - Abhishek](https://www.youtube.com/watch?v=hSQY4N1u3v0)
- [BERT tutorial by Abhishek Thakur](https://www.youtube.com/playlist?app=desktop&list=PL98nY_tJQXZl0WwsJluhc6tGrKWCX2suH)
- [Transformers in Tensorflow](https://www.tensorflow.org/text/tutorials/transformer#setup)
- [NLP guide - kaggle learn](https://www.kaggle.com/learn-guide/natural-language-processing)
- [Hugging Face tutorial - video (30 min)](https://www.youtube.com/watch?v=DQc2Mi7BcuI])

# LLMs

Large Language Models

### Conceitos básicos

- [LLM Visualization](https://bbycroft.net/llm)
  - [State of GPT - Andrej Karpathy](https://www.youtube.com/watch?v=bZQun8Y4L2A)
  - [Eight Things to Know about Large Language Models](https://www.youtube.com/watch?v=RX-gGs_EV7M) - AI Coffee Break With Letitia
  - [The busy person's intro to LLMs - Andrej Karpathy](https://www.youtube.com/watch?v=zjkBMFhNj_g)
  - [Vector databases and use cases](https://www.youtube.com/watch?v=dN0lsF2cvm4)
  - [LLM Based Chatbot to query your Private Knowledge Base - System Architecture](https://twitter.com/Aurimas_Gr/status/1673621343122726912?t=jLJTcYxEaP7-640PYVZJ-g&s=08)
  - [Bits and Bytes + Hugging Face integration](https://huggingface.co/blog/4bit-transformers-bitsandbytes)
  - [RLHF: Reinforcement Learning from Human Feedback](https://huyenchip.com/2023/05/02/rlhf.html)
  - [Understanding PEFT (Parameter-Efficient Finetuning)](https://lightning.ai/pages/community/article/understanding-llama-adapters/)
  - [LoRA](https://lightning.ai/pages/community/tutorial/lora-llm/)
  - [QLoRA: Efficient Finetuning of Quantized LLMs](https://arxiv.org/pdf/2305.14314.pdf)
  - [How to generate text: decoding methods](https://huggingface.co/blog/how-to-generate)
  - [Text generation strategies - Hugging Face Generation docs](https://huggingface.co/docs/transformers/generation_strategies)

### Prompt Engineering basics

- [Debugging code with LLMs](https://x.com/mattshumer_/status/1890441651278971330) - Matt Schumer
- [Prompt Engineering tips](https://www.linkedin.com/posts/ahmed--almubarak_prompts-engineering-ugcPost-7298368175514038275-6gDV/?utm_source=share&utm_medium=member_desktop&rcm=ACoAADGdJpoBgFh12xUvPZJ_GJ3LDaC8Pz05JWQ) - Chip Huyen

Uma das melhores formas de aprender Prompt Engineering é estudando os System Prompts vazados dos grandes modelos comerciais. Eles foram testados extensivamente e contém várias técnicas e tricks que funcionam muito bem.

- [ChatGPT System Prompts](https://x.com/elder_plinius/status/1714291610018779337) e [How to get ChatGPT System Prompt - example](https://x.com/dylan522p/status/1755118636807733456)
- [Claude 3 System Prompt](https://x.com/AmandaAskell/status/1765207842993434880)
- [Grok3 Search - System Prompt](https://x.com/lefthanddraft/status/1893681902957076687?t=LSKSlrExrkAeWXTJvW-aaQ&s=08)
- [ReAct Prompt template](https://smith.langchain.com/hub/langchain-ai/react-agent-template) - from paper "ReAct: Synergizing Reasoning and Acting in Language Models" - mar/2023

### LangChain - Getting Started

- [LangChain Explained in 13 Minutes](https://www.youtube.com/watch?v=aywZrzNaKjs)
  - [LangChain - Quickstart](https://python.langchain.com/en/latest/getting_started/getting_started.html)
  - [8 use cases with LangChain](https://twitter.com/JorisTechTalk/status/1660372052468015105)
  - [Multiple-document retriever](https://www.youtube.com/watch?v=9ISVjh8mdlA)
  - [Talk to your data](https://www.youtube.com/watch?v=xQ3mZhw69bc)
  - [Chatbot memory types in LangChain](https://twitter.com/Luc_AI_Insights/status/1662549087458144257?t=EsgW6ztVSZ4K0UCApf19WA&s=08)

### RAG: Retrieval Augmented Generation

- [RAG: Question-Answering chatbot with LangChain and Harry Potter](https://www.kaggle.com/code/hinepo/harry-potter-question-answering-with-langchain/notebook) - @hinepo
  - [Retrieval augmentation tips](https://twitter.com/waydegilliam/status/1663631626440671232?t=VX8IcvpZDXr-CgXMv_rCFA&s=08)
  - [OpenAI roadmap for building production RAG systems](https://twitter.com/yi_ding/status/1721728060876300461)
  - [RAG + StrOutputParser, RunnablePassthrough, RunnableMap, itemgetter](https://twitter.com/waseemhnyc/status/1728889388954951717?t=Vy6R43k5l74owmvfPVw7wQ)
  - [Building RAG-based LLM Applications for Production](https://www.anyscale.com/blog/a-comprehensive-guide-for-building-rag-based-llm-applications-part-1) - Goku Mohandas

### Advanced RAG

- [Advanced RAG Strategies - RAGOps - video](https://www.youtube.com/watch?v=EuHderGVUs8&t=949s)  
  - [RAG - Contextual Compression](https://www.kaggle.com/code/paulojunqueira/rag-contextual-compression-boardgames-rulebooks#--Introduction-) - [Paulo Junqueira](https://www.kaggle.com/paulojunqueira)
  - [RAG - Hybrid Search](https://www.kaggle.com/code/paulojunqueira/rag-hybrid-search-in-boardgame-rulebooks-pdf#--References-and-Acknowledgment-) - [Paulo Junqueira](https://www.kaggle.com/paulojunqueira)
  - [10 Ways to Improve the Performance of Retrieval Augmented Generation Systems](https://towardsdatascience.com/10-ways-to-improve-the-performance-of-retrieval-augmented-generation-systems-5fa2cee7cd5c)
  - [Advanced RAG - Langchain code examples](https://github.com/langchain-ai/rag-from-scratch/blob/main/rag_from_scratch_5_to_9.ipynb)
  - [RAG Evaluation](https://www.linkedin.com/posts/areganti_genai-llms-rag-activity-7198872711786360832-pAEh/?utm_source=share&utm_medium=member_android)
  - [Advanced RAG - Query Optimization](https://www.kaggle.com/code/gabrielvinicius/advanced-rag-query-optimization/notebook#Query-Rewrite) - [Gabriel Vinicius](https://www.kaggle.com/gabrielvinicius)

### Finetuning LLMs

Rag vs Finetuning

- [RAG vs Finetuning](https://twitter.com/ajratner/status/1698072260152787342)
  - [RAG vs Finetuning again](https://www.linkedin.com/posts/jeremy-arancio_fine-tuning-an-llm-on-your-data-will-not-activity-7109833276969627648-_izS/?utm_source=share&utm_medium=member_desktop)

There are many ways to finetune a LLM. Some examples are: Instruction Finetuning, Preference Finetuning, Class Finetuning and Regression Finetuning.

- [LLM Instruction Finetuning + WandB](https://www.kaggle.com/code/hinepo/llm-instruction-finetuning-wandb) - @hinepo
  - [LLM Alignment: Preference Finetuning - DPO, SLiC, IPO, KTO](https://www.kaggle.com/code/hinepo/llm-alignment-preference-finetuning/notebook) - @hinepo
  - [How to Fine-Tune LLMs in 2024 with Hugging Face](https://www.philschmid.de/fine-tune-llms-in-2024-with-trl) - philschmid
  - [RLHF in 2024 with DPO & Hugging Face](https://www.philschmid.de/dpo-align-llms-in-2024-with-trl) - philschmid
  - [Fine-Tune Your Own Llama 2 Model in a Colab Notebook](https://mlabonne.github.io/blog/posts/Fine_Tune_Your_Own_Llama_2_Model_in_a_Colab_Notebook.html)
  - [Synthetic data creation for LLMs](https://www.kaggle.com/code/hinepo/synthetic-data-creation-for-llms/notebook) - @hinepo
  - [Synthetic data - Hugging Face: Few-shot vs DeBERTa finetuned on Synthetic data](https://huggingface.co/blog/synthetic-data-save-costs)
  - [SPIN: Self-Play Finetuning for LLMs](https://www.kaggle.com/code/hinepo/spin-self-play-finetuning-for-llms) - @hinepo

### Agents

Basics:
  - [Book with code - Antonio Gulli](https://docs.google.com/document/d/1rsaK53T3Lg5KoGwvf8ukOUvbELRtH-V0LnOIFDxBryE/preview?tab=t.0#heading=h.pxcur8v2qagu) - This may be "all" you need about Agents
  - [Tool Calling with LangChain](https://www.youtube.com/watch?v=a_YC8Ru6Jd0)
  - [ReAct: Synergizing Reasoning and Acting in Language Models](https://arxiv.org/pdf/2210.03629) - paper mar/2023
  - [AgentExecutor in Langchain and LangGraph](https://www.youtube.com/watch?v=UcD42NA2WoI)
  - [Function Calling Agents vs. ReACt Agents – What's Right for You?](https://www.youtube.com/watch?v=L6suEeJ3XXc)
  - [Advanced RAG with Agents - video](https://www.youtube.com/watch?v=-ROS6gfYIts&t=2s)
  - [Advanced RAG with Agents - code](https://github.com/langchain-ai/langgraph/blob/main/examples/rag/langgraph_rag_agent_llama3_local.ipynb)
  - [Function Calling with Local Models - Ollama, Llama-3, Phi-3](https://www.youtube.com/watch?v=Ss_GdU0KqE0)

Langchain docs:
  - [Tool calling agent](https://python.langchain.com/v0.1/docs/modules/agents/agent_types/tool_calling/)
  - [Defining Custom Tools](https://python.langchain.com/v0.1/docs/modules/tools/custom_tools/)

Hugging Face Agents
  - [Agents and tools](https://huggingface.co/docs/transformers/main/en/agents)
  - [Transformers Agents](https://huggingface.co/docs/transformers/main/en/transformers_agents)

### DSPy

- [Stop Prompting & Start Programming: DSPy Examples](https://www.kaggle.com/discussions/general/500884)
  - [Automatic Prompt Generation with DSPy Agents](https://www.kaggle.com/code/rkr2020/automatic-prompt-generation-with-dspy-agents)
  - [Intro to DSPy: Goodbye Prompting, Hello Programming!](https://towardsdatascience.com/intro-to-dspy-goodbye-prompting-hello-programming-4ca1c6ce3eb9)
  - [DSPy - Github](https://github.com/stanfordnlp/dspy)
  - [DSPY: Compiling Declarative Language Model Calls Into Self-Improving Pipelines](https://arxiv.org/pdf/2310.03714) - paper oct/2023

### LLMs in Production
  - Deploying open source models at scale - [twitter post](https://x.com/chipro/status/1816521492760580127?t=19B7svBhsWEPJwDum0_6Dw&s=08) and [blog post](https://huyenchip.com/2024/07/25/genai-platform.html) - Chip Huyen
  - [GenAI + Classical ML - Chip Huyen](https://www.linkedin.com/posts/chiphuyen_llms-aiengineering-aiapplications-activity-7211786332660989954-LzO0/?utm_source=share&utm_medium=member_android)
  - [Comcast - Exemplo de um sistema de LLM em Produção](https://arxiv.org/pdf/2405.00801v2)
  - [Microsoft LLM Team - Why is it so hard to ship a simple LLM feature?](https://medium.com/data-science-at-microsoft/why-is-it-so-hard-to-ship-a-simple-llm-feature-ba7de31ffae0)
  - [Patterns for Building LLM-based Systems & Products](https://eugeneyan.com/writing/llm-patterns/)
  - [Building LLM applications for production](https://huyenchip.com/2023/04/11/llm-engineering.html)
  - [Speculative execution for LLMs](https://twitter.com/karpathy/status/1697318534555336961) - Karpathy
  - [Speculative decoding](https://twitter.com/Yampeleg/status/1701290397148856349?t=Iw3PrMwexkboOwg-o3R6sg&s=08) - Yam Peleg
  - [LangChain in Production - Microservice Architecture](https://www.youtube.com/watch?v=I_4jEnDwGwI)
  - [LLM Evaluation](https://www.linkedin.com/posts/eordax_ai-genai-llmops-activity-7186362795985158145-PNCc?utm_source=share&utm_medium=member_desktop)

### Guardrails, Safeguards, Assertions
  - [LLM Guard](https://github.com/laiyer-ai/llm-guard)
  - [LLM Safeguards](https://www.kaggle.com/code/hinepo/llm-safeguards-and-guardrails) - notebook @hinepo
  - [AWS Bedrock](https://aws.amazon.com/pt/blogs/aws/guardrails-for-amazon-bedrock-helps-implement-safeguards-customized-to-your-use-cases-and-responsible-ai-policies-preview/?sc_channel=sm&sc_campaign=Machine_Learning&sc_publisher=LINKEDIN&sc_geo=GLOBAL&sc_outcome=awareness&trk=aiml_reinvent&linkId=251501736)
  - [Azure OpenAI Service - Content Filtering](https://learn.microsoft.com/en-us/azure/ai-services/openai/concepts/content-filter?tabs=warning%2Cpython)
  - [Assertions](https://twitter.com/sh_reya/status/1747304364103041296) - paper jan/2024

### Outras aplicações
  - [Zero-Shot Classification Template - OpenAI](https://www.kaggle.com/code/hinepo/zero-shot-classification-template-openai) - HinePo
  - [Finetuning on Zero-Shot labels](https://www.kaggle.com/code/hinepo/finetuning-on-zero-shot-labels) - HinePo
  - [Topic Modeling com BERTopic, UMAP, HDBSCAN e Llama-2](https://maartengrootendorst.substack.com/p/topic-modeling-with-llama-2)
  - [Mastering Customer Segmentation with Embeddings and Clustering](https://towardsdatascience.com/mastering-customer-segmentation-with-llm-3d9008235f41#3a33)
  - [LLM-PowerHouse - Great collection of code and articles](https://github.com/ghimiresunil/LLM-PowerHouse-A-Curated-Guide-for-Large-Language-Models-with-Custom-Training-and-Inferencing)
  - [LLM Course with code examples](https://github.com/mlabonne/llm-course)

# Computer Vision

### OpenCV
- [Image Processing with OpenCV and Python](https://www.youtube.com/watch?v=kSqxn6zGE0c)
- [How to work with video data](https://www.youtube.com/watch?v=AxIc-vGaHQ0)
- [OpenCV Tutorials](https://github.com/murtazahassan/OpenCV-Python-Tutorials-and-Projects)
- [Image and Video processing - Great Barrier Reef](https://www.kaggle.com/code/soumya9977/learning-to-sea-underwater-img-enhancement-eda/notebook)

### Yolo

Existem várias versões do algoritmo Yolo. Vale a pena dar uma olhada nas evoluções que ocorreram desde o lançamento da versão original em 2015.

Porém, hoje em dia o pacote Ultralytics é a melhor forma de usar Yolo, e inclui várias versões e variações do Yolo e também de outros algoritmos.

Algumas funcionalidades:
  - Tasks: Detection, Segmentation, Classification e Pose Estimation
  - Modes: Train, Validate, Predict, Export, Tracking e até Benchmarking de performance

É muito importante ler toda a documentação do Ultralytics.

- [Ultralytics docs](https://docs.ultralytics.com/)
- [YOLO Introduction](https://www.youtube.com/watch?app=desktop&v=4eIBisqx9_g)
- [YOLOv5 tutorial - notebook](https://colab.research.google.com/github/ultralytics/yolov5/blob/master/tutorial.ipynb)
- [How to Train YOLOv8 Object Detection on a Custom Dataset - using folder structure and LabelImg](https://medium.com/augmented-startups/train-yolov8-on-custom-data-6d28cd348262)
- [How to Train YOLOv8 Object Detection on a Custom Dataset - using Roboflow](https://blog.roboflow.com/how-to-train-yolov8-on-a-custom-dataset/)
- [Supervision docs](https://supervision.roboflow.com/) - A lot of common Computer Vision utilities and features out of the box

Aplicações:

- [YOLOv8 Finetuning for PPE detection](https://www.kaggle.com/code/hinepo/yolov9-finetuning-for-ppe-detection/notebook) - @hinepo
- [YOLOv8 Inference for Red Zone application](https://www.kaggle.com/code/hinepo/yolov9-inference-for-red-zone-application) - @hinepo
- [YOLOv9 + Supervision: Heatmap + Tracking + Counting](https://www.kaggle.com/code/hinepo/yolov9-supervision-heatmap-track-count) - @hinepo
- [YOLO-World: Zero-shot object detection](https://www.kaggle.com/code/hinepo/yolo-world-zero-shot-object-detection)
- [Ultralytics YOLOv8 Solutions: Quick Walkthrough](https://ultralytics.medium.com/ultralytics-yolov8-solutions-quick-walkthrough-b802fd6da5d7)
- [Traffic Analysis with YOLOv8 and ByteTrack - Vehicle Detection and Tracking](https://www.youtube.com/watch?v=4Q3ut7vqD5o)
- [YOLOv8 - Detecting and Counting vehicles](https://www.kaggle.com/code/paulojunqueira/yolo-v8-vehicles-detecting-counting/notebook) - [Paulo Junqueira](https://www.kaggle.com/paulojunqueira)
- [YOLOv8 segmentation in 12 minutes](https://www.youtube.com/watch?v=pFiGSrRtaU4)
- [YOLOv5 - Great Barrier Reef - Train](https://www.kaggle.com/code/awsaf49/great-barrier-reef-yolov5-train#Tensorflow---Help-Protect-the-Great-Barrier-Reef)
- [YOLOv5 - Great Barrier Reef - Inference](https://www.kaggle.com/code/awsaf49/great-barrier-reef-yolov5-infer/notebook#%F0%9F%93%A6-YOLOv5)
- [YOLOv5 tutorial by Abhishek Thakur](https://www.youtube.com/watch?app=desktop&v=NU9Xr_NYslo)

### Basics

- [Computer Vision History by Andrej Karpathy](https://www.youtube.com/watch?v=u6aEYuemt0M&t=1460s) (até 2016)
- [Kernel size in convolution layers](https://medium.com/analytics-vidhya/significance-of-kernel-size-200d769aecb1)
- [Digit Recognizer: Getting Started Competition](https://www.kaggle.com/c/digit-recognizer) - ‘Hello World’ do mundo de CV: Estudar vários notebooks com bom score, e depois criar o seu misturando várias técnicas que vc achou promissoras em outros notebooks, tentando melhorar o score do baseline. Recomendo no mínimo uns 10 dias de estudo para essa competição.
- [chatGPT resolvendo MNIST usando keras](https://twitter.com/svpino/status/1626204842095161344)
- [Pytorch tutorial for image classification](https://www.kaggle.com/hinepo/pytorch-tutorial-cv-99-67-lb-99-26) - @hinepo
- [Ensemble for image classification](https://www.kaggle.com/hinepo/ensemble-by-probabilities-lb-99-43) - @hinepo
- [Transfer learning for CV guide - kaggle learn](https://www.kaggle.com/learn-guide/transfer-learning-cv)
- [Dev Ideas - Treinamentos sobre visão computacional](https://csvisaocomputacional.com.br/)

### Pytorch image models (timm)

O que o Hugging Face é para NLP é análogo ao que a biblioteca timm é para computer vision: um ecossistema open source, consolidado e no estado da arte, que disponibiliza uma API simples e unificada para uso de modelos, além de centenas de excelentes modelos multi-propósito (multi-task, general purpose models), já pré-treinados durante semanas em GPUs e TPUs de dezenas de milhares de dólares, todos prontos para usarmos apenas adicionando uma última camada na rede neural para atender ao nosso task/problema. Isso se chama feature extraction, e evita que tenhamos que treinar esses modelos gigantes from scratch.

- [timm tutorial](https://towardsdatascience.com/getting-started-with-pytorch-image-models-timm-a-practitioners-guide-4e77b4bf9055)
- [timm: pytorch image models](https://github.com/rwightman/pytorch-image-models/tree/master/timm)
- [timm: getting started](https://rwightman.github.io/pytorch-image-models/)
- [timm: overview](https://fastai.github.io/timmdocs/)
- [Pytorch/timm tutorial for transfer learning](https://www.kaggle.com/hinepo/transfer-learning-with-timm-models-and-pytorch) - @hinepo

### Papers

- [AlexNet - paper sep/2012](https://proceedings.neurips.cc/paper/2012/file/c399862d3b9d6b76c8436e924a68c45b-Paper.pdf)
- [ResNet - paper dec/2015](https://arxiv.org/pdf/1512.03385.pdf)
- [ResNeXt - paper apr/2017](https://arxiv.org/pdf/1611.05431.pdf)
- [Squeeze-and-Excitation Networks - SE ResNet - paper may/2019](https://arxiv.org/pdf/1709.01507v4.pdf)
- [Self-Training with Noisy Student - paper jun/2020](https://arxiv.org/pdf/1911.04252.pdf)
- [EfficientNet - paper sep/2020](https://arxiv.org/pdf/1905.11946.pdf)
- [Meta Pseudo Labels - paper mar/2021](https://arxiv.org/pdf/2003.10580v4.pdf)
- [A ConvNet fot the 2020s - ConvNext paper jan/2022](https://arxiv.org/pdf/2201.03545.pdf)

### Vision Transformer (ViT) and others

- [ViT - 5 min video](https://www.youtube.com/watch?v=DVoHvmww2lQ)
- [ViT - Hugging Face](https://huggingface.co/docs/transformers/model_doc/vit)
- [ViT exemplo - tranformer library](https://www.youtube.com/watch?app=desktop&v=Bjp7hebC67E)
- [ViT exemplo - timm library](https://www.youtube.com/watch?v=ovB0ddFtzzA)
- [Swin Transformer](https://www.youtube.com/watch?v=SndHALawoag)

# RecSys

- [OTTO competition](https://www.kaggle.com/competitions/otto-recommender-system) é o melhor e mais completo material sobre sistemas de recomendação, incluindo muitas abordagens, baselines, códigos, discussões e soluções.
- [EDA for e-commerce RecSys: Matrix Factorization + TSNE](https://www.kaggle.com/code/cdeotte/user-eda-with-rapids-tsne-and-matrix-factorization)
- [Digraph - User Behavior](https://www.kaggle.com/code/hinepo/eda-user-behavior-via-digraph-martech)
- [Mastering Recommender Systems | Grandmaster Series E8](https://www.youtube.com/watch?v=7woi3bRa4BU)
- [Facebook recommender systems](https://transparency.fb.com/features/explaining-ranking/)

### Recommendation systems basic concepts:

- [An introduction to Recommender Systems -  a thread 🧵 - Radek Osmulski](https://twitter.com/radekosmulski/status/1631513789484126208)
- [Recommendation Systems for Large Datasets](https://www.kaggle.com/competitions/otto-recommender-system/discussion/364721)
- [Recommender Systems in Python 101](https://www.kaggle.com/code/gspmoreira/recommender-systems-in-python-101/notebook)
- [Teoria - Khrish Naik](https://www.youtube.com/watch?v=EjOlN6uVBOg)
- [Prática - Khrish Naik](https://www.youtube.com/watch?v=R64Lh1Qwl_0)
- [RecSys with KNN - Khrish Naik](https://www.youtube.com/watch?v=kccT0FVK6OY)
- [Outros vídeos da playlist de RecSys - Krish Naik](https://www.youtube.com/watch?v=_hf_y-_sj5Y&list=PLZoTAELRMXVN7QGpcuN-Vg35Hgjp3htvi)
- [Arquitetura de Sistemas de recomendação - Mario Filho](https://www.youtube.com/watch?v=2LbZRaRfGqk&t=2688s)
- [Etsy RecSys: from many GBDT models to one NN](https://www.etsy.com/codeascraft/how-we-built-a-multi-task-canonical-ranker-for-recommendations-at-etsy)

### Validating a recommendation model:

- [How to validate a recommendation model](https://www.kaggle.com/code/radek1/a-robust-local-validation-framework)
- [About validation](https://www.kaggle.com/competitions/otto-recommender-system/discussion/364991)
- [About spliting the data](https://github.com/otto-de/recsys-dataset)
- [Script to split the data - exemplo](https://github.com/otto-de/recsys-dataset/blob/main/src/testset.py)
- [H&M Personalized Fashion Recommendations - How To Setup Local CV](https://www.kaggle.com/competitions/h-and-m-personalized-fashion-recommendations/discussion/308919)

### Common Metrics

- [A top-down perspective on Recall@k](https://www.kaggle.com/competitions/otto-recommender-system/discussion/363874)
- [Recall@k Explained](https://www.kaggle.com/competitions/otto-recommender-system/discussion/363962)

### Algumas abordagens:

- Content based (item similarity / item colaborative filtering)
- User based (user similarity / user colaborative Filtering)
- [Word2Vec, FastText](https://www.kaggle.com/code/radek1/word2vec-how-to-training-and-submission)
  - train from scratch on item sequences
  - use [Annoy lib](https://github.com/spotify/annoy) to find nearest neighbors
- [Co-visitation matrix + heuristic re-ranker (rule-based ranker)](https://www.kaggle.com/code/cdeotte/candidate-rerank-model-lb-0-575)
- [LGBM Ranker](https://www.kaggle.com/code/radek1/polars-proof-of-concept-lgbm-ranker)
- [How To Build a GBT Ranker Model)](https://www.kaggle.com/competitions/otto-recommender-system/discussion/370210)
  - user features
  - item features
  - user-item interactions features
  - co-visitation features
- [Matrix Factorization](https://www.kaggle.com/code/radek1/matrix-factorization-pytorch-merlin-dataloader) & [Matrix Factorization with GPU](https://www.kaggle.com/code/cpmpml/matrix-factorization-with-gpu)
- Autoencoder
- LSTM
- Transformers
- GNN (Graph Neural Networks)
- [RecBole lib](https://recbole.io/docs/)

# Marketing Mix Modeling
- [Curso 2h](https://www.linkedin.com/learning/marketing-attribution-and-mix-modeling/measuring-marketing-performance?resume=false)
  - Multi-touch Attribution Models
  - Marketing Mix Modeling
  - Incrementality and A/B testing
- [MMM models using tree-based ensembles and SHAP](https://towardsdatascience.com/improving-marketing-mix-modeling-using-machine-learning-approaches-25ea4cd6994b)
- [Lightweightmmm (Bayesian) Marketing Mix Modeling](https://github.com/google/lightweight_mmm) and [Lightweightmmm Models](https://lightweight-mmm.readthedocs.io/en/latest/models.html)
- [Artigos de MMM do Mario Filho](https://forecastegy.com/tags/marketing-mix-modeling/)
  - [Marketing Mix Model With LightweightMMM (Python)](https://forecastegy.com/posts/how-to-create-a-marketing-mix-model-with-lightweightmmm/)
  - [What I Learned Watching 7 Hours of Meta's Marketing Mix Modeling Summits](https://forecastegy.com/posts/marketing-mix-modeling-meta-summit/)
  - [Adstock in Marketing Mix Modeling](https://forecastegy.com/posts/adstock-in-marketing-mix-modeling/)
  - [Case Study: Marketing Mix Modeling with Social Media And Google](https://forecastegy.com/posts/marketing-mix-modeling-case-study-social-media-google/)
  - [Don't Let Multicollinearity Mess Up Your Marketing Mix Model](https://forecastegy.com/posts/dont-let-multicollinearity-mess-up-your-marketing-mix-model/)

# Survival Analysis
- [Lifelines - Introduction to survival analysis](https://lifelines.readthedocs.io/en/latest/Survival%20Analysis%20intro.html)
- [Lifelines - Survival regression](https://lifelines.readthedocs.io/en/latest/Survival%20Regression.html)
- [Lifelines - code example](https://github.com/anurag-code/Survival-Analysis-Intuition-Implementation-in-Python/blob/master/Survival%20Analysis%20-%20Quick%20Implementation.ipynb)
- [xgbse - Loft](https://medium.com/loftbr/xgbse-an%C3%A1lise-de-sobreviv%C3%AAncia-robusta-e-eficiente-em-python-b3064da6a1e8)
- [Predictive Maintenance Problems - video](https://www.youtube.com/watch?v=WWeYJQwGTYA)
- [Survival regression with accelerated failure time model in XGBoost](https://arxiv.org/pdf/2006.04920) - paper aug/2021

# Deploy

- [Deploy pipelines: Feature, Training, Inference](https://www.linkedin.com/feed/update/urn:li:activity:7076483314697609216/)
- [Data Roles and Data Value Chain](https://www.linkedin.com/feed/update/urn:li:activity:7076423576031027201/)
- [4 Types of ML model deployment](https://twitter.com/Aurimas_Gr/status/1656959857205936129)
- [You don't need streaming to do real-time machine learning](https://www.linkedin.com/feed/update/urn:li:activity:6969739163927539712/?utm_source=share&utm_medium=member_android) - Chip Huyen
- [Fine-tune and Deploy models with Azure Machine Learning](https://www.youtube.com/watch?v=IQrILl5B0kw)
- [Dealing with Train-serve Skew in Real-time ML Models](https://building.nubank.com.br/dealing-with-train-serve-skew-in-real-time-ml-models-a-short-guide/)
- [A/B Testing with stats](https://www.kaggle.com/code/ekrembayar/a-b-testing-step-by-step-hypothesis-testing)
- [A/B Testing with bootstrap](https://www.kaggle.com/code/yufengsui/datacamp-project-mobile-games-a-b-testing/notebook)
- [Streamlit - Revolutionizing Data App Creation](https://pub.towardsai.net/streamlit-revolutionizing-data-app-creation-e269177d9112)
- [Use docker for deployment, not for development](https://twitter.com/radekosmulski/status/1615856695879798784)
- [Exemplo de Documentação de Modelos - Model Cards for Deployed models](https://meta.m.wikimedia.org/wiki/Machine_learning_models)

### Capítulos do livro "Designing Machine Learning Systemns - Chip Huyen":

- 7 - Model Deployment and Prediction Service
  - Offline evaluation
  - Backtesting
  - Batch x Online
  - Cloud x Edge
- 8 - Data Distribution Shifts and Monitoring
  - How to monitor raw inputs, features, predictions, metrics
- 9 - Continual Learning and Test in Production
  - Online evaluation
  - Shadow Deployment
  - A/B Testing
  - Canary release
  - Interleaving experiments
  - Bandits

### Deploying supervised machine learning models

Deploying is an intervention and humans are agents that react to the predictions. That is the point of most models.

- [Performative prediction](https://twitter.com/ChristophMolnar/status/1569644089724764160)

# MLOps

Intro:
- [Como é o trabalho de um ML Enginner (MLE)?](https://www.youtube.com/watch?v=xCcrMhwzhpI)
- [O que é Feature Store](https://medium.com/data-hackers/o-que-%C3%A9-feature-store-29e35c60dada) - Muito importante
- Capítulo "10 - Infrastructure and Tooling for MLOps" do livro "Designing Machine Learning Systemns - Chip Huyen"

Model Store:
- [MLflow](https://www.youtube.com/watch?app=desktop&v=859OxXrt_TI)
- [MLflow on Databricks](https://www.youtube.com/watch?app=desktop&v=nx3yFzx_nHI&t=1260s)
- [Weights and biases](https://www.youtube.com/watch?v=krWjJcW80_A)

Websites:
- [Made With ML](https://madewithml.com/)
- [Machine Learning Operations](https://ml-ops.org/)
- [MyMLOps - Build your MLOps stack](https://mymlops.com/)

Papers:
- [MLOps: Overview, Definition, and Architecture - paper mai/2022](https://arxiv.org/ftp/arxiv/papers/2205/2205.02302.pdf)
- [Operationalizing Machine Learning: An Interview Study - paper sep/2022](https://arxiv.org/pdf/2209.09125.pdf)

Tools & References:
- [A curated list of awesome MLOps tools - kelvins](https://github.com/kelvins/awesome-mlops)
- [A curated list of references for MLOps - visenger](https://github.com/visenger/awesome-mlops)

# Tech Management
- [How Every "Agile" Sprint Turns into a CHAOTIC MESS](https://www.youtube.com/watch?v=q0bgSuGf5HE)
- [Tech Management at Meta](https://www.jointaro.com/question/tB7borE2AWb1KcuFEcon/is-sprint-planning-and-agile-not-a-thing-at-meta/)
- [Scrum vs Data Science](https://twitter.com/search?q=%40marktenenholtz%20scrum&src=typed_query)
- [Agile, Software Engineering, Developers](https://www.youtube.com/watch?v=gSVBWvoNJ-s)

### About Pair Programming:

Para o desenvolvedor e/ou cientista de dados, a melhor forma de programar e desenvolver um projeto é via Pair Programming. E Pair Programming é melhor via trabalho remoto.

- [Pair Programming at OpenAI and Google](https://twitter.com/_jasonwei/status/1702392357252211070?t=5QR0Z2cTDH_tsIcZOfLEHw&s=08)
  - [Phillip Singer opinion (Kaggle rank 1)](https://twitter.com/ph_singer/status/1702430003823267951)
  - [Greg Brockman opinion (OpenAI President)](https://twitter.com/gdb/status/1728200551849136594)
  - [Jeremy Howard opinion - remote pair programming](https://twitter.com/jeremyphoward/status/1728878957024755986)
  - [Advantages](https://twitter.com/tomcocobrico/status/1728373956980195446)
  - [Pair Programming Advantages - video](https://www.youtube.com/watch?v=E4cg5mmvpwo)

# Vibe Coding

Fun section. Don't take it too seriously. Good for throwaway weekend projects.

- ["Vibe coding"](https://x.com/karpathy/status/1886192184808149383) - Karpathy
- [How to Code with AI in 2025 (For Non-Coders)](https://www.youtube.com/watch?v=zSlkAO9jB8I) - 13 min - Rob Mulla
- [Perigos do Vibe Coding](https://www.linkedin.com/posts/andre-albuquerque_good-product-thinkers-and-good-engineers-activity-7308108920844087298-8VPa/?utm_source=share&utm_medium=member_desktop&rcm=ACoAADGdJpoBgFh12xUvPZJ_GJ3LDaC8Pz05JWQ) e [meme](https://x.com/_lhermann/status/1903006653064024296)
- [Vibe coding: backend, ui, db](https://x.com/SullyOmarr/status/1893757471799308321?t=zitHrQzpBSAI8VcLKu301g&s=08) - 10 minutes
- [10 examples of games built with Vibe Coding](https://x.com/minchoi/status/1903089440886583672)
- [Building a NES game](https://www.youtube.com/watch?v=Z90XGBfvr0c) - 30 min
- [App in < 1h with Cursor](https://x.com/Hesamation/status/1894703277612126560?t=O5XWyXmkZIVmNT51d11cGg&s=08) - 1h
- [AI Does all the Coding - What could possibly go wrong? - Cursor Agent](https://www.youtube.com/live/YtY8-QRcHjg) - 2h - Rob Mulla

# Youtube channels

Abaixo alguns canais nos quais acho válido se inscrever e acompanhar os conteúdos publicados.

- [Abhishek Thakur](https://www.youtube.com/c/AbhishekThakurAbhi)
- [AI Coffee Break with Letitia](https://www.youtube.com/c/AICoffeeBreak)
- [Chai Time Data Science](https://www.youtube.com/c/ChaiTimeDataScience)
- [Medallion Data Science](https://www.youtube.com/channel/UCxladMszXan-jfgzyeIMyvw/videos)
- [Yannic Kilcher](https://www.youtube.com/@YannicKilcher)
- [Krish Naik](https://www.youtube.com/user/krishnaik06)
- [Underfitted](https://www.youtube.com/c/Santiagox0)
- [Andrej Karpathy](https://www.youtube.com/c/AndrejKarpathy/videos)
- [BI Elite](https://www.youtube.com/c/BIElite)

# Perfis no twitter

Algumas sugestões:

- [Chip Huyen](https://twitter.com/chipro)
- [Mark Tenenholtz](https://twitter.com/marktenenholtz)
- [Santiago](https://twitter.com/svpino)
- [François Chollet](https://twitter.com/fchollet)
- [Bojan Tunguz](https://twitter.com/tunguz)
- [abhishek](https://twitter.com/abhi1thakur)
- [Aurimas Griciūnas](https://twitter.com/Aurimas_Gr)
- [Konrad Banachewicz](https://twitter.com/tng_konrad)
- [JFPuget](https://twitter.com/JFPuget)
- [Chris Deotte](https://twitter.com/ChrisDeotte)
- [Goku Mohandas](https://twitter.com/GokuMohandas)
- [Yann Lecun](https://twitter.com/ylecun)
- [Andrew Ng](https://twitter.com/AndrewYNg)
- [James Briggs](https://twitter.com/jamescalam)
- [Jim Fan](https://twitter.com/DrJimFan)
- [Owain Evans](https://twitter.com/OwainEvans_UK)
- [Yam Peleg](https://twitter.com/Yampeleg)
- [arXiv abstract](https://twitter.com/arxivabs)
- [Sanyam Bhutani](https://twitter.com/bhutanisanyam1)
- [Ross Wightman](https://twitter.com/wightmanr)
- [Hugging Face](https://twitter.com/huggingface)
- [Harrison Chase](https://twitter.com/hwchase17)
- [AI Coffee Break with Letitia Parcalabescu](https://twitter.com/AICoffeeBreak)
- [Andrej Karpathy](https://twitter.com/karpathy)
- [Philipp Singer](https://twitter.com/ph_singer)
- [Jeremy Howard](https://twitter.com/jeremyphoward)
- [Ultralytics](https://twitter.com/ultralytics)
- [Harrison Kinsley](https://twitter.com/Sentdex)
- [Martin Henze (Heads or Tails)](https://twitter.com/heads0rtai1s)
- [Anthony Goldbloom](https://twitter.com/antgoldbloom)
- [Machine Learning Mastery](https://twitter.com/TeachTheMachine)

Trilha-de-Estudos-de-Data-Science

Trilha-de-Estudos-de-Data-Science 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

Trilha-de-Estudos-de-Data-Science

Trilha-de-Estudos-de-Data-Science 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

Download README