Awesome-CloudComputing-Datasets

KO | EN

gitlite — search

#python #javascript #android #golang #python3 #react #github-actions #typescript #game #java #cli #ruby

Awesome-CloudComputing-Datasets

★ 33 Open GitHub ↗

No description available.

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

## NOTE:
**This work is developed by the Lab of Professor Weiwei Lin (linww@scut.edu.cn), School of Computer Science and Engineering, South China University of Technology, and Pengcheng Laboratory. The co-authors include Guozhi Liu (csliuguozhi@mail.scut.edu.cn), Haotong Zhang (hoyt.zhang77@gmail.com), Xiaoxuan Luo (714197010@qq.com), Jianpeng Lin (csjianpenglin@mail.scut.edu.cn), etc.**

# Awesome-CloudComputing-Datasets

**Dataset information format：**

```
- Dataset name  Release Time | Public or Not| Paper | Dataset |
  - Publisher:
  - Size:
  - Machines:
  - Duration:
  - Collection Methodology:
```

## Host Cluster Traces
- **Google Power Data ($${\color{green}\text{\textbf{Power}}}$$)  2024 | Yes | [Paper](https://dl.acm.org/doi/abs/10.1145/3373376.3378533) | [Dataset](https://github.com/google/cluster-data/blob/master/PowerData2019.md)**
  - Publisher: Google
  - Size: 45.4KB
  - Machines: 57
  - Duration: 1 month
  - Collection Methodology: Real-world
 
- **Azure LLM Inference Dataset ($${\color{purple}\text{\textbf{LLMs Inference}}}$$) 2024 | Yes | [Paper](https://ieeexplore.ieee.org/abstract/document/10609649) | [Dataset](https://github.com/Azure/AzurePublicDataset/blob/master/AzureLLMInferenceDataset2024.md)**
  - Publisher: Microsoft
  - Size: 313KB or 702KB
  - Machines: -
  - Duration: 1 day or 9 days
  - Collection Methodology: Real-world

- **Acme ($${\color{red}\text{\textbf{GPU}}}$$， $${\color{purple}\text{\textbf{LLMs Pretrain, Inference, and SFT}}}$$) 2023 | Yes | [Paper](https://www.usenix.org/conference/nsdi24/presentation/hu) | [Dataset](https://github.com/InternLM/AcmeTrace)**
  - Publisher: Google
  - Size: 80GB
  - Machines: 588
  - Duration: 6 months
  - Collection Methodology: Real-world
 
- **Alibaba GPU Traces ($${\color{red}\text{\textbf{GPU}}}$$) 2020 | Yes | [Paper](https://www.usenix.org/conference/nsdi22/presentation/weng) | [Dataset](https://github.com/alibaba/clusterdata/tree/master/cluster-trace-gpu-v2020)**
  - Publisher: Alibaba
  - Size: 1.36GB
  - Machines: 1800
  - Duration: 2 months
  - Collection Methodology: Real-world

- **Helios ($${\color{red}\text{\textbf{GPU}}}$$)  2020 | Yes | [Paper](https://dl.acm.org/doi/abs/10.1145/3458817.3476223) | [Dataset](https://github.com/S-Lab-System-Group/HeliosData)**
  - Publisher: SenseTime
  - Size: 343MB
  - Machines: 802
  - Duration: 6 months
  - Collection Methodology: Real-world
    
- **Marconi100 ($${\color{green}\text{\textbf{Power}}}$$) 2020 | Yes | [Paper](https://dl.acm.org/doi/abs/10.1145/3528416.3530867) | [Dataset](https://zenodo.org/search?q=Datacenter&l=list&p=1&s=10&sort=bestmatch)**
  - Publisher: Italian National Supercomputing Center
  - Size: 33.4GB
  - Machines: 960
  - Duration: 9 months
  - Collection Methodology: Real-world
 
- **Stadia Cloud Gaming Dataset  2020 | Yes | [Paper](https://www.sciencedirect.com/science/article/pii/S0140366422000810) | [Dataset](https://github.com/wn-upf/Stadia_cloud_gaming_dataset_2020)**
  - Publisher: Marc Carrascosa
  - Size: 153MB
  - Machines: -
  - Duration: 5 months
  - Collection Methodology: Real-world

- **SURFsara ($${\color{green}\text{\textbf{Power}}}$$) 2019 | Yes | [Paper](https://arxiv.org/abs/2107.11832) | [Dataset](https://github.com/sara-nl/SURFace)**
  - Publisher: SURFsara
  - Size: 41.3GB
  - Machines: 5
  - Duration: 8 months
  - Collection Methodology: Real-world

- **Google Cluster Data V1  2009 | Yes | [Paper](https://www.sciencedirect.com/science/article/pii/S1877050920309820) | [Dataset](https://github.com/google/cluster-data/blob/master/TraceVersion1.md)**
  - Publisher: Google
  - Size: 29.8MB
  - Machines: -
  - Duration: 7 hours
  - Collection Methodology: Real-world
    
- **Google Cluster Data V2  2011 | Yes | [Paper](https://ieeexplore.ieee.org/abstract/document/6529276) | [Dataset](https://github.com/google/cluster-data/blob/master/ClusterData2011_2.md)**
  - Publisher: Google
  - Size: 186GB
  - Machines: 12583
  - Duration: 29 days
  - Collection Methodology: Real-world

- **Google Cluster Data V3  2019 | Yes | [Paper](https://onlinelibrary.wiley.com/doi/full/10.1002/cpe.7559) | [Dataset](https://github.com/google/cluster-data/blob/master/ClusterData2019.md)**
  - Publisher: Google
  - Size: 2.4TB
  - Machines: 96000
  - Duration: 1 month
  - Collection Methodology: Real-world
 
- **SPEC Cloud Iaas  2018 | No | [Paper](https://ieeexplore.ieee.org/abstract/document/8758926) | [Dataset](https://www.spec.org/cloud_iaas2018/)**
  - Publisher: SPEC
  - Size: -
  - Machines: -
  - Duration: 1 year
  - Collection Methodology: Real-world

- **Alibaba Cluster Trace V1  2017 | Yes | [Paper](https://ieeexplore.ieee.org/abstract/document/10092039) | [Dataset](https://github.com/alibaba/clusterdata/tree/master/cluster-trace-v2017)**
  - Publisher: Alibaba
  - Size: 232MB
  - Machines: 1300
  - Duration: 12 hours
  - Collection Methodology: Real-world
    
- **Alibaba Cluster Trace V1  2018 | Yes | [Paper](https://ieeexplore.ieee.org/abstract/document/10092039) | [Dataset](https://github.com/alibaba/clusterdata/tree/master/cluster-trace-v2018)**
  - Publisher: Alibaba
  - Size: 98GB
  - Machines: 4000
  - Duration: 8 days
  - Collection Methodology: Real-world

- **Dionatrafk  2018 | Yes | [Paper](https://ieeexplore.ieee.org/abstract/document/8671604) | [Dataset](https://github.com/dionatrafk/workload_prediction)**
  - Publisher: Dionatr a F . Kirchoff
  - Size: 321KB
  - Machines: -
  - Duration: 1 month
  - Collection Methodology: Real-world

- **ATLAS Cluster Traces  2011, 2018, 2016 | Yes | [Paper](https://www.usenix.org/system/files/login/articles/login_winter18_07_amvrosiadis.pdf) | [Dataset](https://ftp.pdl.cmu.edu/pub/datasets/ATLAS/)**
  - Publisher: Carnegie Mellon University
  - Size: 16.7MB
  - Machines: 1600
  - Duration: 5 years
  - Collection Methodology: Real-world
 
- **Philly ($${\color{red}\text{\textbf{GPU}}}$$)  2017 | Yes | [Paper](https://www.usenix.org/conference/atc19/presentation/jeon) | [Dataset](https://github.com/msr-fiddle/philly-traces)**
  - Publisher: Microsoft
  - Size: 6.6GB
  - Machines: 552
  - Duration: 3 months
  - Collection Methodology: Real-world

- **SPEC CPU  2006, 2017 | No | [Paper](https://ieeexplore.ieee.org/abstract/document/6809346) | [Dataset](https://www.spec.org/cpu2017/)**
  - Publisher: SPEC
  - Size: -
  - Machines: -
  - Duration: 1 year
  - Collection Methodology: Real-world

- **Autoscale Analyser  2015 | Yes | [Paper](https://ieeexplore.ieee.org/abstract/document/7820029/) | [Dataset](https://github.com/hsbhathiya/AutoscaleAnalyser/tree/master/datasets/cloud_traces)**
  - Publisher: H.S. Bhathiya
  - Size: 1.086MB
  - Machines: 2
  - Duration: 47 hours
  - Collection Methodology: Real-world
 
- **Intel Netbatch logs  2012 | Yes | [Paper](https://www.sciencedirect.com/science/article/pii/S1568494619307215) | [Dataset](https://www.cs.huji.ac.il/labs/parallel/workload/l_intel_netbatch/index.html)**
  - Publisher: Ohad Shai
  - Size: 2.53GB
  - Machines: -
  - Duration: 1 month
  - Collection Methodology: Real-world
 
- **OpenCloud Hadoop Workload  2010 | Yes | [Paper](https://dl.acm.org/doi/abs/10.1145/3409772) | [Dataset](https://ftp.pdl.cmu.edu/pub/datasets/hla/dataset.html)**
  - Publisher: Parallel Data Lab
  - Size: -
  - Machines: 64
  - Duration: 20 months
  - Collection Methodology: Real-world

- **Yahoo Webscope Dataset  2010 | Yes | [Paper](https://ieeexplore.ieee.org/abstract/document/7904276) | [Dataset](https://webscope.sandbox.yahoo.com/catalog.php?datatype=s&guccounter=1)**
  - Publisher: Yahoo
  - Size: 33KB
  - Machines: -
  - Duration: -
  - Collection Methodology: Real-world
 
- **SWIM Workload  2009, 2010 | Yes | [Paper](https://ieeexplore.ieee.org/abstract/document/6005383) | [Dataset](https://github.com/SWIMProjectUCB/SWIM/wiki/Workloads-repository)**
  - Publisher: Facebook
  - Size: 2.14MB
  - Machines: 3000
  - Duration: 1 day
  - Collection Methodology: Real-world

- **SPEC power_ssj2008 ($${\color{green}\text{\textbf{Power}}}$$) 2008 | No | [Paper](https://www.sciencedirect.com/science/article/pii/S0167739X16306872) | [Dataset](https://www.spec.org/power_ssj2008/results/)**
  - Publisher: SPEC
  - Size: -
  - Machines: -
  - Duration: 1 year
  - Collection Methodology: Real-world

- **GWA-T-4 AuverGrid  2006 | Yes | [Paper](https://www.sciencedirect.com/science/article/pii/S1084804515001241) | [Dataset](http://gwa.ewi.tudelft.nl/datasets/gwa-t-4-auvergrid)**
  - Publisher: Delft University of Technology
  - Size: 13MB
  - Machines: 5
  - Duration: -
  - Collection Methodology: Real-world
 
- **GWA-T-1 DAS2  2005 | Yes | [Paper](https://link.springer.com/article/10.1007/s11227-023-05163-w) | [Dataset](http://gwa.ewi.tudelft.nl/datasets/gwa-t-1-das2)**
  - Publisher: Delft University of Technology
  - Size: 35MB
  - Machines: 5
  - Duration: -
  - Collection Methodology: Real-world

- **GWA-T-10 SHARCNet  2005 | Yes | [Paper](https://www.sciencedirect.com/science/article/pii/S1084804515001241) | [Dataset](http://gwa.ewi.tudelft.nl/datasets/gwa-t-10-sharcnet)**
  - Publisher: Delft University of Technology
  - Size: 33MB
  - Machines: 10
  - Duration: -
  - Collection Methodology: Real-world

- **GWA-T-3 NorduGrid  2003 | Yes | [Paper](https://www.sciencedirect.com/science/article/pii/S0167739X16306872) | [Dataset](http://gwa.ewi.tudelft.nl/datasets/gwa-t-3-nordugrid)**
  - Publisher: Delft University of Technology
  - Size: 20MB
  - Machines: 75
  - Duration: -
  - Collection Methodology: Real-world

- **Parallel Workloads Archive  2005 | Yes | [Paper](https://link.springer.com/article/10.1007/s11227-023-05163-w) | [Dataset](https://www.cs.huji.ac.il/labs/parallel/workload/logs.html)**
  - Publisher: DG Feitelson
  - Size: -
  - Machines: -
  - Duration: -
  - Collection Methodology: Real-world

## VM Cluster Traces
- **Alibaba MicroServices Traces V1  2021 | Yes | [Paper](https://dl.acm.org/doi/abs/10.1145/3472883.3487003) | [Dataset](https://github.com/alibaba/clusterdata/tree/master/cluster-trace-microservices-v2021)**
  - Publisher: Alibaba
  - Size: 61.1GB
  - Machines: 10000
  - Duration: 12 hours
  - Collection Methodology: Real-world

- **Alibaba MicroServices Traces V2  2022 | Yes | [Paper](https://dl.acm.org/doi/abs/10.1145/3472883.3487003) | [Dataset](https://github.com/alibaba/clusterdata/tree/master/cluster-trace-microservices-v2022)**
  - Publisher: Alibaba
  - Size: 2TB
  - Machines: 40000
  - Duration: 13 days
  - Collection Methodology: Real-world

- **CERIT-SC Workloads  2016 | Yes | [Paper](https://link.springer.com/chapter/10.1007/978-3-319-77398-8_2) | [Dataset](https://jsspp.org/workload/index.php)**
  - Publisher: Dalibor Klusacek
  - Size: 11MB
  - Machines: -
  - Duration: 1 year
  - Collection Methodology: Real-world
 
- **OpenNebula Virtual Machine Profiling Dataset  2021 | Yes | [Paper](https://link.springer.com/chapter/10.1007/978-3-319-77398-8_2) | [Dataset](https://ieee-dataport.org/documents/opennebula-virtual-machine-profiling-intrusion-detection-system#files)**
  - Publisher: Prasad Purnaye
  - Size: 2.6MB
  - Machines: 6
  - Duration: 63 hours
  - Collection Methodology: Real-world
 
- **Fruktus  2020 | Yes | [Paper](https://link.springer.com/article/10.1007/s10115-022-01721-5) | [Dataset](https://www.cs.huji.ac.il/labs/parallel/workload/logs.html)**
  - Publisher: Piotr Nawrocki
  - Size: 176KB
  - Machines: -
  - Duration: 11 months
  - Collection Methodology: Real-world
 
- **Chameleon Cloud traces  2017 | Yes | [Paper](https://www.researchgate.net/profile/Zhuangwei-Kang/publication/356981403_Chameleon_and_HTC_a_Match_Made_in_Heaven/links/6207d9eb634ff774f4c6c19d/Chameleon-and-HTC-a-Match-Made-in-Heaven.pdf) | [Dataset](https://www.scienceclouds.org/cloud-traces)**
  - Publisher: Science Clouds
  - Size: 1.5GB
  - Machines: -
  - Duration: 5 years
  - Collection Methodology: Real-world

- **Azure Public Dataset  2017, 2019 | Yes | [Paper](https://dl.acm.org/doi/abs/10.1145/3477132.3483590) | [Dataset](https://github.com/Azure/AzurePublicDataset/tree/master)**
  - Publisher: Microsoft
  - Size: 117GB, 235GB
  - Machines: 2 million, 2.6 million
  - Duration: 30 days, 30 days
  - Collection Methodology: Real-world
 
- **Scout  2018 | Yes | [Paper](https://dl.acm.org/doi/abs/10.14778/3407790.3407845) | [Dataset](https://github.com/oxhead/scout)**
  - Publisher: Hsu Chin-Jung
  - Size: 587MB
  - Machines: -
  - Duration: 1 day
  - Collection Methodology: Real-world
 
- **IBM Docker Registry traces  2018 | Yes | [Paper](https://www.usenix.org/conference/fast18/presentation/anwar) | [Dataset](https://dssl.cs.vt.edu/drtp/)**
  - Publisher: Ali Anwar
  - Size: 1.5GB
  - Machines: -
  - Duration: 75 day
  - Collection Methodology: Real-world

- **Business Critical Workloads  2015 | Yes | [Paper](https://www.sciencedirect.com/science/article/pii/S131915782100094X) | [Dataset](http://gwa.ewi.tudelft.nl/datasets/Bitbrains)**
  - Publisher: Delft University of Technology
  - Size: 284MB
  - Machines: 1250
  - Duration: 1 month
  - Collection Methodology: Real-world
    
- **Planetlab VM traces  2011 | Yes | [Paper](https://ieeexplore.ieee.org/abstract/document/8301555) | [Dataset](https://github.com/beloglazov/planetlab-workload-traces)**
  - Publisher: Beloglazov Anton
  - Size: 5.2MB
  - Machines: -
  - Duration: 10 days
  - Collection Methodology: Real-world
    
## Web Traces
- **ISCX  2016 | Yes | [Paper](https://ieeexplore.ieee.org/abstract/document/8252132) | [Dataset](https://www.unb.ca/cic/datasets/index.html)**
  - Publisher: Canadian Institute for Cybersecurity
  - Size: 26.3MB
  - Machines: -
  - Duration: 1 year
  - Collection Methodology: Real-world
 
- **Amazon Resource Cost  2012 | Yes | [Paper](https://dl.acm.org/doi/abs/10.1145/2479871.2479908) | [Dataset](https://research.cs.queensu.ca/home/mian/index_files/Page485.htm)**
  - Publisher: Queen's University
  - Size: 87.1KB
  - Machines: -
  - Duration: 4 days
  - Collection Methodology: Real-world
 
- **Cloud Intrusion Detection Dataset  2012 | Yes | [Paper](https://dl.acm.org/doi/abs/10.1145/3297663.3309676) | [Dataset](http://groups.di.unipi.it/~hkholidy/projects/cidd/)**
  - Publisher: DARPA Intrusion Detection Evaluation Group of MIT Lincoln Laboratory
  - Size: 70MB
  - Machines: 45
  - Duration: 7 weeks
  - Collection Methodology: Real-world

- **Wikipedia Web Traces from WikiBench  2007 | No | [Paper](https://ieeexplore.ieee.org/abstract/document/9860236) | [Dataset](https://wikitech.wikimedia.org/wiki/Analytics/Archive/Data/Pagecounts-raw)**
  - Publisher: Wikipedia
  - Size: 5TB
  - Machines: -
  - Duration: 9 years
  - Collection Methodology: Real-world

- **KDD Cup 1999's Dataset  1998 | Yes | [Paper](https://dl.acm.org/doi/abs/10.1145/2479871.2479908) | [Dataset](https://kdd.ics.uci.edu/databases/kddcup99/kddcup99)**
  - Publisher: MIT Lincoln Labs
  - Size: 1.31MB
  - Machines: -
  - Duration: -
  - Collection Methodology: Real-world

- **NASA HTTP Traces  1995 | Yes | [Paper](https://onlinelibrary.wiley.com/doi/full/10.1002/dac.5634) | [Dataset](https://www.kaggle.com/datasets/adchatakora/nasa-http-access-logs/download?datasetVersionNumber=1)**
  - Publisher: Kennedy Space Center
  - Size: 20.7MB
  - Machines: -
  - Duration: 1 month
  - Collection Methodology: Real-world

Awesome-CloudComputing-Datasets

Explore Similar Repositories

Was this content helpful?

Recent Feedback

Download README