KO
|
EN
gitlite — search
Search
#python
#java
#python3
#arduino
#golang
#machine-learning
#rust
#html
#flask
#javascript
#seismology
#nodejs
perseus
★ 10
Open GitHub ↗
No description available.
Download README (.md)
Explore Similar Repositories
Hachiko
:
Simple tool for creating pitch timeline
Spring-Boot-RESTFul-APIs
:
Spring Boot RESTFul APIs Course Repository
scrape-yt
:
Simple lib to scrape information from youtube such as search results, video information, related videos, playlist information and up next video
f_service
:
Simpler, safer and more composable operations
more_todo
:
A todo app to learn Moor.
// repository documentation
Was this content helpful?
★ 0
(0 ratings)
Select Rating:
★
★
★
★
★
Submit Feedback
Recent Feedback
×
Download README
Do you want to download the
README.md
file for
perseus
?
Download (.md)
# PERSEUS: Characterizing Performance and Cost of Multi-Tenant Serving for CNN Models ## About This repo is for the short paper [PERSEUS: Characterizing Performance and Cost of Multi-Tenant Serving for CNN Models](https://arxiv.org/abs/1912.02322) in proceedings of IC2E 2020. In this paper, we looked at the problem of efficiency and cost saving deep learning inference in the cloud environment. More concretely, we tackled the problem using multi-tenant model serving -- instead of having GPU servers hosting one model dedicately, we serve multiple models on individual GPU servers, subject to the GPU memory capacity. In doing so, we improved the utilization of hardware resources, especially GPU. To achieve this task, we built a measurement framework PERSEUS to characterize and measure the performence and cost trade-offs doing multi-tenant model serving. #### Highlight - We evaluated multi-tenant model serving using PERSEUS on several metrics such as inference throughput, monetary cost, and GPU utilization. We showed that multi-tenant serving can lead to up to 12% cost reduction, while maintaining the SLA requirement of model serving. - We identified several potential improvements from the deep learning framework's perspective, to provide better support for serving models, especially on CPUs. <div align="center"><img src="./data/img/multi_peak_throughput.png" width="400" height="250"></div> **Fig 1. Throughput comparison measured of dedicated serving vs. multi-tenant serving.** <div align="center"><img src="./data/img/multi.png" width="400" height="250"></div> **Fig 2. Monetary saving with multi-tenant serving.** ## How to use the code Please see the instructions in the individual modules in `code` folder. ## Citation If you would like to cite the paper, please cite it as: ```bibtex @article{lemay2019perseus, title={Perseus: Characterizing Performance and Cost of Multi-Tenant Serving for CNN Models}, author={Matthew LeMay and Shijian Li and Tian Guo}, year={2019}, eprint={1912.02322}, archivePrefix={arXiv}, primaryClass={cs.DC} } ``` ## Acknowledgement We would like to thank National Science Foundation grants #1755659 and #1815619, and Google Cloud Platform Research credits. ## Contact More project information can be found in our lab's [project site](https://cake-lab.github.io/projects/). - Mattew LeMay [mlemay@wpi.edu](mlemay@wpi.edu) - Shijian Li [sli8@wpi.edu](sli8@wpi.edu) - Tian Guo [tian@wpi.edu](tian@wpi.edu)
