apparate

★ 24 Open GitHub ↗

Artifact for "Apparate: Rethinking Early Exits to Tame Latency-Throughput Tensions in ML Serving" [SOSP '24]

Explore Similar Repositories

SpecEE : Repo for SpecEE: Accelerating Large Language Model Inference with Speculative Early Exiting (ISCA25)
early-exit-papers : A curated list of early exiting (LLM, CV, NLP, etc)
rbqe : Official implementation of "Early Exit or Not: Resource-Efficient Blind Quality Enhancement for Compressed Images" (ECCV'20).
for-in : Iterate over the enumerable properties of an object, and return an object with properties that evaluate to true from the callback. Exit early by returning `false`.
TIDE : Dynamic per-token early exit for LLM inference. Skip layers tokens don't need
// repository documentation