Lsglang

(★ 111)

Lsglang is a special extension of sglang that fully utilizes CPU and GPU computing resources with an efficient GPU parallel + NUMA parallel architecture, suitable for MOE model hybrid inference.

File Explorer

Download Latest Version (.zip)

Showing a partial file list — download the zip above to see everything.

- - forward-batch-init-new-purity.md
  - general-code-style.md
  - modify-component-must-read.md
  - no-dataclasses.md
  - no-getattr-defensive.md
  - schedule-batch-out-of-place-mutation.md
  - unit-test-admission.md
- devcontainer.json
- Dockerfile
- - 1-bug-report.yml
  - 2-feature-request.yml
  - 3-playground-verified-cell.yml
- - lychee-ci.toml
  - lychee.toml
- - _docker-build-and-publish.yml
  - _docker-cleanup-nightly.yml
  - _npu-pr-test-stage.yml
  - _npu-single-node-test-stage.yml
  - _pr-test-check-changes.yml
  - _pr-test-rust-ext-build.yml
  - _pr-test-sgl-kernel-build.yml
  - _pr-test-stage-cpu.yml
  - _pr-test-stage.yml
  - amd-aiter-scout.yml
  - amd-ci-job-monitor.yml
  - auto-tune.yml
  - bot-bump-docs-version.yml
  - bot-bump-flashinfer-version.yml
  - bot-bump-kernel-version-to-sglang.yml
  - bot-bump-kernel-version.yml
  - bot-bump-sglang-version.yml
  - bot-cherry-pick.yml
  - cancel-pr-workflows-on-close.yml
  - cancel-unfinished-pr-tests.yml
  - ci-auto-bisect.yml
  - ci-coverage-overview.yml
  - ci-failure-monitor.yml
  - ci-model-inventory.yml
  - close-inactive-issues.yml
  - close-stale-prs.yml
  - diffusion-ci-gt-gen-npu.yml
  - diffusion-ci-gt-gen.yml
  - full-test-npu.yml
  - labeler.yml
  - lint.yml
  - list-active-pr-runs.yml
  - nightly-72-gpu-gb200.yml
  - nightly-amd-mi355x-disagg.yml
  - nightly-experimental-sgl-router-docker.yml
  - nightly-link-check.yml
  - nightly-release-gateway.yml
  - nightly-test-amd-miles-rocm720.yml
  - nightly-test-amd-rocm720.yml
  - nightly-test-amd.yml
  - nightly-test-intel.yml
  - nightly-test-musa.yml
  - nightly-test-npu-e2e-multi-node.yml
  - nightly-test-npu-e2e-single-node.yml
  - nightly-test-npu.yml
  - nightly-test-nvidia.yml
  - open-pr-copy-from-oss.yml
  - open-pr-copy-to-oss.yml
  - patch-docker-dev.yml
  - pr-benchmark-rust.yml
  - pr-gate.yml
  - pr-states.yml
  - pr-test-amd-extra.yml
  - pr-test-amd-rocm720.yml
  - pr-test-amd.yml
  - pr-test-arm64.yml
  - pr-test-extra.yml
  - pr-test-jit-kernel.yml
  - pr-test-mlx.yml
  - pr-test-multimodal-gen.yml
  - pr-test-musa.yml
  - pr-test-npu.yml
  - pr-test-rust-exts.yml
  - pr-test-rust.yml
  - pr-test-sgl-kernel.yml
  - pr-test-sgl-router.yml
  - pr-test-xeon.yml
  - pr-test-xpu.yml
  - pr-test.yml
  - release-branch-cut.yml
  - release-docker-amd-miles-rocm700-nightly.yml
  - release-docker-amd-miles-rocm720-nightly.yml
  - release-docker-amd-nightly.yml
  - release-docker-amd-rocm720-nightly.yml
  - release-docker-amd-rocm7_15-nightly.yml
  - release-docker-amd.yml
  - release-docker-dev.yml
  - release-docker-gateway.yml
  - release-docker-intel-xpu-nightly.yml
  - release-docker-npu-nightly.yml
  - release-docker-npu.yml
  - release-docker-runtime.yml
  - release-docker-xeon.yml
  - release-docker.yml
  - release-pypi-gateway.yml
  - release-pypi-nightly.yml
  - release-pypi-pr.yml
  - release-pypi.yml
  - release-tag.yml
  - release-whl-deepep.yml
  - release-whl-deepgemm.yml
  - release-whl-kernel-xpu.yml
  - release-whl-kernel.yml
  - rerun-test.yml
  - retag-docker.yml
  - runner-utilization.yml
  - seed-rust-ext-cache.yml
  - slash-command-handler.yml
  - stress-test.yml
  - sync-lmsys-sglang-blogs.yml
  - trivy-scan-dev.yml
  - weekly-test-nvidia.yml
  - weekly-update-est-time.yml
  - xpu-ci-job-monitor.yml
- audit_permission.py
- CI_PERMISSIONS.json
- CODEOWNERS
- FOLDER_README.md
- labeler.yml
- MAINTAINER.md
- pull_request_template.md
- update_ci_permission.py
- logo.png
- logo.svg
- logo_square.png
- logo_square.svg
- - bench_sglang.py
  - README.md
- - bench_attention_sink_triton.py
- - bench_cutedsl_kda_decode.py
  - bench_fused_gate_cumsum.py
  - bench_gdn_decode.py
  - bench_gdn_prefill.py
  - bench_gdn_prefill_cutedsl.py
  - bench_gdn_qkv_split.py
  - bench_int8_checkpoint_reuse.py
  - bench_kda_decode.py
  - bench_kda_flashinfer_mtp.py
  - bench_kda_prefill_cutedsl.py
- - bench_segment_tracking.py
- - benchmark_rope_index.py
- - 405b_sglang.sh
  - 405b_trt.sh
  - 405b_vllm.sh
  - config.md
  - README.md
- - bench_sglang.py
  - convert_parquet_to_json.py
  - parquet_to_json.sh
  - README.md
- - README.md
- - benchmark_layernorm_gated.py
- - bench_sglang.py
  - README.md
- - bench_sglang.py
  - README.md
- - bench.sh
  - bench_client.py
  - bench_storage.py
  - bench_zerocopy.py
- - bench_hicache_write_back.py
  - bench_long_context.py
  - bench_mix.py
  - bench_mix.sh
  - bench_multiturn.py
  - bench_serving.py
  - bench_warm_cache.py
  - data_processing.py
  - download.sh
  - nextqa.py
  - perf.py
  - README.md
- - bench_input_ids_validator.py
- - bench_fused_gate_sigmoid_mul_add.py
  - bench_fused_rmsnorm_fp8_quant.py
  - bench_fused_sigmoid_mul.py
  - bench_paged_mqa_metadata.py
- - bench_hf_llava_bench.sh
  - bench_hf_mme.sh
  - bench_sglang.py
  - bench_sglang_mme.sh
  - download_images.py
  - questions.jsonl
  - README.md
- - launch_server.py
  - lora_bench.py
- - bench_hf.py
  - bench_sglang.py
  - download_data.sh
  - README.md
- - bench_hf.py
  - bench_sglang.py
  - data_utils.py
  - eval_utils.py
  - prompt_format.yaml
  - README.md
- - bench_sglang.py
  - eval_utils.py
  - generate_report.py
  - README.md
- - bench_embeddings.py
  - bench_score.py
  - util.py
- - bench_token_storage.py
- bench_adaptive_speculative.py
- - .zshrc
  - yank
- - apply_deepep_k3_patch.sh
  - kimi_k3_cu12.Dockerfile
  - kimi_k3_cu13.Dockerfile
- arm64.Dockerfile
- compose.yaml
- Dockerfile
- gateway.Dockerfile
- k8s-sglang-distributed-sts.yaml
- k8s-sglang-service.yaml
- npu.Dockerfile
- rocm.Dockerfile
- sagemaker.Dockerfile
- serve
- sgl-deep-ep.Dockerfile
- sgl-deep-gemm.Dockerfile
- sgl-router.Dockerfile
- xeon.Dockerfile
- xpu.Dockerfile
- - Autoregressive-benchmark-card.png
  - Autoregressive-card.png
  - Classification-card.png
  - Diffusion-benchmark-card.png
  - Diffusion-card.png
  - dLLM-card.png
  - Embedding-card.png
  - LLM-card.png
  - Omni-card.png
  - Rerank-card.png
  - Reward-card.png
  - VLA-card.png
  - VLM-card.png
- - intro.mdx
- - deepseek_v4_flash.ipynb
- - supported-models.mdx
- - Approach-Medium.woff2
  - Approach-Regular.woff2
- - dpa.png
- - logo.png
- - check_cookbook_configs.mjs
  - gen_redirects.py
  - update_lmsys_sglang_blogs.py
- .gitignore
- .mintignore
- AGENTS.md
- CONTRIBUTING.md
- custom.css
- docs.json
- favicon.png
- index.mdx
- LICENSE
- README.md
- - .gitignore
  - example_image.png
- - qwen3_reranker.jinja
  - qwen3_vl_reranker.jinja
  - tool_chat_template_deepseekr1.jinja
  - tool_chat_template_deepseekv3.jinja
  - tool_chat_template_deepseekv31.jinja
  - tool_chat_template_deepseekv32.jinja
  - tool_chat_template_llama3.1_json.jinja
  - tool_chat_template_llama4_pythonic.jinja
  - vision_template_sarashina_vl.jinja
- - update.py
- - docker-compose.yaml
  - opentelemetry.yaml
  - prometheus.yaml
  - README.md
  - tracing_compose.yaml
- - chain_of_verification.py
  - lora.py
  - multimodal_embedding.py
  - openai_chat_with_response_prefill.py
  - qwen3_vl_reranker.py
  - README.md
  - reward_model.py
  - vertex_predict.py
- - deploy_and_serve_endpoint.py
- - modelopt_quantize_and_export.py
- - .gitignore
  - BENCHMARKS.md
  - Cargo.toml
  - deny.toml
  - README.md
  - rust-toolchain.toml
- - __init__.py
  - _platform_stubs.py
  - bench_offline_throughput.py
  - bench_one_batch.py
  - bench_one_batch_server.py
  - bench_serving.py
  - check_env.py
  - compile_deep_gemm.py
  - launch_server.py
  - README.md
- MANIFEST.in
- pyproject.toml
- pyproject_cpu.toml
- pyproject_npu.toml
- pyproject_other.toml
- pyproject_xpu.toml
- setup.py
.codespellrc
.coveragerc
.dockerignore
.gitignore
.isort.cfg
.pre-commit-config.yaml
CODE_OF_CONDUCT.md
LICENSE
README.md
README_cn.md

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

Lsglang

File Explorer

Today's picks

Was this content helpful?

Recent Feedback

Lsglang

File Explorer

Today's picks

Was this content helpful?

Recent Feedback

Download README