Open-source activity

Wei-Cheng (Wayne) Chiu ships upstream.

GPU kernels, LLM serving, and correctness fixes across the open-source inference stack.

74 prs 20 completed 54 in-flight

Showing 74 of 74 pull requests

[Bugfix] Return HTTP 500 for non-streaming generate errors

vllm-project/vllm 90.8K

#491952 weeks ago

[Bugfix] Skip Qwen3 deepstack buffers without vision

vllm-project/vllm 90.8K

#49397last month

[Bugfix] Fix CUDA graph profiling for long GDN warmups

vllm-project/vllm 90.8K

#50848last month

fix(planner): resolve aggregate workers by component type

ai-dynamo/dynamo 7.9K

#11578last month

fix: avoid SM120 MXFP8 MXFP4 autotune crash

flashinfer-ai/flashinfer 6.3K

#4066last month

Register wayne.is-a.dev

is-a-dev/register 11.1K

#441172 months ago

[JIT] Fuse FP8 KV-cache quantization into the store kernel

sgl-project/sglang 33.2K

#316522 months ago

[DeepSeek-V4][fix] Detect MTP routed-expert layout separately

NVIDIA/TensorRT-LLM 14.5K

#162762 months ago

fix(attention): correct SM120 NVFP4 qk_correction layout, row-sum reduction, and lse

flashinfer-ai/flashinfer 6.3K

#38382 months ago

test(moe): align EP-offset tests and docs with global expert ids (gh #3547)

flashinfer-ai/flashinfer 6.3K

#35912 months ago

[Quantization] Fix misleading NVFP4 Marlin linear warning on FP4-native GPUs

vllm-project/vllm 90.8K

#481992 months ago

fix: make the cutlass MoE gemm profiler MXFP8-aware (autotune crash on MXFP8xMXFP8)

flashinfer-ai/flashinfer 6.3K

#36142 months ago

[Bugfix] Handle E8M0 block scales in CUTLASS and Triton FP8 linear kernels

vllm-project/vllm 90.8K

#479882 months ago

fix(moe): return bf16 expert_weights from trtllm_fp4_block_scale_moe (#3595)

flashinfer-ai/flashinfer 6.3K

#36442 months ago

[None][fix] Exclude SM12x from bf16x2 FP8 cvt specialization in moe_a2a

NVIDIA/TensorRT-LLM 14.5K

#159342 months ago

fix(b12x): bucket MoE workspace allocation to stop kernel-cache thrash

flashinfer-ai/flashinfer 6.3K

#38362 months ago

feat(vllm): support NixlPushConnector in PD coordination and NIXL detection

ai-dynamo/dynamo 7.9K

#112192 months ago

fix(config): read num_nextn_predict_layers from the outer hf_config as fallback

sgl-project/sglang 33.2K

#300922 months ago

[MoE/b12x] Auto-select FLASHINFER_B12X NVFP4 MoE backend on SM120, reject EP deployments

vllm-project/vllm 90.8K

#475772 months ago

test(comm): order rank streams after input fills in dcp_alltoall tests

flashinfer-ai/flashinfer 6.3K

#38152 months ago

fix(common): make multimodal media loaders importable without torch

ai-dynamo/dynamo 7.9K

#111792 months ago

test(sgl-kernel): gate ue8m0 and flashmla prefill tests on supported architectures

sgl-project/sglang 33.2K

#299022 months ago

fix(config): derive scaling in the KimiVL, MiniCPM3, and DeepseekVL2 MLA branches

sgl-project/sglang 33.2K

#298962 months ago

fix(config): treat v_head_dim=0 as unset when deriving model shapes (deepseek-vl2-tiny warmup crash)

sgl-project/sglang 33.2K

#298902 months ago

test: fix CUDA OOM in batch-prefill custom-mask test on 24GB CI GPUs

flashinfer-ai/flashinfer 6.3K

#36092 months ago

[Bugfix] Fix InternVL/InternVL3 LoRA loading TypeError in adapter fallback

InternLM/lmdeploy 8K

#46842 months ago

[Bugfix] Fix ImportError in get_chat_template for builtin chat-template names

InternLM/lmdeploy 8K

#46902 months ago

fix(mamba): reject SM120/SM121 in SSDCombined with a clear error

flashinfer-ai/flashinfer 6.3K

#36682 months ago

[SM120] Emit grouped NVFP4 (ue4m3) block-scaled GEMM in cutlass_library

NVIDIA/cutlass 10.4K

#33462 months ago

[Kernel][SM120] NVFP4 grouped MoE: pingpong schedule at large per-exp…

vllm-project/vllm 90.8K

#464812 months ago

fix(trace): handle combined paged KV cache tensor in _get_tensor (#3685)

flashinfer-ai/flashinfer 6.3K

#36902 months ago

fix(server_args): gate trtllm_mla prefill (not decode) off SM120/SM121

sgl-project/sglang 33.2K

#288372 months ago

[Bugfix] Forward upstream error message in Anthropic streaming converter (#46028)

vllm-project/vllm 90.8K

#461622 months ago

fix(logging): allow exporting OTLP traces without logs

ai-dynamo/dynamo 7.9K

#108062 months ago

[CuTeDSL] Fix make_ptr ZeroDivisionError on sub-byte dtypes

NVIDIA/cutlass 10.4K

#33343 months ago

[CuTeDSL] Fix cute.is_major crash on List[int] mode

NVIDIA/cutlass 10.4K

#33333 months ago

fix(kv-router): cancel in-flight recovery when worker is removed

ai-dynamo/dynamo 7.9K

#106163 months ago

[Bugfix] Fix double-counted max_q_seqlen in decode delta kv_seqlens

InternLM/lmdeploy 8K

#46853 months ago

fix(mla): don't recommend backend='cutlass' on SM120/SM121 MLA fallback

flashinfer-ai/flashinfer 6.3K

#36693 months ago

fix(topk): eliminate multi-CTA radix top-k stream hangs on SM120/SM121

flashinfer-ai/flashinfer 6.3K

#36153 months ago

[Offload] Skip device-map entries with no local module in dispatch_with_map

vllm-project/compressed-tensors 321

#7373 months ago

[Bugfix] Clear reqs_status on async lookup timeout to prevent KeyError on recall

LMCache/LMCache 11.6K

#37223 months ago

[Test] Cover multimodal models without language models (#45806)

vllm-project/vllm 90.8K

#458113 months ago

[Bugfix] DiffusionGemma: only pop a request's logprobs when it commits (#45689)

vllm-project/vllm 90.8K

#457543 months ago

[Fix] Update speculative draft weights on update_weights_from_distributed

sgl-project/sglang 33.2K

#282573 months ago

fix(jit): quote -ccbin host compiler path so it works with spaces

flashinfer-ai/flashinfer 6.3K

#36373 months ago

fix(jit): cap ninja parallelism by available memory to avoid OOM (#3634)

flashinfer-ai/flashinfer 6.3K

#36363 months ago

fix: correct MHA PP KV-pointer slicing for uneven full-attn splits (#27740)

sgl-project/sglang 33.2K

#282293 months ago

[PT2E][X86] Fall back to plain linear annotation for reused nn.Linear with fusable post op

#44803 months ago

[Bugfix] Reject structured outputs for diffusion decoders with a clear error

vllm-project/vllm 90.8K

#454683 months ago

[Kernel] Add SM120/SM121 dispatch for int8_scaled_mm

sgl-project/sglang 33.2K

#281373 months ago

[PT2E][X86] Extend reused-module fallback to nn.Conv2d (follow-up to #4480)

#44823 months ago

feat(moe): add SM120/SM121 dispatch for fp8_blockwise_scaled_grouped_mm

sgl-project/sglang 33.2K

#281253 months ago

[Bugfix] Return the tokenizer from maybe_make_thread_pool so it survives pickling

vllm-project/vllm 90.8K

#454603 months ago

[Bugfix] Set type/role explicitly in streaming message_start event

vllm-project/vllm 90.8K

#453763 months ago

fix: give concurrent streams disjoint row_states buffers in multi-CTA top-k

flashinfer-ai/flashinfer 6.3K

#36253 months ago

Add tuned fused-MoE Triton config for Gemma-4 FP8 on RTX PRO 6000 Blackwell (SM120)

sgl-project/sglang 33.2K

#280773 months ago

[BugFix] Don't strand parked async-KV-load requests behind an unschedulable queue head

vllm-project/vllm 90.8K

#454063 months ago

Clamp default fused-MoE Triton configs to the device shared memory limit (SM120/SM121)

sgl-project/sglang 33.2K

#280383 months ago

[Bugfix] Fix int32 overflow in concat_mla_q flat warp index

vllm-project/vllm 90.8K

#453843 months ago

Fix BF16 RL weight update with non-routed flashinfer_trtllm MoE backend

sgl-project/sglang 33.2K

#280153 months ago

fix: apply rope overrides to text_config for VLM-format models

sgl-project/sglang 33.2K

#279833 months ago

[Bugfix] Scope block-fp8 fused MoE tolerance to SM120 (#45332)

vllm-project/vllm 90.8K

#453343 months ago

[Bugfix] Skip Ray actor kill after the driver has disconnected

vllm-project/vllm 90.8K

#453283 months ago

test: bound peak memory in mxfp4 groupwise group GEMM test (32GB OOM)

flashinfer-ai/flashinfer 6.3K

#35943 months ago

[Bugfix] Abort requests that arrive while the engine is asleep

vllm-project/vllm 90.8K

#453263 months ago

test: cover diffkv (qk192/v128) + attention-sink triton attention kernels

sgl-project/sglang 33.2K

#279633 months ago

fix(attention): sm120 shared-memory-safe block size in context_attention_fwd

sgl-project/sglang 33.2K

#279623 months ago

[Bugfix] Reject NVFP4 MoE checkpoints with missing per-expert scales

vllm-project/vllm 90.8K

#453203 months ago

Fix MiMo-V2 expert routing on TopK-bypassing MoE backends

sgl-project/sglang 33.2K

#279163 months ago

test: cover XQA decode head_dim=256 with small page sizes (#2638)

flashinfer-ai/flashinfer 6.3K

#35773 months ago

feat: add RTX PRO 6000 Blackwell GPU SKU

ai-dynamo/dynamo 7.9K

#105793 months ago

[#14146][bugfix] Fix CuteDSL MoE ghost-token and global-index bugs

NVIDIA/TensorRT-LLM 14.5K

#151553 months ago

[None][bugfix] Guard e_score_correction_bias load for DeepSeek-V2-Lite

NVIDIA/TensorRT-LLM 14.5K

#151543 months ago