HGPU group@hgpu

High performance computing on graphics processing units (GPU): AMD/ATI, nVidia, Intel Xeon Phi, CUDA, OpenCL, OpenGL, GPGPU, HPC

hgpu.org/
Joined May 2011
10 Photos and videos
KernelOPT: Dispatch-Aware Agentic Search for GPU Kernel Optimization #Triton #CUDA #DeepLearning #LLM hgpu.org/?p=31275

KernelOPT: Dispatch-Aware Agentic Search for GPU Kernel Optimization

Deep learning inference and training performance depends critically on GPU kernel efficiency. Modern compilers such as PyTorch Inductor automatically generate GPU kernels from high-level model code…

hgpu.org
2
202
Xtrace: High-Fidelity GPU Intra-Kernel Tracing via Binary-Level Instruction Splicing #LLM #Performance hgpu.org/?p=31274

Xtrace: High-Fidelity GPU Intra-Kernel Tracing via Binary-Level Instruction Splicing

Modern GPU kernels fuse increasingly more work into a single kernel, and intra-kernel tracing has become the mainstream method to profile them. Tracing inserts probes into the kernel to record its …

hgpu.org
5
335
Microarchitectural Memory Bandwidth Saturation, KV-Cache Paging Dynamics, and Time-to-First-Token Latency: A Comparative Benchmark of vLLM, TensorRT-LLM, and FlashAttention-3 on NVIDIA Hopper H100 versus AMD Instinct MI300X #CUDA #ROCm #AMD #vLLM hgpu.org/?p=31273

Microarchitectural Memory Bandwidth Saturation, KV-Cache Paging Dynamics, and Time-to-First-Token...

The commercial and scientific utility of Large Language Models (LLMs) hinges directly on the efficiency of hyperscale inference serving infrastructure. LLM inference operates across two fundamental…

hgpu.org
1
101
Accelerating the Mitigation of LLM Inference Nondeterminism Across GPU Architectures #CUDA #LLM #Package hgpu.org/?p=31272

Accelerating the Mitigation of LLM Inference Nondeterminism Across GPU Architectures

Large language model (LLM) outputs are expected to be reproducible under greedy decoding, yet in practice the same model, prompt, and software stack produce different outputs on different GPUs. The…

hgpu.org
1
5
228
pytest-gpu-proof: Enabling Cloud-CPU Continuous Integration for GPU Code with Local GPU Attestation #Python #Cloud #CI #Package hgpu.org/?p=31271

pytest-gpu-proof: Enabling Cloud-CPU Continuous Integration for GPU Code with Local GPU Attestation

GPU acceleration is now routine across robotics, but cloud-hosted GPU continuous integration (CI) runners are expensive, resulting in severe under-testing of GPU-accelerated code. We present pytest…

hgpu.org
75
DeepSeek-V4-Flash on AMD gfx90a: Correctness Recovery and Inference Performance Engineering #AMD #HIP #LLM #Performance #DeepSeek hgpu.org/?p=31229

DeepSeek-V4-Flash on AMD gfx90a: Correctness Recovery and Inference Performance Engineering

We present the enablement, correctness recovery, and performance engineering of DeepSeek-V4-Flash inference on AMD Instinct MI250 GPUs using the gfx90a/CDNA2 architecture. The system integrates nat…

hgpu.org
1
2
11
732
Automated Instruction Encoding Synthesis for Modern GPU ISA Compression #CUDA #ISA #HardwareArchitecture #Package hgpu.org/?p=31226

Automated Instruction Encoding Synthesis for Modern GPU ISA Compression

Modern GPU kernels increasingly stress the instruction supply path, while fixed instruction containers can leave substantial footprint slack. This paper presents an automated encoding-synthesis fra…

hgpu.org
2
7
542
PrefixBench-H100: Characterizing Prefix Reuse and Time-to-First-Token in H100 LLM Serving #CUDA #LLM #Performance #Package hgpu.org/?p=31227

PrefixBench-H100: Characterizing Prefix Reuse and Time-to-First-Token in H100 LLM Serving

Repeated prompt prefixes are increasingly common in LLM serving workloads, appearing in system prompts, templated retrieval-augmented generation pipelines, agent frameworks, and multi-turn conversa…

hgpu.org
4
333
Accelerating the Solving of Many Tiny General Linear Systems on GPUs: Application to Constitutive Laws #CUDA #LinearAlgebra hgpu.org/?p=31228

Accelerating the Solving of Many Tiny General Linear Systems on GPUs: Application to Constitutive...

Many applications require solving large numbers of independent linear systems on GPUs. While this need is well addressed for small to large systems, tiny ones, understood here as systems of dimensi…

hgpu.org
2
1
266
AutoTuneBench: Trustworthy Measurement for Agent Auto-Tuning of LLM Serving Engines #CUDA #LLM #Performance #Package hgpu.org/?p=31225

AutoTuneBench: Trustworthy Measurement for Agent Auto-Tuning of LLM Serving Engines

Large language model agents tune GPU kernels and serving engines through a closed loop of propose, measure, and keep, but the measurements behind this loop are not trustworthy. We characterize four…

hgpu.org
179
Stencil Computation at the Intersection of AI and HPC #Triton #SYCL #AI #HPC #Stencil hgpu.org/?p=31200

Stencil Computation at the Intersection of AI and HPC

Tensor compilers such as TinyTC and OpenAI Triton were originally developed for AI workloads, but the same tiling and memory abstractions can be applied to implement efficient high-order stencils f…

hgpu.org
273
MaxKernel: Agentic Kernel Generation for TPUs #TPU #Package hgpu.org/?p=31198

MaxKernel: Agentic Kernel Generation for TPUs

Designing and authoring high-performance custom kernels for accelerators is a complex task that requires deep hardware-level expertise. Large Language Models (LLM) can be leveraged together with re…

hgpu.org
4
7
849
Taming Bitwise Behavior in GPU Kernels with Tensor Core: Black-Box Reconstruction, Compiler Enforcement, and Static Verification #CUDA #PTX #Triton hgpu.org/?p=31199

Taming Bitwise Behavior in GPU Kernels with Tensor Core: Black-Box Reconstruction, Compiler...

Determinism and numerical reproducibility are increasingly required of GPU kernels in machine learning systems, yet deterministic implementations of the same kernel can still differ bit for bit. Fl…

hgpu.org
2
8
626
Every Kernel Is a Join: Automatic Multi-GPU Parallelism for AI Computations in Einsummable #CUDA #Triton #LLM #AI #Package hgpu.org/?p=31197

Every Kernel Is a Join: Automatic Multi-GPU Parallelism for AI Computations in Einsummable

Distributing an AI computation across the GPUs of a multi-GPU server is one of the central problems in systems-for-AI. We present Einsummable, a prototype system that accepts a PyTorch-like descrip…

hgpu.org
1
4
30
2,127
Hardware-Aware FP4 FlashAttention-4 #CUDA #PTX #FP4 #Package hgpu.org/?p=31196

Hardware-Aware FP4 FlashAttention-4

Blackwell’s 4-bit floating-point (FP4) tensor cores do not automatically make attention faster because softmax conversion and on-chip dependencies dominate once its matrix products shrink. We…

hgpu.org
3
18
1,412
PTXBench: Benchmark and Adapt LLMs for GPU Kernel Optimization with Architecture-specific PTX #PTX #LLM #Package hgpu.org/?p=31135

PTXBench: Benchmark and Adapt LLMs for GPU Kernel Optimization with Architecture-specific PTX

We introduce PTXBench, a benchmark for evaluating and adapting large language models (LLMs) to use architecture-specific PTX for GPU kernel optimization. PTXBench measures functional correctness, w…

hgpu.org
2
5
26
2,045
Benchmarking Confidential Computing Performance on NVIDIA Blackwell GPUs #CUDA #Security hgpu.org/?p=31134

Benchmarking Confidential Computing Performance on NVIDIA Blackwell GPUs

This paper measures the performance impact of running large language model inference and training inside a Trusted Execution Environment (TEE) on NVIDIA B200 GPUs, using Intel Trust Domain Extensio…

hgpu.org
1
3
377
Concurrency Response of Plain Global Loads on the NVIDIA H100 #CUDA #Performance #HardwareArchitecture hgpu.org/?p=31133

Concurrency Response of Plain Global Loads on the NVIDIA H100

The bandwidth a memory-bound GPU kernel sustains is set by how many bytes it keeps in flight. We use Little’s Law here as throughput accounting, not as a measured hardware pool. CUDA fills th…

hgpu.org
1
306
An HPC Approach to Accelerate Tensor Decompositions #CUDA #HPC hgpu.org/?p=31132

An HPC Approach to Accelerate Tensor Decompositions

Quantum systems grow in complexity so rapidly that even modest models become difficult to simulate, creating a strong need for methods that can handle high-dimensional data, also known as tensors. …

hgpu.org
2
12
798
RealisticTritonBench: A Benchmark for Triton-Kernel Generation in Real-World AI Frameworks #Triton #CUDA #LLM #Package hgpu.org/?p=31131

RealisticTritonBench: A Benchmark for Triton-Kernel Generation in Real-World AI Frameworks

In modern AI frameworks, GPU kernels are key to overall system performance. Combining usability, portability, and near-handwritten CUDA performance, Triton is widely adopted for implementing GPU ke…

hgpu.org
223
Load more