GPU kernels to accelerator architecture

A hiring manager should be able to answer three questions from this portfolio: can I make a kernel correct, can I locate the real performance limit, and can I turn that diagnosis into a better mapping between a model and a machine? The supplied LeetGPU backlog is useful because it spans all three levels, from coalesced loads to a complete Llama block and then to graph algorithms and multi-agent simulation. The ordering below turns that breadth into one argument instead of a collection of unrelated solutions.

The evidence contract

All stages share one benchmark harness. That matters more than the language used for any one solution. A CUDA implementation exposes the machine model directly; a Triton implementation tests whether the same blocked algorithm can be expressed productively. Both feed the same oracle, shape matrix, timer, profiler, and report.

The shape suite has two jobs. Canonical powers of two make regressions easy to compare, while awkward shapes reveal tail predicates, alignment assumptions, excessive padding, and kernels tuned to one demo. A result is not ready for the portfolio if it silently drops the awkward cases.

Stage 1 · Memory transactions and coalescing

Start with Vector Addition, Matrix Copy, Matrix Addition, Matrix Transpose, Reverse Array, and Interleave Arrays. Color Inversion and RGB to Grayscale make vectorized loads and packed layouts visible. ReLU, Leaky ReLU, Sigmoid, SiLU, Value Clipping, GEGLU, and SwiGLU then show that adding arithmetic often changes nothing because the kernel is still waiting on bytes.

Each task gets a deliberately poor version and a corrected version: scalar versus vectorized access, aligned versus offset input, row- versus column-major traversal, and a branchy tail versus a predicated tail. The write-up should predict transaction count before opening the profiler, then reconcile the prediction with requested and actual sectors.

Completion gate

• Exact output for copy, reorder, and integer image transforms; fp32 activations within atol=1e-6 and rtol=1e-5 of the reference across aligned, one-element-offset, odd, and larger-than-L2 inputs.
• Define B_copy from a same-run, warmed, coalesced copy kernel. For inputs of at least 64 MiB, target at least 90 percent of B_copy for Matrix Copy, 75 percent of the byte-derived roof for Vector Addition, and 70 percent for an out-of-place tiled Matrix Transpose.
• Publish one profile pair in which the optimized mapping reduces sectors per request relative to the intentional anti-pattern. The profile, rather than a generic claim about coalescing, is the evidence.

Stage 2 · Reductions, scans, and selection

The core sequence is Reduction, Dot Product, Prefix Sum, Segmented Exclusive Prefix Sum, and Stream Compaction. Histogramming exposes atomic contention; Count Array Element and the 2D/3D count variants test indexing; Top K Selection, Max Subarray Sum, and Softmax connect collective primitives to model serving. Subarray Sum and its 2D/3D forms are useful checks that the indexing layer remains separate from the collective.

Build a progression from a global-memory tree to shared-memory blocks, warp shuffles, and a multi-block finalization. Record work, depth, launch count, and bytes for every version. For scans, explain the trade between a work-efficient tree and extra synchronization. For histograms, publish both uniform random keys and a single-bin adversary so atomics are not evaluated only on a friendly distribution.

Completion gate

• Use an fp64 CPU oracle for fp32 reductions and require relative error no worse than 5e-5; integer scans, compaction indices, counts, and histograms must match exactly. Test sizes immediately around block and warp boundaries.
• At 16 million elements, target at least 70 percent of B_copy as effective bandwidth for Reduction and at least two-thirds of same-run CUB DeviceScan throughput for Prefix Sum.
• Preserve a profile that accounts for every full-array pass and demonstrates that the final version has no unexplained global round trip. Report deterministic and fastest variants separately when floating-point reduction order changes.

Stage 3 · Tiled linear algebra and convolution

Matrix Multiplication grows into a basic GEMM, Batched Matrix Multiplication, FP16 Batched Matrix Multiplication, INT8 Quantized MatMul, and INT4 weight-only MatMul. Sparse Matrix-Vector and Sparse Matrix-Dense Matrix Multiplication make the cost of metadata and imbalance explicit. The spatial branch covers 1D, 2D, and 3D Convolution, Gaussian Blur, 2D Max Pooling, Batch Normalization, and Group Normalization. Ordinary Least Squares and Logistic Regression exercise composition rather than adding a new primitive.

The implementation ladder should be visible in code and charts: naive global-memory GEMM, shared-memory blocking, register tiling, vectorized and asynchronous copies where supported, then a Tensor-Core or Triton version. Rectangular decoder shapes matter as much as square textbook matrices. Every quantized kernel must state accumulator width, scale granularity, rounding, saturation, and the bytes consumed by scales and zero points.

Completion gate

• Test square, rectangular, batched, non-multiple-of-tile, and transposed layouts. Compare fp32 against an fp64 oracle with a documented size-aware tolerance; compare int8 accumulation exactly before requantization.
• On a documented 4096 × 4096 × 4096 fp32 case, target at least 10× the naive in-repo kernel and 50 percent of same-run cuBLAS SGEMM. A second chart must include decode-like skinny matrices, even if that exposes a much larger gap.
• Close the stage only after one convolution and one GEMM include an arithmetic-intensity calculation, a measured roofline point, shared-memory bank-conflict data, and an explanation of the remaining library gap.

What the ladder is really optimizing

The roofline does not prescribe an optimization; it prevents the wrong one. A point far below the sloped bandwidth roof may have poor access or insufficient parallelism. A point on that roof needs fewer HBM bytes, not more FLOPs. Tiling, fusion, quantization, and keeping weights or KV state closer to compute all move a workload toward the right by increasing useful work per off-chip byte.

Stage 4 · Transformer inference kernels

Begin with Token Embedding, RMS Normalization, Rotary Positional Embedding, Softmax Attention, and a SwiGLU MLP Block. Add Multi-Head and Causal Self-Attention, Grouped Query Attention, Sliding Window Attention, ALiBi, and INT8 KV-Cache Attention. The serving path then needs Weight Dequantization, INT4 weight-only MatMul, LoRA Linear, MoE Top-K Gating, Top-p Sampling, and Speculative Decoding Verification. A GPT-2 block and a Llama-style block are integration tests, not separate résumé bullets.

SSM Selective Scan and Causal Depthwise Conv1d are valuable stretch kernels because they test whether the harness generalizes beyond attention. Linear, decaying-causal, and sliding-window attention should be compared by state size and data movement, not placed in a single “attention” bucket. The final report separates time to first token from inter-token latency, because prefill and decode occupy different parts of the roofline.

Completion gate

• Cover batches 1 and 8, sequence lengths 1, 128, 2,048, and 16,384 where memory permits, hidden sizes 1,024 and 4,096, odd head counts, and both causal and padded inputs. State fp32, fp16, and quantized tolerances separately; target cosine similarity of at least 0.999 for the dequantized INT8 KV-cache attention output.
• Fuse at least one real chain, such as residual plus RMSNorm or dequantization plus linear. Against the unfused in-repo path, target at least 20 percent fewer profiler-reported HBM bytes and 15 percent lower median latency on one predeclared shape.
• Benchmark the complete Llama-style block against eager PyTorch and torch.compile in the same run. Publish p50 and p95 latency, tokens per second, peak allocation, kernel count, prompt length, KV length, and any graph breaks, even when the custom path loses.

Stage 5 · Graphs, simulation, and sorting

Sorting, Radix Sort, and Parallel Merge develop partitioning and load balance. BFS Shortest Path and All-Pairs Shortest Paths add frontier management and topology-dependent locality. K-Means, Nearest Neighbor, and the Multi-Agent boids simulation turn spatial neighborhoods into a reusable uniform-grid primitive. A 2D Jacobi Stencil, Monte Carlo Integration, FFT/2D FFT, Matrix Power, and Linear Recurrence round out structured scientific workloads without pretending they share one bottleneck.

The capstone is the boids simulation: retain the exact O(N²) implementation as a small-input oracle, then add spatial binning, prefix-summed cell offsets, compacted agent lists, and a neighbor pass. That composition reuses Stages 1 and 2 and produces a simulation primitive relevant to robotics, world models, and reinforcement-learning environments.

Completion gate

• Sorting and BFS outputs must match exact CPU references; K-Means must publish convergence criteria and seeded initialization; boids must test finite state, bounded speed, periodic boundaries, and a small-step match against the all-pairs oracle.
• For 16 million random uint32 keys, target at least 60 percent of same-run CUB radix-sort throughput, including allocation and temporary-storage policy in the report.
• For 100,000 uniformly distributed boids, target p95 simulation step latency below 16.7 ms. From 25,000 to 100,000 agents, the binned version must grow by less than 6×, while the retained all-pairs baseline documents the quadratic reference behavior.

The bridge to accelerator work

Etched and MatX are reference points because their public material emphasizes large-model hardware and the system around it; they are not affiliations or claims about undisclosed architectures. The public Etched site describes co-design across chips, racks, software, and manufacturing for inference. MatX publicly describes a large-model target spanning training, reinforcement learning, prefill, and decode, with weights typically in SRAM, KV state typically in HBM, scale-up/scale-out interconnect, and direct hardware control. The portfolio response should be a set of inspectable models, not speculation about either company’s silicon.

The hardware capstone should extend the existing FPGA matrix-multiply accelerator with a DMA-facing tiled buffer, double buffering, a quantized MAC path, and randomized verification. A companion compiler prototype can lower a small graph IR containing GEMM, RMSNorm, RoPE, attention, and elementwise fusion into tile schedules for the simulator. The minimum evidence is an analytical traffic model, tests that conserve every byte, a sweep that emits machine-readable Pareto data, and synthesis timing/resource reports clearly labeled by tool and device.

The underlying theory and measured examples already have natural homes on this site: Parallel Computing covers work-depth analysis, CUDA, Triton, scan, reduction, and tiling; Advanced Systems Architecture covers HBM, rooflines, systolic dataflows, precision, and interconnects. This roadmap turns those course-style explanations into a portfolio whose unit of progress is a reproducible artifact.

A concise project rubric

Score every published stage before calling it finished. The release gate is at least 80/100, at least 24/30 in correctness, and no missing benchmark or raw-data link.

A smaller project that passes this rubric is stronger hiring evidence than a long kernel checklist with no oracle, baseline, or explanation. The intended final shape is one repository, one benchmark schema, five stage reports, and one accelerator capstone that can trace every hardware choice back to a measured workload.

Primary references

1. NVIDIA, CUDA Programming Guide and Nsight Compute Profiling Guide.
2. Triton project, official documentation and kernel tutorials.
3. NVIDIA CCCL, CUB documentation, used for same-run collective and sorting baselines.
4. Etched, Frontier Inference Clusters and the official role index.
5. MatX, official product and workload overview.