https://vito-lin-dev.github.io/profile/ Vito Lin writes Rust close to the metal and CUDA close to the silicon, and sends the fixes upstream. en Thu, 01 Oct 2026 00:34:33 GMT https://vito-lin-dev.github.io/profile/writing/2026-08-pretraining_and_fine-tuning_nanoGPT-124m_from_scratch/ https://vito-lin-dev.github.io/profile/writing/2026-08-pretraining_and_fine-tuning_nanoGPT-124m_from_scratch/ Tue, 18 Aug 2026 09:00:00 GMT Building a 124M-parameter GPT-2 from scratch, then turning it from a text-completion model into an instruction-following one with SFT — plus the systems bugs that ate more time than the model itself. https://vito-lin-dev.github.io/profile/writing/2026-08-llama-3-training-infrastructure/ https://vito-lin-dev.github.io/profile/writing/2026-08-llama-3-training-infrastructure/ Sun, 09 Aug 2026 09:00:00 GMT GQA and document masking in Llama 3's architecture, and the 4D parallelism strategy — TP, CP, PP, DP — that Meta used to fit the 405B run across a topology-aware cluster. https://vito-lin-dev.github.io/profile/writing/2026-07-vllm-paged-attention/ https://vito-lin-dev.github.io/profile/writing/2026-07-vllm-paged-attention/ Thu, 30 Jul 2026 09:00:00 GMT PagedAttention as virtual memory, copy-on-write prefix sharing, and all-or-nothing preemption — reading the vLLM paper as a catalogue of OS techniques applied to the KV cache. https://vito-lin-dev.github.io/profile/writing/2026-07-mooncake-serving-architecture/ https://vito-lin-dev.github.io/profile/writing/2026-07-mooncake-serving-architecture/ Sat, 25 Jul 2026 09:00:00 GMT Notes on Mooncake's FAST '25 paper: disaggregated prefill/decode, a cluster-wide KVCache pool, and prefix hashing that skips prefill on a cache hit. https://vito-lin-dev.github.io/profile/writing/2026-07-ai-infra-resource/ https://vito-lin-dev.github.io/profile/writing/2026-07-ai-infra-resource/ Sun, 19 Jul 2026 09:00:00 GMT Everything I read, watched, and cloned this year while moving into AI Infrastructure — GPU programming, LLM internals, and the systems around serving them. https://vito-lin-dev.github.io/profile/writing/2026-05-high_performance_gemm/ https://vito-lin-dev.github.io/profile/writing/2026-05-high_performance_gemm/ Tue, 26 May 2026 09:00:00 GMT Shared memory isn't the end of the optimization pipeline — how loading tile fragments into registers gets 2.5x–3x more throughput than shared memory alone, and why register pressure is the tradeoff that limits it. https://vito-lin-dev.github.io/profile/writing/2026-05-cuda-oxide/ https://vito-lin-dev.github.io/profile/writing/2026-05-cuda-oxide/ Wed, 13 May 2026 09:00:00 GMT cuda-oxide compiles Rust directly to PTX, with no wrappers, DSLs, or FFI overhead. Notes on why that excites me, and on the patches I contributed to get there. https://vito-lin-dev.github.io/profile/writing/2026-05-cuda-compile/ https://vito-lin-dev.github.io/profile/writing/2026-05-cuda-compile/ Mon, 11 May 2026 09:00:00 GMT Between a .cu file and Tensor Cores sits a multi-stage compiler pipeline — CUDA C++, PTX, ptxas, SASS, and the fatbins that let one binary run on both an RTX 3090 and an H100. https://vito-lin-dev.github.io/profile/writing/2026-04-wave-quantization/ https://vito-lin-dev.github.io/profile/writing/2026-04-wave-quantization/ Mon, 20 Apr 2026 09:00:00 GMT Wave quantization: why grid sizes that don't divide evenly across SMs leave a nearly-idle last wave, why persistent kernels only half-fix it, and how Stream-K splits the remainder across SMs instead. https://vito-lin-dev.github.io/profile/writing/2026-04-gpu-hardware/ https://vito-lin-dev.github.io/profile/writing/2026-04-gpu-hardware/ Mon, 13 Apr 2026 09:00:00 GMT SMs, warps, and SIMT execution — why GPU performance is about latency hiding and occupancy rather than raw thread count, and why memory, not compute, is usually the real bottleneck. https://vito-lin-dev.github.io/profile/writing/2026-04-cuda-practice/ https://vito-lin-dev.github.io/profile/writing/2026-04-cuda-practice/ Sun, 05 Apr 2026 09:00:00 GMT You don't need to own an NVIDIA GPU to start learning CUDA — a rundown of eight cloud platforms with free GPU access, from Colab and Kaggle to CUDA-specific practice sites. https://vito-lin-dev.github.io/profile/writing/2026-04-vector-multiplication/ https://vito-lin-dev.github.io/profile/writing/2026-04-vector-multiplication/ Thu, 02 Apr 2026 09:00:00 GMT The difference between a slow and a fast matmul kernel isn't thread count — it's memory access pattern. How tiling turns a memory-bound problem into a compute-efficient one. https://vito-lin-dev.github.io/profile/writing/2026-03-cuda-memory/ https://vito-lin-dev.github.io/profile/writing/2026-03-cuda-memory/ Sun, 29 Mar 2026 09:00:00 GMT Registers, shared memory, global memory, constant memory, texture memory — where each one sits on-chip or off-chip, and why most CUDA performance issues trace back to global memory. https://vito-lin-dev.github.io/profile/writing/2026-03-execution-model/ https://vito-lin-dev.github.io/profile/writing/2026-03-execution-model/ Wed, 25 Mar 2026 09:00:00 GMT The thread/block/grid hierarchy, why GPUs schedule warps instead of individual threads, and why launch configuration is a performance decision, not just boilerplate. https://vito-lin-dev.github.io/profile/writing/2026-03-cuda-vector-additon/ https://vito-lin-dev.github.io/profile/writing/2026-03-cuda-vector-additon/ Sun, 22 Mar 2026 09:00:00 GMT The host/device split and the six-step memory dance every CUDA program follows — allocate, copy to GPU, execute, copy back, free — starting from the simplest possible kernel.