ML Systems & GPU Engineer — I make LLM inference run faster on real hardware: attention kernels, KV-cache management, multi-GPU serving, and compiler-level optimization.

CS @ Johns Hopkins University (BS, May 2028) · Research Assistant @ Johns Hopkins Medicine 📍 Baltimore, MD · 🗓️ Available for a 12-week internship, May–Aug 2027

🔗 Technical blog: https://minseoc03.github.io — deep dives on FlashAttention, GPU runtimes, and low-latency inference

LLM serving & inference optimization on production-grade hardware.

• Multi-agent serving pipeline (Navigator, w/ NVIDIA Safety & JHTV). DAG-driven, six-model pipeline (70B generator + five FP8-quantized critics: 2×34B, 3×8B) on Triton Inference Server + TensorRT-LLM across 2×B200. Diagnosed the bottleneck as orchestration-bound (not compute-bound) by correlating GPU utilization against per-stage queue depth, then improved decode throughput 2.89× via generator GPU isolation, async BLS + in-flight batching, KV-cache block reuse over shared prefixes, and adaptive early stopping — sustaining ~3,100 conversations/hour (~3.3s / ~13s P95, auto-labeled / escalated). Now extending to heterogeneous SGLang + TensorRT-LLM serving with cross-engine KV-cache transfer as the open bottleneck.
• KV-cache placement for human-approval gates (self-initiated research, H100 SLURM cluster). Characterizing a serving regime where agents idle for minutes–hours at approval gates, unifying HBM capacity / PCIe-NVMe bandwidth / resume-recompute into one goodput metric for direct policy comparison. Targeting NeurIPS 2026 ML-for-Systems Workshop & MLSys 2027.
• Fused Triton kernels for medical imaging — 9.7× speedup over unfused PyTorch ops by eliminating per-op kernel-launch/dispatch overhead and intermediate HBM round-trips.
• FlashAttention-2 from scratch in both Triton and CUDA (forward + backward, tiled online-softmax): ~2× speedup and ~5× peak-memory reduction vs. naive PyTorch attention at 4K sequence length. Plus a toy LLVM compiler and an end-to-end MLIR pipeline (IR parsing, pass execution, Bazel integration).

• Languages: C, C++, CUDA, Python, x86-64 Assembly
• ML & Inference: PyTorch, Triton, TensorRT-LLM, Triton Inference Server, vLLM, SGLang, MLIR/LLVM
• Systems & Profiling: Linux, SLURM, Nsight Systems/Compute, perf, Prometheus, Grafana, CMake, Bazel

See pinned repositories below — mlsys-self-study (FlashAttention-2 in Triton + CUDA, LLVM/MLIR), medical-enhancement-triton (fused DICOM kernels), and approval-gate-serving (KV-placement measurement harness).

• 📝 Blog: https://minseoc03.github.io
• 💼 LinkedIn: https://www.linkedin.com/in/minseoc03
• 📧 cfi3288@gmail.com