Deep dive into CUDA and inference-engine experience
Filed under Nvidia
Free read 2 of 3Describe one project: the model or workload, hardware, and the performance problem (latency or throughput target).
The question
The hiring manager asked you to go deep on your CUDA work (specific kernels and optimisations) and on model serving and inference engines such as vLLM, plus your exact contributions and the skills you want to learn next. This assesses real hands-on depth.
Approach
Describe one project: the model or workload, hardware, and the performance problem (latency or throughput target).
State exactly which kernels you wrote or tuned, and what limited them: memory-bound versus compute-bound. Be precise about which part was yours.
Give concrete optimisations you actually applied: coalesced memory access, shared-memory tiling, occupancy tuning, kernel fusion, streams. For serving, discuss batching, KV-cache management and quantisation, and how vLLM-style systems handle them.
Quote measured before-and-after numbers and how you profiled them. End with honest next skills, such as profiling tools, Triton or distributed inference.
Tips
- What they are testing: Depth of hands-on knowledge and honesty about your own contribution versus the library or team.
- Common mistake: Describing optimisations you only read about, or giving no measured numbers to back claims.
Follow-ups
- What were your exact contributions to the projects mentioned?
- What technical skills do you want to pick up next?
Related Nvidia questions
Collected from public candidate posts and coding forums. Updated Nov 2026.