Engineering Manager, LLM Inference & Deployment at Scale
Sign up free to see how well your resume matches this role.
About this role
NVIDIA is seeking an engineering manager to lead engineering activities related to productizing Deep Learning models. Academic and commercial groups around the world are using GPUs to redefine Artificial Intelligence and data analytics, and to power data centers. Join the team building software which will be used by the entire world. Interact with the scientific community to implement and improve the latest algorithms. Ability to work in a multifaceted, product-centric environment is required and excellent interpersonal skills are also a requirement.
If you have a good understanding for deep learning and a strong algorithmic background, with exposure to large scale LLM/VLM deployment, inference optimization, and leadership experience, then this role may be a great one for you! In this role you will lead and mentor forward-thinking engineers and will own related activities and interactions with teams across NVIDIA. You will be working with key internal partners on priority alignment across relevant teams for roadmap development of highly optimized novel and state-of-the-art numerical, analytics, and deep learning algorithms and associated R&D duties. If the idea of pushing the boundaries of state-of-the-art research and development excites you, and are interested in getting exposure to the entire DL SW stack, come join the team that build the GPU-accelerated DL platform used by the entire world.
What you’ll be doing:
Plan, schedule, mentor, and lead the execution of projects and activities of the team. Including creating, optimizing, and deploying inference DL workloads.
Collaborate with internal customers to align priorities across business units
Coordinate projects across different geographic locations
Grow and develop a world-class team
Travel to conferences, other sites, or visit customers occasionally
What we need to see:
Minimum requirement of BSc or equivalent experience
8+ overall years related of overall experience, including 3 years of management/leadership experience
Experience leading multiple software engineering projects
Strong experience with Large Language Models (LLMs) and Large Visual-Language Models (VLMs)
Excellent programming, debugging, performance analysis, and test design skills
Great communication
Ways to stand out from the crowd:
Experience with inference of DL models
Experience doing performance analysis and tuning
Exposure to inference platforms such as TensorRT-LLM, vLLM, and SGLang
Project management tools (e.g. JIRA, Microsoft Project)
You will also be eligible for equity and benefits.
This posting is for an existing vacancy.
NVIDIA uses AI tools in its recruiting processes.
NVIDIA is committed to fostering an inclusive work environment and proud to be an equal opportunity employer. As we highly value diversity in our current and future employees, we do not discriminate (including in our hiring and promotion practices) on the basis of race, religion, color, national origin, gender, gender expression, sexual orientation, age, marital status, veteran status, disability status or any other characteristic protected by law.Company
Company facts come from this company's own listings. We only show what the postings themselves carry.