Principal Software Engineer, DGX Cloud Production Engineering

NVIDIA DGX Cloud is scaling GPU infrastructure across internal, partner, and cloud environments. We are looking for Principal Software Engineers to help shape the technical direction for production engineering, Kubernetes-based operations, automation, and reliability across large-scale GPU clusters. This role is for senior technical leaders who can define architecture, lead through influence, build critical systems, and turn ambiguous infrastructure problems into durable software and operating models. What you’ll be doing: • Define and execute the technical strategy for DGX Cloud cluster operations, building the automation, GitOps, and Day 2 reliability needed to operate large-scale GPU clusters across NVIDIA Cloud Partners (NCPs) and on-prem environments. • Lead design and implementation of systems for cluster lifecycle, validation, repair, upgrades, observability, and readiness. • Establish patterns for Kubernetes-based GPU cluster operations across partner and on-prem environments. • Identify and eliminate operational toil through software, APIs, automation, and agent-assisted workflows. • Set technical standards for production readiness, SLOs, incident response, handoff gates, and operational acceptance. • Mentor engineers and influence platform, infrastructure, storage, networking, security, and workload teams. What we need to see: • 15+ years of experience building and operating large-scale distributed systems or cloud infrastructure. • Deep experience with Kubernetes, Linux, infrastructure automation, and production operations. • Strong programming experience in Go, Python, or similar. • Proven ability to lead complex cross-org technical initiatives. • Experience designing reliable systems with clear SLOs, observability, incident response, and automation. • BS/MS in Computer Science or equivalent experience. Ways to stand out from the crowd: • Experience with GPU clusters, AI/ML infrastructure, Kubernetes operators, GitOps, BMaaS/VMaaS, managed Kubernetes, or multi-cloud fleet operations. • Experience building internal platforms, control planes, lifecycle automation, or production readiness frameworks. • Track record of turning operational pain into reusable software, APIs, and engineering standards. NVIDIA is leading the way in groundbreaking developments in Artificial Intelligence, High-Performance Computing and Visualization. The GPU, our invention, serves as the visual cortex of modern computers and is at the heart of our products and services. We have some of the most forward-thinking and hard-working people on the planet working for us. If you're creative, hard-working and self-motivated, we want to hear from you! Your base salary will be determined based on your location, experience, and the pay of employees in similar positions. The base salary range is 272,000 USD - 431,250 USD. You will also be eligible for equity and benefits. Applications for this job will be accepted at least until May 22, 2026. This posting is for an existing vacancy. NVIDIA uses AI tools in its recruiting processes. NVIDIA is committed to fostering a diverse work environment and proud to be an equal opportunity employer. As we highly value diversity in our current and future employees, we do not discriminate (including in our hiring and promotion practices) on the basis of race, religion, color, national origin, gender, gender expression, sexual orientation, age, marital status, veteran status, disability status or any other characteristic protected by law. Originally posted on Himalayas Software-Engineer, Principal-Software-Engineer, Cloud-Production-Engineering, Infrastructure-Engineering, Platform-Engineering, Site-Reliability-Engineering, Production-Engineering, GPU-Infrastructure-Engineering, Principal-Software-Development-Engineer