Principal Staff Software Engineer, Systems Infrastructure

At LinkedIn, our approach to flexible work is centered on trust and optimized for culture, connection, clarity, and the evolving needs of our business. The work location of this role is hybrid, meaning it will be performed both from home and from a LinkedIn office on select days, as determined by the business needs of the team. 

LinkedIn’s AI Infrastructure organization is responsible for building the foundational platforms that power AI across LinkedIn. The LLM Serving team builds the critical infrastructure that enables efficient, reliable, and large-scale deployment of large language models and other advanced AI models in production.

This team sits at the center of LinkedIn’s AI platform, owning the layer between model training and production serving. The work focuses on making large-scale models run faster, cheaper, and more efficiently on GPUs at LinkedIn scale. The team builds and extends high-performance serving infrastructure and contributes to leading open-source technologies such as SGLang, vLLM, and related model serving frameworks.

We are looking for a Principal Staff Software Engineer with deep expertise at the intersection of systems, machine learning, GPU infrastructure, and large-scale inference. This is a highly technical, company-level leadership role for an engineer who can set long-term technical direction while remaining deeply hands-on across the serving stack.

You will help define the architecture and evolution of LinkedIn’s next-generation LLM serving platform, driving improvements in performance, efficiency, reliability, scalability, and cost across AI workloads. The role requires the ability to operate across model architecture, runtimes, compilers, kernels, distributed systems, and hardware while influencing technical strategy across multiple teams and organizations.

Responsibilities

• Set the long-term technical strategy and architecture for LinkedIn’s large-scale LLM serving and inference infrastructure.
• Lead the design, development, and evolution of high-performance online and offline inference platforms for LLMs and other advanced AI models.
• Drive major improvements in inference latency, throughput, GPU utilization, reliability, scalability, and infrastructure cost.
• Architect serving systems that operate efficiently across large GPU fleets and support a diverse set of models, products, and production workloads.
• Optimize model execution across the full stack, including model architecture, serving runtime, compiler, kernel, memory, networking, and hardware layers.
• Drive adoption of model optimization techniques such as quantization, pruning, compression, batching, caching, and memory optimization.
• Improve GPU efficiency through low-level systems work, including CUDA and Triton optimization, kernel-level improvements, runtime tuning, scheduling, and hardware-aware performance engineering.
• Make critical architectural trade-offs across latency, throughput, model quality, capacity, reliability, developer experience, and cost.
• Partner with ML, infrastructure, product, and research teams to identify systemic serving bottlenecks and shape the long-term AI infrastructure roadmap.
• Evaluate and drive adoption of emerging inference technologies, serving architectures, accelerators, and open-source frameworks.
• Contribute to and extend open-source LLM serving technologies such as SGLang, vLLM, Triton, TensorRT, Ray, or similar frameworks.
• Lead complex cross-organizational initiatives from architecture and design through production rollout, adoption, and operational maturity.
• Mentor senior technical leaders, raise the engineering bar, and influence AI infrastructure strategy across LinkedIn.

Department: Engineering
Function: Sales
Experience Level: Director