Principal Engineer - GPU and LLM Infrastructure

About this role:
Wells Fargo is seeking a Principal Engineer - GPU & LLM Infrastructure to lead the end-to-end strategy and operations of our enterprise GPU platforms within Digital Technology's AI Capability Engineering group. In this role, you will design and evolve GPU architecture across on-premises and cloud environments, guide POCs through production readiness, and oversee Day-2 operations for large-scale, multi-cloud deployments.
You will serve as the technical authority for Nvidia/Run:AI orchestration, drive alignment with OpenShift AI, and enable high-performance LLM/SLM inferencing using Triton and vLLM. A core part of the role is ensuring our GenAI platforms are secure, resilient, scalable, and fully observable to meet the demands of enterprise-grade AI workloads.
In this role, you will:

• Act as an advisor to leadership to develop or influence GPU buildout for highly complex business and technical needs across multiple groups
• Lead the strategy and resolution of highly complex and unique challenges requiring in-depth evaluation across multiple areas or the enterprise, delivering solutions that are long-term, large-scale and require vision, creativity, innovation, advanced analytical and inductive thinking
• Translate advanced technology experience, an in-depth knowledge of the organizations tactical and strategic business objectives, the enterprise technological environment, the organization structure, and strategic technological opportunities and requirements into technical engineering solutions
• Provide vision, direction and expertise to leadership on implementing innovative and significant business solutions
• Maintain knowledge of industry best practices and new technologies and recommends innovations that enhance operations or provide a competitive advantage to the organization
• Strategically engage with all levels of professionals and managers across the enterprise and serve as an expert advisor to leadership
• Design and implement GPU cluster topologies (H100/H200, NVLink/NVSwitch), networking, and storage paths for high-throughput inferencing; publish sizing and performance baselines.

Required Qualifications:

• 7+ years of Engineering experience, or equivalent demonstrated through one or a combination of the following: work experience, training, military experience, education

Desired Qualifications:

• 1+ years of experience with NVIDIA GPU and CUDA ecosystems, including CUDA, cuDNN, NVLink/NVSwitch, MIG, NCCL, GPU profilers, and performance tuning for H100/H200 architectures
• 1+ years of experience with LLM/SLM runtimes, such as vLLM, TensorRT-LLM, and Triton; hands-on work with model quantization (FP8, INT4 AWQ/GPTQ), KV-cache optimization strategies, and disaggregated prefill/decode pipelines
• 1+ years of experience in orchestration and GPU workload management, including GPU resource managers (collections/departments/projects/workloads), OCP/GKE administration, quota management, preemption and fair-share enforcement, GPU scheduling and timeslicing, Helm/Kustomize, upgrade validation, and admission controls
• 1+ years of experience with API and gateway platforms, including Apigee authentication/authorization, quota and rate-limit configuration, OpenAPI specifications, SDK generation, SLA operations, and API versioning/deprecation workflows
• 1+ years of experience in observability and evaluation tooling, including Arize-like systems for tracing and evaluations, SLO development, alerting design, retention/export workflows, and dashboard creation
• 1+ years of experience in performance engineering, including throughput and latency modeling (token/sec, batch shaping, cache policies) and cost/performance optimization strategies for LLM/SLM workloads

Job Expectations:

• Hybrid onsite at required locations
• No visa sponsorship available.
• No relocation assistance for this position.

Pay Range
Reflected is the base pay range offered for this position. Pay may vary depending on factors including but not limited to demonstrated examples of prior performance, skills, experience, or work location. Employees may also be eligible for incentive opportunities.
$159,000.00 - $305,000.00
Benefits
Wells Fargo provides eligible employees with a comprehensive set of benefits, many of which are listed below. Visit Benefits - Wells Fargo Jobs for an overview of the following benefit plans and programs offered to employees.

• Health benefits
• 401(k) Plan
• Paid time off
• Disability benefits
• Life insurance, critical illness insurance, and accident insurance
• Parental leave
• Critical caregiving leave
• Discounts and savings
• Commuter benefits
• Tuition reimbursement
• Scholarships for dependent children
• Adoption reimbursement

Posting End Date:
5 Feb 2026
* Job posting may come down early due to volume of applicants.
We Value Equal Opportunity
Wells Fargo is an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, disability, status as a protected veteran, or any other legally protected characteristic.
Employees support our focus on building strong customer relationships balanced with a strong risk mitigating and compliance-driven culture which firmly establishes those disciplines as critical to the success of our customers and company. They are accountable for execution of all applicable risk programs (Credit, Market, Financial Crimes, Operational, Regulatory Compliance), which includes effectively following and adhering to applicable Wells Fargo policies and procedures, appropriately fulfilling risk and compliance obligations, timely and effective escalation and remediation of issues, and making sound risk decisions. There is emphasis on proactive monitoring, governance, risk identification and escalation, as well as making sound risk decisions commensurate with the business unit's risk appetite and all risk and compliance program requirements.
Applicants with Disabilities
To request a medical accommodation during the application or interview process, visit Disability Inclusion at Wells Fargo .
Drug and Alcohol Policy
Wells Fargo maintains a drug free workplace. Please see our Drug and Alcohol Policy to learn more.
Wells Fargo Recruitment and Hiring Requirements:
a. Third-Party recordings are prohibited unless authorized by Wells Fargo.
b. Wells Fargo requires you to directly represent your own experiences during the recruiting and hiring process.