Senior AI Engineer

Department Icon IT / Software Development & Related
Posted: 13 hours ago
8-10 years
Bengaluru / Bangalore, Karnataka
IT / Software Development & Related
work from office
Full Time
Closes 15 Jan 2027

Posted: 13 hours ago
Let Jobaaj One apply for you

An AI agent that applies to roles like this on employers' own sites. 25 applications free, no card needed.

Job Description
About Company
Similar Jobs
Please verify your account first! Send OTP

Job Description

About the Team & Mission

The ML Platform team maximizes the speed, throughput, and hardware efficiency of our production Large Language Model (LLM) inference stack. As models scale and real-time demand grows, post-training optimization and aggressive server-level utilization are our primary levers for delivering low-latency AI at sustainable infrastructure economics.

We operate at the intersection of LLM serving engines, post-training compression, and bare-metal accelerator architecture. Our mission is to squeeze maximum compute and memory bandwidth out of every server node—turning heavy foundation models into ultra-fast, cost-efficient endpoints.

Role Overview

We are hiring a deeply technical Senior / Staff Software Engineer (Individual Contributor) with 8+ years of experience to partner directly with the Leadership in architecting and scaling our LLM inference platform.

In this high-leverage IC role, you will serve as our primary technical anchor for inference performance. You will own end-to-end serving architecture, drive post-training optimization pipelines, eliminate hardware bottlenecks, and guide engineers through complex systems challenges.

Key Responsibilities

  • Post-Training Optimization: Architect and productionize model compression and acceleration pipelines—including quantization (FP8/INT8/INT4), distillation, and speculative decoding—to slash latency (TTFT/ITL) while preserving model quality.
  • Hardware & Server Maximization: Drive bare-metal utilization across multi-GPU and multi-node topologies by optimizing memory bandwidth, tensor core occupancy, interconnects, and parallelism strategies (tensor, pipeline, expert, and disaggregated serving).
  • Inference Engine & Kernel Engineering: Extend modern inference frameworks (such as vLLM or SGLang) and build custom fused kernels and batching schedulers to eliminate compute, memory, and CPU host bottlenecks.
  • Profiling & Automated Diagnostics: Build deep telemetry, roofline modeling, and automated bottleneck-detection workflows to pinpoint hardware stalls and prevent performance regressions in production.
  • Technical Leadership: Partner with Leadership to shape our multi-quarter inference roadmap, author high-clarity RFCs and influence hardware capacity decisions.
  • Mentor Junior Engineers: Level up junior and mid-level engineers through code reviews, pair programming, and hands-on systems coaching.

Minimum Qualifications

  • Education: Bachelors or Masters degree in Computer Science, Computer Engineering, Electrical Engineering, or equivalent practical systems experience.
  • Experience:8+ years of hands-on software engineering experience in high-performance computing (HPC), ML inference systems, GPU programming, or low-latency distributed infrastructure.
  • LLM Inference Mastery: Proven track record optimizing large-scale LLM serving in production, with a deep understanding of transformer memory footprints, KV-cache mechanics, continuous batching, and latency/throughput trade-offs.
  • Hardware & Systems Depth: Strong grasp of accelerator microarchitecture (compute vs. memory-bound workloads, roofline analysis, memory hierarchies) and multi-GPU server interconnects.
  • Programming Proficiency: Expert-level coding skills in C++ and Python, with experience building low-overhead, production-critical systems software.

Preferred Qualifications

  • GPU & Compiler Internals: Practical experience writing or profiling custom accelerator kernels and working close to ML compilers or runtime execution graphs.
  • Production Compression & Decoding: Track record deploying quantized models or speculative decoding techniques in high-throughput production environments.
  • Open-Source Inference Familiarity: Experience working inside or contributing to modern open-source LLM serving engines and attention libraries.
  • Complex Workload Scaling: Background optimizing Mixture-of-Experts (MoE) routing overhead or ultra-long-context inference workloads.
  • Automated Performance Tooling: Experience building automated benchmarking harnesses, hardware utilization models, or AI-assisted trace debugging tools.

Why Flipkart

Flipkart is Indias leading e-commerce platform, where innovation meets impact at scale. As part of the AI Solutions team, you will shape the foundational AI infrastructure powering experiences for 500M+ users, millions of sellers, and complex supply chains. Flipkart is part of the Walmart-owned Flipkart Group, which includes Flipkart Wholesale, Flipkart Health+, Cleartrip, Shopsy, Flipkart Travels, and Myntra. Join us to build the platforms that make AI at Flipkart possible.

Skills

CPythonDebuggingSoftware Engineer

If a job posting appears fraudulent, asks for payment, contains misleading information, or violates our guidelines, please report it immediately. Our team will review it promptly, Jobaaj does not charge any fee from the applicants.

About Company

Flipkart is a leading e-commerce company committed to transforming commerce in India through technology and innovation.

Similar Jobs

View all
Loading...

Looking for something similar?

Bag Logo
Jobaaj
Get jobs like this by email

New IT / Software Development & Related roles in Bengaluru / Bangalore.

Job Alert
Google hiring for Specific Roles Apply Now!
1 min ago
New Opportunity
Amazon is hiring freshers Apply Now!
5 min ago
Featured Jobs
Microsoft opening 50+ positions Apply Now!
10 min ago

One email a day. Unsubscribe any time.