Associate Director Of Engineering – Conversational AI

Company Overview

Rakuten Group is a global leader in digital innovation, pioneering advancements across E-commerce, Fintech, Telecommunications, and AI-driven services. With a strong commitment to technology, data intelligence, and customer-centric solutions, Rakuten continues to reshape experiences worldwide.

Our AI & Data Division plays a pivotal role in this transformation, driving cutting-edge research and large-scale deployment of Large Language Models (LLMs), Generative AI, Conversational AI, and advanced Data Analytics. By harnessing AI-powered automation, intelligent search, and predictive analytics, we are building scalable, high-impact solutions that enhance customer interactions, optimize business processes, and unlock new opportunities across the Rakuten ecosystem.

Department Overview

The Conversational Experiences Department (CXD) is dedicated to building production-ready, scalable, and high-performance AI systems that redefine Conversational AI. Our focus is on:

? Conversational AI Frameworks Traditional NLP-powered chatbots, AI-driven search, and multi-modal assistants.

? Next-Generation GenAI Platforms LLM-powered chat experiences, voice and multi-modal AI, real-time response generation.

? AI Engineering at Scale Deploying robust AI models in production environments, ensuring high availability, low latency, and enterprise-grade security.

We are looking for a technical leader who brings deep expertise in both AI/ML algorithms and scalable engineering systemssomeone who has built traditional Conversational AI platforms and is now transitioning to GenAI-driven, multi-modal AI solutions.

Position Overview

As Associate Director of Engineering Conversational AI, you will lead a team of AI engineers and software architects to develop and scale enterprise-grade Conversational AI systems. This role requires expertise in AI model development, cloud-scale engineering, and production readiness.

This is an engineering-first leadership role, with a strong algorithmic focus on NLP, LLMs, and Deep Learning models. The ideal candidate has hands-on experience designing and deploying AI-powered applications in high-traffic, real-time environments.

Key Responsibilities

1. AI Productization & Engineering Leadership

• Lead development of Next Generation LLM/ Multimodal driven Chatbots & IVR using Agentic Frameworks
• Architect and develop Conversational AI systems that are highly scalable, secure, and efficient.
• Oversee MLOps/ LLMOps pipelines, AI deployment, and cloud-native architectures, optimizing AI workloads for latency, accuracy, and cost efficiency.

2. Algorithmic Advancements & Model Innovation

• Drive end-to-end development of AI/ML models, from data collection to inference optimization.
• Develop and optimize multi-modal AI models that combine text, voice, and visual inputs.
• Implement advanced Retrieval-Augmented Generation (RAG) techniques and Reinforcement Learning for AI-driven conversations.

3. Engineering Excellence & Production Readiness

• Ensure low-latency, high-availability AI services that scale to millions of users.
• Design cloud-based microservices architectures for AI applications using Kubernetes, Docker, and serverless computing.
• Optimize LLM inference with quantization, distillation, and model caching techniques to reduce compute costs and improve performance.
• Ensure the engineering team delivers high-quality software solutions that adhere to defined production SLOs and maintain high standards of quality and reliability.

4. Leadership & Team Development

• Build, mentor, and scale a high-performance AI engineering team, fostering innovation and technical excellence.
• Drive cross-functional collaboration with product, UX, and data science teams to align AI capabilities with business goals.
• Establish best practices for CI/CD in AI development, automated model retraining, and continuous evaluation frameworks.

Collaborate extensively with Product & Project Management, and Customer Success Management to ensure a seamless alignment between engineering output and overall business strategy

Mandatory Qualifications

• 15-20 years of software engineering experience, including 5+ years in AI/ML leadership roles.
• Deep expertise in:
• Conversational AI & NLP (LLMs, Transformers, RNNs, Semantic Search)
• GenAI Frameworks (GPT-4, Claude, Llama, Gemini, Hugging Face, LangChain. Langgraph)
• AI Model Deployment at scale (TensorFlow, PyTorch, ONNX, OpenAI API)
• Cloud Platforms (GCP, Azure, AWS) and MLOps pipelines (Kubeflow, MLflow)
• AI-Powered Search (vector databases, semantic search, retrieval-based chat)
• Microservices development Java, Python, Spring Boot, NoSQL, Distributed Systems
• Experience building and deploying large-scale, production AI systems in high-traffic environments.
• Strong leadership, with the ability to hire, mentor, and scale AI engineering teams.

Preferred Qualifications

• Experience in both traditional Conversational AI and next-gen GenAI-based assistants
• Expertise in AI-driven personalization, AI safety, and responsible AI practices
• Strong software architecture knowledge, including microservices, distributed computing, and API-driven AI deployment
• Experience working in startups or high-growth AI-first organizations, leading AI-driven transformation