Fullstack AI Engineer

About the team

The Conversational Experience (CXD) team builds Raptor, Rakutens conversational AI platform serving multiple business units across Japan and global markets. Raptor handles production traffic from real customers — resolving queries, escalating to human agents when needed, and continuously improving through telemetry, evals, and BU feedback. Recent and ongoing work includes RAG over BU knowledge bases, multi-turn intent routing, LLM-based summarization, and LLM observability. We run production AI in front of real customers — not a research lab and not a demo factory. We care about resolution rate, escalation rate, P95 latency, hallucination rate, cost per session, and BU trust.

About the role

Were hiring a Senior AI Engineer to own slices of Raptors backend and AI layer end-to-end — retrieval, agent orchestration, evals, observability, and the admin tooling our internal users rely on. This role is 70% Python backend and AI engineering, 30% frontend (admin consoles, eval dashboards, BU-facing configuration UIs). If you want to build chat UIs as your day job, this isnt the role. If you want to make production chatbots actually work — and have the data to prove they do — read on.

What youll do

• Build and own AI features end-to-end — retrieval, agent/router logic, prompt design, tool-calling, and post-processing — from prototype to production traffic.
• Improve retrieval quality — chunking, embeddings, hybrid search, reranking, and grounded generation against BU knowledge bases. Measure hit rate, not vibes.
• Run the eval loop — offline eval harnesses, golden sets, LLM-as-judge graders, regression suites. Every prompt or model change ships behind an eval delta.
• LLM observability — instrument session-level traces, latency percentiles, retrieval hit rate, hallucination signals, intent-misrouting, dead-ends, and escalations.
• Latency and cost optimization — streaming, caching, model routing, batching, prompt compression. Cost-per-session is a first-class metric.
• Backend services — Python services (FastAPI, async workers) that handle real production load with proper error handling, retries, and graceful degradation.
• Spec-driven development — for non-trivial features, write a short design spec (problem, options, chosen approach, evals, rollout) before writing code. We use AI coding tools heavily but reject vibe-coding: specs and evals are how we keep speed without losing the plot.
• Partner with PMs, conversation designers, and BU stakeholders to translate ambiguous product intent into a working system. Push back when the data says the proposal wont work.
• Admin and eval tooling — React/TypeScript surfaces our team and BU stakeholders use to configure flows, inspect sessions, review eval runs, and ship changes safely.
• Guardrails and AI safety — input/output safety checks, PII handling, prompt-injection defenses, refusal patterns appropriate to each BU context.
• On-call and production support — debug live incidents, root-cause regressions, own postmortems.

What were looking for

Experience

• 5–8 years of software engineering experience, with at least 2 years building production LLM/AI applications that served real users (not demos).
• Strong Python backend experience — FastAPI or similar, async/await, Pydantic, background workers, testing discipline.
• Retrieval / RAG in production — vector stores (pgvector, Pinecone, Weaviate, or similar), hybrid search, reranking.
• Hands-on with at least one agent/orchestration framework (LangChain, LangGraph, LlamaIndex, or custom equivalent) and a clear point of view on when *not* to use multi-agent patterns.
• LLM evals — golden sets, regression harnesses, LLM-as-judge, gating releases on eval deltas.
• LLM observability — LangSmith, Langfuse, Arize, OpenTelemetry for LLMs, or equivalent home-grown tracing.
• SQL and warehouse data (BigQuery, Snowflake, Redshift, or similar) — chatbot work is data work.
• Working frontend skills in React + TypeScript sufficient to ship internal tooling and dashboards.
• Spec-driven development habit — comfortable writing short design docs and evals before code, using AI coding tools as accelerants rather than replacements for thinking.
• Bachelors degree in CS, Engineering, or equivalent practical experience.

Nice to have

• Experience shipping conversational AI for Japanese-language users, or other non-English markets.
• Experience with messaging platforms (WhatsApp Business, Slack, in-app chat) or telephony/voice channels.
• Fine-tuning, adapters, or distillation for cost/latency wins.
• Prompt-versioning and feature-flagged prompt rollouts.

How you work

• Comfortable with ambiguity — LLM products change weekly, and youll often be the one converging the spec.
• Strong written communication — our work crosses Tokyo, Seattle, and Bangalore time zones, and the written artifact is the artifact.
• Owner mindset — ship, measure, iterate without needing to be pushed.
• Comfortable disagreeing with stakeholders when the data supports it, and equally comfortable being wrong and updating.

Tech youll work with

• Backend / AI: Python, FastAPI, async workers, LangChain/LangGraph, frontier LLM APIs, pgvector / vector search, Postgres.
• Observability & evals: LLM tracing tools, custom eval harnesses, OpenTelemetry.
• Frontend: React, TypeScript, a modern component library — pragmatic, not religious.
• Data: SQL on the warehouse, event instrumentation.
• Infra: Containerized services, CI/CD, on-call rotation.

Why this role

Raptor is a real production system with real users and real BU customers. The work has a clear loop: ship a change, watch the metrics move (or not), learn, repeat. Youll work with strong engineers across Bengaluru, Tokyo, and Seattle on problems that matter to Rakutens customer experience.

Rakuten is committed to cultivating and preserving a culture of inclusion and connectedness. We are able to grow and learn better together with a diverse team and inclusive workforce. The collective sum of the individual differences, life experiences, knowledge, innovation, self-expression, and talent that our employees invest in their work represents not only part of our culture, but our reputation and Rakutens achievement as well. In recruiting for our team, we welcome the unique contributions that you can bring in terms of their education, opinions, culture, ethnicity, race, sex, gender identity and expression, nation of origin, age, languages spoken, veterans status, color, religion, disability, sexual orientation, and beliefs.