Senior Engineer, Agent Harness Engineering (IR-557)
An AI agent that applies to roles like this on employers' own sites. 25 applications free, no card needed.
AI summary of this job
Beta Read in 10 secondsA Senior Engineer, Agent Harness Engineering role at Intellectsoft in Bengaluru, Karnataka, India for people with 5–10 years of experience. You'll build agent harnesses, deliver production systems, and manage evaluation infrastructure using Python, SQL, Pandas, LLM, RAG, Data Engineering, AWS, Azure, Terraform, GitHub Actions, CI/CD, DevOps, GitHub, PyTest, and Consulting.
- Bengaluru
- Work from office
- 5–10 years
What you'll do
- Design and build core agent harnesses
- Take agentic systems from initial concept to production
- Build and maintain rigorous evaluation suites
Good fit if you
- Have 5–10 years of experience
- Bring strong Python and software engineering skills
- Know LLM and agent frameworks
AI-generated from the job description. Check the full description before you apply.
Job Description
Intellectsoft is a software development company delivering innovative solutions since 2007. We operate across North America, Latin America, the Nordic region, the UK, and Europe. We specialize in industries like Fintech, Healthcare, EdTech, Construction, Hospitality, and more, partnering with startups, mid-sized businesses, and Fortune 500 companies to drive innovation and scalability. Our clients include Jaguar Motors, Universal Pictures, Harley-Davidson, and many more where our teams are making daily impactTogether, our team delivers solutions that make a difference. Learn more at www.intellectsoft.net
Our customer advises global pharmaceutical and life sciences clients across commercial strategy, clinical development, regulatory operations, and medical affairs. An increasing share of this work is powered by AI agents designed to handle complex scientific and commercial datasets. This role is split roughly evenly between internal research (developing reusable agent harness components, benchmark evaluations, and reliability frameworks) and client delivery (architecting, deploying, and scaling production-ready agentic systems).
Requirements
- Professional Experience: 3 years of professional software engineering experience, including substantive work building LLM-based or agentic systems (beyond prompt-level experimentation).
- Production Deployment: Proven track record of deploying user-facing AI applications to production, including operational experience with monitoring, versioning, rate limiting, cost control, and incident response.
- Core Software Engineering & Python: Strong Python skills with production-grade test coverage, typing, asyncio, Pydantic, Pytest, packaging, version control, CI/CD, and robust API design.
- LLM & Agent Frameworks: Hands-on experience with LLM APIs, function/tool calling, structured outputs, streaming, and context window management. Experience building or extending orchestration frameworks (such as LangGraph, deepagents, OpenAI Agents SDK, or custom harnesses) with a clear view on framework utility.
- Non-Deterministic Evaluation: Ability to define success criteria, design evaluation datasets, and systematically evaluate non-deterministic AI outputs.
- Cloud & Shipping: Working knowledge of containerization and at least one major cloud provider to ship and maintain services independently.
- Communication & Adaptability: Strong written and verbal communication skills with the ability to represent technical positions credibly and thrive amid shifting project priorities.
Nice to have
- Domain Knowledge: Prior exposure to healthcare, pharma, or biotech data (clinical trial data, real-world evidence, regulatory submissions, pharmacovigilance, or commercial analytics) and standards (CDISC, SNOMED, MedDRA).
- Regulated Environments: Familiarity with GxP, 21 CFR Part 11, HIPAA, GDPR, audit logging, and formal system validation.
- Observability & Protocols: Hands-on experience with LLM tracing tools (LangSmith, Langfuse, Braintrust, OpenTelemetry) or developing and hosting MCP servers.
- Retrieval & Models: Experience with RAG pipelines, vector search (pgvector, Qdrant, Weaviate), fine-tuning, distillation, or model serving (vLLM, Bedrock, Vertex AI).
- DevOps & Data Engineering: Infrastructure as Code (Terraform, GitHub Actions, AWS/Azure) alongside large-scale data processing using Polars, Pandas, DuckDB, and advanced SQL.
- Consulting & Ecosystem Engagement: Prior consulting/client services experience, active use of agentic developer tooling (such as Claude Code), or open-source contributions/publications in agent architectures and evaluations.
Responsibilities
- Harness & Orchestration Architecture: Design and build core agent harnesses, covering function calling, tool interfaces, context assembly, retrieval integration, subagent delegation, retry logic, and failure handling.
- Prototype to Production Delivery: Take agentic systems from initial concept to scalable production, managing deployment, monitoring, cost, latency, and operational support against live enterprise client data.
- Evaluation Infrastructure: Build and maintain rigorous evaluation suites, task benchmarks, scoring rubrics, and regression tests using tools like Langfuse, Pydantic Logfire, or custom SDKs to drive architectural decisions.
- System & Tool Integration: Integrate client enterprise systems, data sources, and internal APIs as accessible agent tools, including building and managing Model Context Protocol (MCP) servers.
- Solution Consulting & Requirements Translation: Translate ambiguous client requirements into robust agent architectures, clearly advising stakeholders when an agentic pattern is—or is not—the right solution.
- Stakeholder Communication: Interface directly with client stakeholders and articulate technical trade-offs clearly to non-technical audiences.
- Tooling Transfer & Documentation: Contribute reusable harness components to shared internal tooling and document design decisions to meet audit, validation, and handover standards.
Benefits
- Employment-based cooperation
- Comprehensive insurance for you and your family (health, life, and accident)
- Paid PTO policy (vacation, sick leaves, and public holidays)
- Tech equipment provided
- Udemy courses, workshops, trainings & expert knowledge-sharing
- Flexible hours & work setup
Skills
PythonSQLPandasLLMRAGData EngineeringAWSAzureTerraformGitHub ActionsCI/CDDevOpsGitHubPyTestConsultingIf a job posting appears fraudulent, asks for payment, contains misleading information, or violates our guidelines, please report it immediately. Our team will review it promptly, Jobaaj does not charge any fee from the applicants.
Similar Jobs
View allLooking for something similar?
New IT / Software Development & Related roles in Bengaluru.
One email a day. Unsubscribe any time.

