Research Associate - AI Evaluations

Department Icon Corporate Finance
Posted: 1 week ago
0-2 years
Bengaluru, India
Corporate Finance
work from office
Full Time
Closes 3 Jan 2027

Posted: 1 week ago
Let Jobaaj One apply for you

An AI agent that applies to roles like this on employers' own sites. 25 applications free, no card needed.

AI Summary
Job Description
About Company
Similar Jobs
Please verify your account first! Send OTP

AI summary of this job

Beta Read in 10 seconds

A Research Associate - AI Evaluations role at Karya in Bengaluru for people with 0–2 years of experience. You'll help design and run community-facing research that shapes how AI systems are evaluated. You'll contribute to research workstreams end-to-end using Python, Go, R, SQL, Networking, Recruitment, Logistics, Project Management, Communication, and Teams.

  • Bengaluru
  • Work from office
  • 0–2 years
  • Social Science degree

What you'll do

  • Contribute to research design and methods
  • Build and adapt research toolkits
  • Plan and conduct focus group discussions

Good fit if you

  • Have a degree in a social science discipline
  • Bring at least 2–3 years of research experience
  • Show fluency in spoken and written Hindi
Sounds like you? Applying takes under a minute.

AI-generated from the job description. Check the full description before you apply.

Helpful?

Job Description

About The Role

At Karya, we’re building AI evaluation systems that measure and understand whether AI systems actually work for the communities they’re meant to serve. We’re looking for a Research Associate to help design and run community-facing research that shapes how AI systems are evaluated. You’ll contribute to research workstreams end to end: shaping the research design, building toolkits, running fieldwork, and turning what you learn into evaluation frameworks and datasets. You’ll contribute to a range of evaluation projects across the team. These could include building a gender-bias benchmark for AI chatbots in domains like agriculture; conducting grounded research into what people in India actually use AI for, in which languages and in what contexts, to build evaluation categories from real use; or contributing to conceptual and methods work on what “community” and “participatory” should mean in AI evaluation, pressure-tested against fieldwork with Karya’s worker community. 

The role suits someone with roots in social science who has worked where technology, impact and AI meet, and who cares about gender equity and doing research with communities.

Concretely, this means:

  • Contribute to research design, including research questions, methods, site selection and sampling, working closely with project leads
  • Build and adapt research toolkits: discussion guides, oral consent scripts in local languages, training materials and feedback tools
  • Plan and conduct focus group discussions, interviews and pilots with communities, including participant recruitment and field logistics
  • Coordinate with field teams during data collection, keeping research consent-first and respectful of participants’ time and context
  • Clean, cull and analyse qualitative and quantitative data through a gender and intersectional lens
  • Synthesise insights into findings that strengthen evaluation frameworks, datasets and toolkits
  • Contribute to desk research, reports and external outputs
  • Support project management by tracking timelines, deliverables and decisions across internal teams (Ops, Tech) and external partners

Why This Role

You’ll be embedded with the Evaluations team within Karya’s tech team, where social science and engineering work as one practice. We believe you can’t judge whether an AI system works for people without understanding both the people and the system, and how social and technical structures shape each other. So researchers and engineers on our team work closely together, each bringing what the other can’t. Your research will shape what gets built: community insights become evaluation categories, fieldwork defines what a good answer looks like, and research design decides whose voices go into a dataset. Working alongside engineers, you’ll learn how AI systems are built, tested and deployed, and how they reach the communities we work with. If you’re a social scientist who wants to understand AI from the inside, this is a good place to do it. 

What We Are Looking For

Qualifications

  • At least 2–3 years of research experience, ideally at a tech, impact or AI-focused organisation
  • A degree in a social science discipline (e.g., sociology, development studies, gender studies, anthropology), or equivalent experience
  • Grounding in gender and intersectionality (caste, class, region) through study, research or practice
  • Experience contributing to research design and delivering research projects
  • Hands-on qualitative and field research experience, including running FGDs
  • Working data skills: SQL, quantitative analysis, and some coding (e.g., Python or R)
  • Fluency in spoken and written Hindi
  • Willingness to travel for fieldwork

Skills And Competencies

  • Applied AI/ML literacy: enough to engage meaningfully with model evaluation, human-in-the-loop systems and language technologies
  • Socio-technical systems thinking, meaning you understand technology and society as shaping each other rather than as separate concerns
  • Strong written and verbal communication in Hindi and English
  • Comfort working in ambiguous, early-stage research settings

Good to have

  • Proficiency in Indian languages beyond Hindi
  • Familiarity with agriculture or rural livelihoods
  • Exposure to participatory methods 

People matter at Karya and these are some of the perks and benefits we created for our team:

  • Flexible vacation and leave policy
  • Flexible work options
  • Insurance as per industry standards
  • Access to industry stalwarts and networking opportunities

Karya is an equal opportunity and affirmative action employer.

Qualified applicants will receive consideration without regard to their race, colour, religion, sex, sexual orientation, gender identity and disability.

Karya invites all qualified interested applicants to apply for career opportunities. If you are a person with a disability and need a reasonable accommodation to use our search to apply for a career opportunity, reach out at hr@karya.in.

Skills

PythonGoRSQLNetworkingRecruitmentLogisticsProject ManagementCommunicationTeams

If a job posting appears fraudulent, asks for payment, contains misleading information, or violates our guidelines, please report it immediately. Our team will review it promptly, Jobaaj does not charge any fee from the applicants.

About Company

Karya is a technology platform that empowers low-income communities by providing them with digital work, fair wages, and upward economic mobility, while helping companies collect high-quality data for AI training.

Similar Jobs

View all
Loading...

Looking for something similar?

Bag Logo
Jobaaj
Get jobs like this by email

New Corporate Finance roles in Bengaluru.

Job Alert
Google hiring for Specific Roles Apply Now!
1 min ago
New Opportunity
Amazon is hiring freshers Apply Now!
5 min ago
Featured Jobs
Microsoft opening 50+ positions Apply Now!
10 min ago

One email a day. Unsubscribe any time.