AI Scraping & LLM Engineering Manager – Python ( 9 To 15 Yrs )
An AI agent that applies to roles like this on employers' own sites. 25 applications free, no card needed.
Job Description
AI Scraping & LLM Engineering Manager – Python
Experience: 9 to 15 Years
Location: Remote (Work from Home) / Bangalore / India
Mode of Engagement: Full-time
No of Positions: 2
Educational Qualification: B.E / B.Tech / M.Tech / MCA / Computer Science / IT
Industry: AI / Data Engineering / Enterprise Automation / SaaS / LegalTech
Notice Period: Immediate / 15 Days Preferred
What We Are Looking For:
- 9–15 years of experience in building large-scale web scraping, browser automation, and data extraction systems using Python.
- Proven expertise in Scrapy, Selenium, Playwright, Puppeteer, and modern browser automation frameworks for enterprise-grade scraping projects.
- Strong experience in building AI-powered scraping workflows using LLMs for intelligent extraction, data normalization, classification, and automation.
- Experience in designing scalable distributed scraping pipelines with proxy rotation, anti-bot handling, CAPTCHA solving, and high-concurrency crawling systems.
- Proven ability to lead engineering teams, manage delivery timelines, conduct code reviews, and drive architecture decisions for AI/data platforms.
Responsibilities:
- Lead the architecture and development of AI-driven scraping and browser automation platforms for structured and unstructured web data extraction.
- Build intelligent scraping pipelines leveraging LLMs, LangChain, LlamaIndex, OpenAI, Claude, or open-source AI models for automated extraction and enrichment.
- Design and manage scalable scraping infrastructure using Scrapy, Selenium, Playwright, Docker, Kubernetes, queues, and cloud-native deployment systems.
- Collaborate with AI, product, and engineering teams to create next-generation intelligent data acquisition and automation solutions.
- Mentor and manage scraping/data engineering teams while establishing coding standards, best practices, monitoring, and scalable engineering processes.
- Optimize system performance, reliability, scalability, and deployment workflows using AWS/GCP, CI/CD pipelines, databases, and distributed processing systems.
Qualifications:
- Bachelors or Masters degree in Computer Science, Engineering, IT, or related field.
- Strong hands-on expertise in Python, Scrapy, Selenium, Playwright, FastAPI/Django, APIs, and scalable scraping architectures.
- Experience working with LLMs, AI automation frameworks, vector databases, semantic search, embeddings, and intelligent extraction systems.
- Proficiency in Docker, Kubernetes, AWS/GCP, PostgreSQL/MySQL/MongoDB, Redis, CI/CD, and cloud infrastructure management.
- Prior experience leading engineering teams and delivering enterprise-scale scraping, AI automation, or data intelligence products.
- Excellent analytical, communication, leadership, and problem-solving skills with experience working in Agile development environments.
Skills
PythonCloud InfrastructureData ExtractionMysqlDjangoAiIf a job posting appears fraudulent, asks for payment, contains misleading information, or violates our guidelines, please report it immediately. Our team will review it promptly, Jobaaj does not charge any fee from the applicants.
About Company
Similar Jobs
View allLooking for something similar?
New Data Science Analytics & Machine Learning roles in Bengaluru / Bangalore.
One email a day. Unsubscribe any time.

