An AI agent that applies to roles like this on employers' own sites. 25 applications free, no card needed.
AI summary of this job
Beta Read in 10 secondsAn NLP Data Engineer role at AiLogic Neural Network Pvt Ltd in Hyderabad, Telangana, India for people with 2–5 years of experience. You'll build scalable data processing pipelines for NLP and LLM applications using Python, SQL, HTML, NLP, PyTorch, Hugging Face, LLM, RAG, Data Science, Data Engineering, Apache Spark, ETL, CI/CD, Apache, and Git.
- Hyderabad
- Work from office
- 2–5 years
- Bachelor's or Master's
What you'll do
- Design and maintain scalable data pipelines
- Build document ingestion and processing workflows
- Collaborate with AI/ML engineers on datasets
Good fit if you
- Have strong experience with Python programming
- Know NLP concepts like tokenization and text processing
- Understand Git and CI/CD practices
AI-generated from the job description. Check the full description before you apply.
Job Description
About the job About Us AiLogic Neural Network Pvt Ltd is an AI-driven product company focused on building advanced language technology solutions, including machine translation, document intelligence, and large-scale NLP systems. We are looking for a highly motivated Data Engineer to join our growing AI team and contribute to the development of scalable data processing pipelines for NLP and LLM applications. Roles & Responsibilities Design, develop, and maintain scalable data pipelines for processing large volumes of structured and unstructured data. Build document ingestion and processing workflows for PDFs, scanned documents, HTML pages, and other text sources. Implement OCR, PDF parsing, HTML parsing, and text extraction pipelines. Develop document chunking and preprocessing frameworks for NLP and LLM-based applications. Work with Hugging Face models and NLP libraries for text processing tasks. Create and optimize data transformation workflows using Python, Apache Spark, and Spark SQL. Develop and manage Vector Database pipelines for embedding storage and retrieval. Implement text normalization, sentence segmentation, deduplication, and data quality processes. Design and implement data masking, classification, and categorization solutions. Collaborate with AI/ML engineers to prepare datasets for model training and inference. Optimize large-scale data processing workflows for performance, scalability, and cost efficiency. Maintain CI/CD pipelines and follow software engineering best practices. Monitor, troubleshoot, and improve production data processing systems. Mandatory Skills Strong experience with Python programming. Hands-on experience in NLP concepts such as: Tokenization Text Processing Hugging Face Transformers Experience in: PDF Parsing OCR HTML Parsing Text Extraction Document Chunking Experience with Apache Spark and Spark SQL. Working knowledge of Vector Databases. Good understanding of Git and CI/CD practices. Experience building data pipelines and ETL workflows. Strong debugging and problem-solving skills. Preferred Skills Text Normalization Sentence Segmentation Exact Deduplication and Near Deduplication Data Masking Data Classification & Categorization Embedding Generation and Retrieval Pipelines Large-scale Document Processing Systems RAG (Retrieval-Augmented Generation) Pipelines Performance Optimization Skills CPU Distribution and Parallel Processing Pre-batch Generation Techniques Chunking Optimization Strategies Stream Processing vs Batch Processing GPU and CPU Parallel Distribution CUDA Optimization PyTorch Performance Tuning Spark Performance Optimization Qualifications Bachelor's or Master's degree in Computer Science, Data Science, Information Technology, or a related field. 2–3 years of experience in Data Engineering, NLP Engineering, or AI Data Processing. Experience working with large-scale datasets and distributed computing frameworks.
Skills
PythonSQLHTMLNLPPyTorchHugging FaceLLMRAGData ScienceData EngineeringApache SparkETLCI/CDApacheGitIf a job posting appears fraudulent, asks for payment, contains misleading information, or violates our guidelines, please report it immediately. Our team will review it promptly, Jobaaj does not charge any fee from the applicants.
Similar Jobs
View allLooking for something similar?
New Data Science Analytics & Machine Learning roles in Hyderabad.
One email a day. Unsubscribe any time.

