Data Engineer (ETL Specialist)

Department Icon Data Science Analytics & Machine Learning
Posted: 1 month ago
2-5 years
MAA, India
Data Science Analytics & Machine Learning
work from office
Full Time
Closes 21 Nov 2026

Posted: 1 month ago
Let Jobaaj One apply for you

An AI agent that applies to roles like this on employers' own sites. 25 applications free, no card needed.

Job Description
About Company
Similar Jobs
Please verify your account first! Send OTP

Job Description

POSITION SUMMARY

BOT VFX, a visual effects services company with global clients, is looking for a Data Engineer (ETL Specialist) as a full-time contract role. This is strictly an on location role in Chennai, India.
In this role, you will be responsible for transforming petabytes of unstructured, legacy VFX data (including heavy video plates, multi-layered OpenEXR sequences, and proprietary DCC workfiles) into highly structured, sanitized, and compressed datasets.
The ideal candidate brings deep expertise in large-scale data engineering and ETL pipeline architecture alongside an understanding of physical infrastructure limits. You will bridge legacy storage/tape systems, VFX pipeline scripts, and Creative Innovation needs. Reporting to the CIO, the role works closely with IT/IO and Creative Innovation teams.

POSITION RESPONSIBILITY
  • Lead the architecture, implementation, and automation of scalable ETL/ELT pipelines capable of moving and processing tens of terabytes of unstructured media data daily.
  • Implement automated "Purge & Keep" business rulesets to systematically clean directory trees, wiping intermediate render caches, temporary scratch files, and duplicates while preserving core visual assets.
  • Build high-throughput processing pipelines to transcode, compress, and restructure massive image sequences (linear EXR, DPX) and video formats into AI-friendly data structures (e.g., WebDataset shards).
  • Integrate and normalize lightweight metadata scrapings (camera telemetry, tracking curves, geometry tags) extracted from VFX workfiles (Silhouette, Nuke) and bind them deterministically to their corresponding visual assets.
  • Manage high-velocity data movement across tiered storage layers (Hot NVMe/SSD staging, Warm Object Storage, and Deep Cloud/Tape Archives), ensuring the pipeline never bottlenecks network bandwidth or staging limits.
  • Establish robust data quality assurance (QA) validation loops within the pipeline to catch corrupted frames, missing frame ranges, and accidental inclusion of restricted IP or sensitive assets.

REQUIRED SKILLS
  • At least 3–5 years of experience as a Data Engineer building large-scale, automated ETL/ELT pipelines, with a proven track record of handling unstructured data, heavy media assets, or computer vision datasets.
  • Strong technical grounding in pipeline development using Python (and bash/shell scripting) along with heavy data processing frameworks (e.g., Apache Spark, Ray, Dask).
  • Hands-on experience with modern data architectures, containerization (Docker, Kubernetes), and cloud/on-prem block/object storage systems.
  • Solid problem-solving capabilities focused on maximizing throughput under strict hardware constraints (balancing egress speeds with staging capacities).
  • Exceptional communication skills with the ability to collaborate effectively across multidisciplinary teams, including VFX technical directors, systems engineers, and AI research teams.

Preferred Skills
  • Basic familiarity with VFX file structures, directory archetypes, image sequence formatting, or open-source media utilities (FFmpeg, OpenEXR toolset).

Skills

PythonBashShell ScriptingComputer VisionData EngineeringApache SparkETLDockerKubernetesApacheQAQuality AssuranceCommunicationTeams

If a job posting appears fraudulent, asks for payment, contains misleading information, or violates our guidelines, please report it immediately. Our team will review it promptly, Jobaaj does not charge any fee from the applicants.

About Company

BOT VFX is a premier visual effects (VFX) studio providing high-end VFX services for feature films, television, and commercials globally.

Similar Jobs

View all
Loading...

Looking for something similar?

Bag Logo
Jobaaj
Get jobs like this by email

New Data Science Analytics & Machine Learning roles in MAA.

Job Alert
Google hiring for Specific Roles Apply Now!
1 min ago
New Opportunity
Amazon is hiring freshers Apply Now!
5 min ago
Featured Jobs
Microsoft opening 50+ positions Apply Now!
10 min ago

One email a day. Unsubscribe any time.