Spark Engineer

About the job Role: Spark Engineer Experience: 08 to 12yrs Notice Period: 60-90 Days Work Location: TCS-Pune, Hyderabad Mode of Interview: Virtual Responsibilities :- Seeking an experienced Spark Engineer with 8–12 years of IT experience in enterprise data engineering and ETL solutions. Strong hands-on expertise in PySpark / Apache Spark for developing and optimizing large-scale data processing applications. Extensive experience with IBM DataStage for designing, developing, and maintaining complex ETL workflows and data pipelines. Proficiency in Python for data processing, automation, transformation, and reusable utilities. Strong working knowledge of Linux/Unix and Shell Scripting for job execution, automation, monitoring, and troubleshooting. Experience in designing and implementing high-volume batch and distributed data processing pipelines. Good understanding of Spark performance tuning, including partitioning, caching, memory optimization, and query optimization. Strong knowledge of SQL, data transformation, data quality, and ETL/ELT concepts. Ability to troubleshoot DataStage and Spark jobs, analyze failures, and improve pipeline performance and reliability. Experience working with Git, CI/CD practices, production deployments, and enterprise data engineering environments is preferred. Strong problem-solving, debugging, communication, and collaboration skills, with the ability to work with cross-functional teams. Note : Ex TCSer can't apply