Data Engineer - AWS & Python

Department Icon Data Science Analytics & Machine Learning
Posted: 1 year ago
6-10 years
Tamil Nadu, Vellore
Data Science Analytics & Machine Learning
Work from Office
Full Time

Posted: 1 year ago
This job's closing date (23 Oct 2025) has passed, so the role may already be filled. See live similar jobs.
Let Jobaaj One apply for you

An AI agent that applies to roles like this on employers' own sites. 25 applications free, no card needed.

Job Description
Similar Jobs
Please verify your account first! Send OTP

Job Description

    As a Data Engineer, you will be responsible for designing, developing, and optimizing data pipelines and ETL workflows using AWS Glue, AWS Lambda, and Apache Spark. Your role will involve implementing big data processing solutions utilizing AWS EMR and AWS Redshift. You will also be tasked with developing and maintaining data lakes and data warehouses on AWS, including S3, Redshift, and RDS.

    Ensuring data quality, integrity, and governance will be a key aspect of your responsibilities, which will be achieved through leveraging AWS Glue Data Catalog and AWS Lake Formation. It will be essential for you to optimize data storage and processing for both performance and cost efficiency. Working with structured, semi-structured, and unstructured data across various storage formats such as Parquet, Avro, and JSON will be part of your daily tasks.

    Automation and orchestration of data workflows using AWS Step Functions and Apache Airflow will also fall within your scope of work. You will be expected to implement best practices for CI/CD pipelines in data engineering with AWS CodePipeline and AWS CodeBuild. Monitoring, troubleshooting, and optimizing data pipeline performance and scalability will be critical to ensuring smooth operations.

    Collaborating with cross-functional teams, including data scientists, analysts, and software engineers, will be necessary to drive successful outcomes. Your role will require a minimum of 6 years of experience in data engineering and big data processing. Proficiency in AWS cloud services like AWS Glue, AWS Lambda, AWS Redshift, AWS EMR, and S3 is paramount. Strong skills in Python for data engineering tasks, hands-on experience with Apache Spark and SQL, as well as knowledge of data modeling, schema design, and performance tuning are essential.

    Understanding AWS Lake Formation and Lakehouse principles, experience with version control using Git, and familiarity with CI/CD pipelines are also required. Knowledge of data security, compliance, and governance best practices is crucial. Experience with real-time streaming technologies such as Kafka and Kinesis will be an added advantage. Strong problem-solving, analytical, and communication skills are key attributes for success in this role.,

Skills

PythonVersion ControlJsonAws CloudCloudSqlAWS S3Data ModelingData GovernanceCICDPythonAWS LambdaComplianceApache SparkGitAWS RedshiftSQLAWS EMRAWS GlueData Security

If a job posting appears fraudulent, asks for payment, contains misleading information, or violates our guidelines, please report it immediately. Our team will review it promptly, Jobaaj does not charge any fee from the applicants.

Similar Jobs

View all
Loading...

Looking for something similar?

Bag Logo
Jobaaj
Get jobs like this by email

New Data Science Analytics & Machine Learning roles in Tamil Nadu.

Job Alert
Google hiring for Specific Roles Apply Now!
1 min ago
New Opportunity
Amazon is hiring freshers Apply Now!
5 min ago
Featured Jobs
Microsoft opening 50+ positions Apply Now!
10 min ago

One email a day. Unsubscribe any time.