An AI agent that applies to roles like this on employers' own sites. 25 applications free, no card needed.
Job Description
Job description / Role
Responsibilities:
- Design and Build Scalable Data Pipelines: Architect, develop, and maintain efficient, scalable, and secure data pipelines to handle large datasets across multiple data sources, ensuring reliability and performance.
- Cloud Platform Expertise: Utilize AWS and GCP services (e.g., Amazon S3, Redshift, BigQuery, Dataflow, Cloud Storage, Dataproc) to implement and optimize cloud-based data infrastructure.
- Data Integration: Integrate various data sources, both structured and unstructured, from internal systems and third-party providers, to enable cross-functional teams to access actionable insights.
- Collaboration: Work closely with data scientists, analysts, and other stakeholders to understand data requirements and provide solutions that drive insights for business intelligence, reporting, and analytics.
- Data Quality & Governance: Implement best practices for data quality, data lineage, and governance to ensure the accuracy and compliance of data across pipelines and systems.
- Optimization and Automation: Continuously optimize data workflows and automate processes to improve efficiency and reduce latency in data operations.
- Performance Tuning: Optimize data storage, retrieval, and processing performance on cloud platforms to ensure optimal cost and time efficiency.
- Security & Compliance: Ensure data privacy and security standards are maintained in alignment with company policies and industry regulations.
Requirements:
- Experience: 5+ years of hands-on experience as a data engineer, with a focus on data engineering in cloud environments (AWS and GCP).
- Cloud Technologies: Deep expertise in using AWS (e.g., S3, Redshift, Lambda, Glue) and GCP (e.g., BigQuery, Dataflow, Cloud Storage) to build and manage data pipelines and infrastructure.
- Data Engineering Skills: Strong knowledge of ETL/ELT processes, data warehousing concepts, and distributed computing frameworks (e.g., Apache Spark, Hadoop, Airflow).
- Programming: Proficiency in Python, SQL, and other relevant programming languages for data engineering.
- Database Knowledge: Experience working with both relational and NoSQL databases (e.g., PostgreSQL, MySQL, DynamoDB, MongoDB).
- Version Control & CI/CD: Familiarity with version control systems (e.g., Git) and CI/CD pipelines for automating deployments and testing.
- Data Processing Frameworks: Experience with data processing and orchestration frameworks such as Apache Airflow, Apache Kafka, or similar technologies.
- Problem Solving: Strong analytical and troubleshooting skills with the ability to resolve complex data and system issues in a timely manner.
- Media Industry Knowledge (Preferred): Familiarity with data needs and challenges within the media industry (e.g., content analytics, user behavior analysis, and media streaming data).
Preferred Qualifications:
- Certifications: AWS Certified Solutions Architect, Google Professional Data Engineer, or similar certifications in cloud technologies.
- Big Data Technologies: Experience with big data tools, Spark, or similar distributed data processing technologies.
- Machine Learning (Optional): Exposure to machine learning platforms or working with data science teams to enable ML models is a plus.
Skills
Big DataPythonData IntegrationData PrivacyData ScienceData WarehousingData Warehousing ConceptsEtlData ProcessingMachine LearningMysqlData EngineerAnalyticsMlSqlIf a job posting appears fraudulent, asks for payment, contains misleading information, or violates our guidelines, please report it immediately. Our team will review it promptly, Jobaaj does not charge any fee from the applicants.
About Company
Similar Jobs
View allLooking for something similar?
New Data Science Analytics & Machine Learning roles in UAE.
One email a day. Unsubscribe any time.

