Senior Data Engineer

About us:-

ArabyAds is a global technology company that empowers e-commerce marketing. It offers multiple digital advertising platforms across different touchpoints of a brand’s journey in a customer lifecycle ranging from customer acquisition, retention, and monetization.

Headquartered in Dubai, ArabyAds was founded in 2013 to leverage innovation and deliver measurable results to its advertisers with data-led, transparency-first & omnichannel marketing. Entrepreneur Middle East recognized ArabyAds as “The best e-commerce marketing platform 2021.”

We are a team of +400 passionate people across the Middle East, Europe, India & South East Asia and Africa. We are an agile, fast-growing, interdisciplinary team of passionate experts in media planning, influencer marketing, consumer research, developers & coders, data scientist, artificial intelligence, and machine learning with a shared mission to create sustainable value for our clients. ArabyAds is proud to be an equal opportunity employer regardless of your race, colour, gender, religion, age, disability, or marital status. We currently have people from 19 nationalities across our 13 locations and are happy to celebrate all festivals.

Responsibilities :

- Design, Develop, Maintain and own data engineering projects built from requirements by different stakeholders, including but not limited to product teams, data scientists, data analysts and BI.

- Fulfill any requirement related to data pipelines, data warehouses and databases required by the application.

- Create ingestion pipelines to collect data from different sources and push to targets.

- Design and Create SQL, tables, procedures, functions etc.

- Operate and Maintain existing Hadoop Cluster.

- Write Spark Tasks/Applications mainly in Java.

- Evaluate, test and update data applications according to changing requirements.

- Monitor and report data systems health and security.

- Support internal teams to use our systems, troubleshoot & identify issues, and resolve them.

- Follow best coding practices and participate in the code review process.

- Use efficiently the development environment, compiling tools, version control and bug tracking systems for all projects.

Technical Competencies :

Must Have :

- 3-10 years of experience in Data Engineering.

- Good Knowledge/Experience in Java, C# and .net core.

- Good Knowledge/Experience Hadoop Stack, Oozie, Spark, Kafka

- Hands-on in writing SQL queries, Stored Procedures and Database code

- Good Knowledge with concepts, design and mechanics of traditional databases, data warehouses and data lakes

- Good Knowledge with ETL, ELT, Transformation, Processing, Computation etc.

- Experience with relational databases such as MySQL, PostgreSQL, MariaDB.

- Good Knowledge/Experience with Git, CI/CD, and writing unit tests.

- Good Knowledge/Experience working on an agile development project.

- Good Knowledge/Experience with Jira and the Atlassian tool suite.

- Good Knowledge/Experience in Linux Shell Scripting

Nice to Have :

- Experience with NoSQL databases such as Aerospike, DynamoDB, Redis, Elasticsearch, Cassandra, MongoDB

- Good knowledge of Snowflake databases, schemas, tables, procedures, functions, views, stages, pipes etc.

- Knowledge on one of the ETL tools

- Experience with performance and scalability issues.

- Knowledge of Python is a plus

- Non Technical Competencies

- Good English and communication skills.

- Positive and can do attitude.

- Able to work independently as well as in a team.

- Able to take up challenges and come up with innovative solutions.

- We highly appreciate self-starters, initiators and contributors.