Job Description
About Company
Similar Jobs
Please verify your account first!
Send OTP
Please click on the Apply to verify the status of jobs posted more than 15 days ago, as they may have expired. Similar Jobs
Job Description
Responsibilities
Qualifications
- Design, develop, and deploy end-to-end automated ETL pipelines for efficient data extraction, transformation, and loading.
- Collaborate with data scientists and analysts to understand data requirements and design optimal data architectures.
- Create data transformation processes that cater to various data formats, including structured and unstructured data, such as JSON and XML.
- Build and maintain distributed systems for data storage and processing, leveraging technologies like Hadoop, Hive, Spark and RDBMS database.
- Develop data transformation and enrichment processes to ensure data quality and accuracy.
- Monitor and troubleshoot performance issues, identifying areas for optimisation and implementing necessary improvements.
- Implement security measures to protect sensitive data and ensure compliance with relevant regulations.
- Bachelors degree or higher in computer science/statistics/engineering.
- 3-4 years of experience in building and deploying large-scale data processing pipelines using distributed storage platforms like HDFS, S3, and NoSQL databases in a production environment.
- Proven experience as a Data Engineer or similar role and successful design and implementation of big data solutions.
- Strong programming skills in languages such as shell scripting, Python or Scala.
- Strong understanding of distributed computing concepts and big data technologies.
- Hands-on experience in distributed processing platforms like Hadoop,Hive Spark/PySpark and Spark-SQL
- Specialize in handling unstructured data formats such as XML and JSON, transforming them into structured formats for analysis.
- Implement scheduling mechanisms for running ETL jobs, ensuring data availability for reporting and analysis.
- Monitor pipeline performance, troubleshoot issues, and optimize workflows for efficiency.
- Strong SQL skills and familiarity with query optimization techniques.
- Experience with data modelling, database design, and performance optimization.
- Solid understanding of ETL processes and tools (e.g., Hadoop, Apache Spark, Air flow, PySpark and shell scripting).
- Proficiency in working with relational databases (e.g., MySQL, PostgreSQL) and NoSQL databases (e.g., MongoDB, Cassandra)
- Familiarity with cloud platforms such as AWS, Google Cloud, or Azure for data storage and processing.
- Knowledge of data warehousing concepts and tools (e.g., Amazon Redshift, Snowflake).
- Excellent problem-solving skills and attention to detail.
- Strong communication skills to collaborate effectively with team members.
Looking to get Placed? Try our Placement Guarantee Plan
Skills
Data ExtractionData ProcessingData QualityData WarehousingEtlImplementationMysqlPythonQualityReportingSqlTransformationXmlIf an employer asks you to pay any kind of fee, please notify us immediately. Jobaaj does not charge any fee from the applicants and we do not allow other companies also to do so.
About Company
ARK Invest is an investment management firm that builds thematic exchange-traded funds (ETFs) focused on disruptive innovation. We invest in companies that are revolutionizing industries, with a focus on the next generation of technology.
Important dates & deadlines?
Application Deadline
30 Sep 23, 04:55 PM IST
Similar Jobs
View All
Loading...
Don't Miss out any Updates
Subscribe now for the latest job alerts
and never miss an update
Job Alert
Google hiring for Specific Roles Apply Now!
1 min ago
New Opportunity
Amazon is hiring freshers Apply Now!
5 min ago
Featured Jobs
Microsoft opening 50+ positions Apply Now!
10 min ago
