Job Description

Details:
------------


### About the Role:


As part of a cross-functional team of engineers, data scientists, and product owners you will be responsible for designing, implementing, optimizing, and evaluating our product (specific product language here) algorithms. If you love seeing your ideas blossom through their whole life cycle from concept to governing the interaction with millions of users - then Avrioc is the place to be for you!

### Responsibilities:

• Collaborate with several stakeholders to deeply understand the needs of data practitioners to deliver at scale.
• Data Engineers help to define, build and maintain Data Platform.
• Work on emerging technologies, building distributed applications.
• Development effort End-to-End for on-time delivery of high-quality solutions that conform to requirements, conform to the architectural vision, and comply with all applicable standards.
• Present technical solutions, capabilities, considerations, and features in business terms.
• Help to further develop critical initiatives, such as Data Discovery, Data Lineage and Data Quality.
• Build data systems, pipelines, analytical tools and programs.
• Conduct complex data analysis and report on results.

### Skills and Attributes:

• Strong technical expertise in data modeling, data ingestion, distributed processing, data warehousing, and ETL processes.
• Experience working with Apache Kafka and Spark Streaming.
• Exposure to connecting and querying different data sources, including NoSQL databases (e.g., MongoDB), MySQL, Elasticsearch, and Kafka topics.
• Proficiency in SQL, Python, PySpark, Spark SQL and a solid understanding of the Databricks platform.
• Ability to build large-scale batch and real-time data pipelines.
• Knowledge of best practices for optimizing Databricks cluster configurations (size, type, memory) based on data requirements and delivering optimized code.
• Experience in maintaining, monitoring, and handling enhancement requests for data pipelines.
• Familiarity with visualization tools such as Kibana and QlikView.
• Exposure to Docker, Kubernetes, and GitLab.
• Experience with JIRA processes and Confluence for documentation and project management.


Requirements:
-----------------


### Qualification & Requirements:

• 5+ Years of Experience.
• Degree in Computer Science, Data Science, Mathematics, IT, or a related field.
• 5 years of Experience as a data engineer or similar role.


Location: Masdar