Job Description

Our Team


Our team comprises highly skilled professionals who bring a wealth of international experience in technology entrepreneurship. We thrive on ambition and recognize that fun and performance go hand in hand. Together, we create a dynamic environment where excellence is the currency of success.

Position Overview


As a Data Engineer, you will play a crucial role in designing, building, and maintaining our data infrastructure. You will be responsible for ensuring the availability, integrity, and efficiency of our data pipelines, enabling our organization to make informed, data-driven decisions. The value you bring will be instrumental in delivering measurable value to Contango and beyond using data and AI.


Responsibilities

• Leadership and ownership: own overall data lake and data warehouse architecture with the responsibilities to design, architect, and maintain the data warehouse infrastructure to support data management, data governance, analytics, ML & AI needs and address future challenges.
• Data pipeline development: Design, build, and maintain scalable data pipelines for ingesting, processing, and transforming data from various sources, ensuring data quality, scalability, and reliability.
• Data modelling: Develop and maintain data models, architecture patterns, schemas, and structures that support the needs of data analysts, data scientists, and other stakeholders.
• ETL (Extract, Transform, Load): Create and optimize ETL processes to extract data from diverse sources, transform it into usable formats, and load it into data warehouses or other storage solutions.
• Data governance, quality and validation: Establish and enforce data governance policies, standards, procedures, and best practices and implement data quality checks, validation processes, and error handling to ensure the accuracy and consistency of data.
• Performance tuning : Continuously monitor and optimize the performance of data pipelines to meet business requirements and scalability needs.
• Data security: Implement and maintain data security measures to protect sensitive information and ensure compliance with data privacy regulations.
• Collaboration: Work closely with data analysts, data scientists, and other stakeholders to understand their data requirements and provide support in data access and availability.
• Documentation: Maintain thorough documentation of data engineering processes, data models, and data dictionaries.
• Stay informed: Keep up to date with emerging trends and technologies in data engineering to ensure our data infrastructure remains cutting-edge.

Qualifications

• Bachelor's degree in computer science, information technology, or a related field (master's degree is a plus)
• Demonstrable experience of minimum 8 years with at least 5 years in a leadership position as a Senior Data Engineer or similar, with a portfolio of projects 10+ years of experience in data engineering with a focus on designing and implementing data lakes, data warehouses, data modelling, ETL processes, and data transformation pipelines
• Experience in cloud-based data platforms (e.g., Spark, Azure Synapse Analytics etc.)
• Proficiency in programming languages such as Python, Java, or Scala
• Strong knowledge of database systems (SQL and NoSQL), data warehousing, and big data technologies
• Knowledge of data governance principles, data quality management, and regulatory compliance
• Experience with orchestration services (i.e., Azure Data Factory (ADF), Azure Logic Apps, Azure Functions, Databricks Jobs, Airflow etc.)
• Familiarity with data engineering, integration tools and frameworks and ETL processes
• Experience with cloud platforms like AWS, GCP, or Azure
• Experience with API development and integration with data services
• Excellent communications, leadership, and collaboration skills
• Ability to work in a fast-paced, dynamic environment and lead multiple projects and teams to deliver high-quality results
• Technical Expertise:
+ Programming languages : Python, Java, Scala, and R.
+ Data management & databases : Oracle, SAP, SQL, NoSQL, Data Warehousing
+ Big data technologies : Apache Hadoop, Spark, Kafka, etc.
+ Cloud platforms: Experience with Microsoft Fabric, Azure (Synapse Analytics, Databricks, Machine Learning, AI Search, Functions, etc.), Databricks on Azure, AWS (S3, Redshift, SageMaker, etc.), Google Cloud Platform (BigQuery, Dataflow, etc.).
+ Data governance: Experience with Ab Initio, Informatica, Collibra, Purview, IBM InfoSphere, Great Expectations, Deepchecks, Databricks' Unity Catalog, Databricks' Delta Sharing, Databricks' Catalog Explorer, Databricks' Audit Logging, and Databricks' Identify Management.
+ Data visualization : Familiarity with Matplotlib, Seaborn, Tableau, Power BI
+ DevOps & MLOps : CI/CD principles, Docker, MLFlow, Kubernetes

Benefits

• Competitive salary and benefits package.
• Opportunity to work on cutting-edge technology projects.
• Collaborative and innovative work environment.
• Professional growth and training opportunities.


If you possess the requisite experience, qualifications, and leadership acumen to drive our success and shape the future of Data and AI innovation within the UAE, we invite you to submit your application.