@Marc Ellis is #HIRINGRole: Databricks Data Engineer
Location: Dubai
Duration: 3 monthsKEY ACCOUNTABILITIES: * Strong background in data engineering, cloud technologies, and specifically Databricks.

• Design, develop, test, deploy, and maintain optimized ETL pipelines using Databricks technologies.
• Experience using Python, Scala, SQL, or another programming language for data manipulation and automation.
• Optimize and troubleshoot data processing performance to ensure efficiency and reliability.
• Write and maintain clear, concise documentation for data processing and systems.
• Handle raw data ingestion through delta live tables, ensuring data quality across the data pipeline.
• Manage enterprise-level complex data and metadata changes in the Databricks environment using delta live tables.
• Collaborate with stakeholders, data architects, and management to provide Databricks solutions that align with project engagement.
• Develop and deploy ETL jobs and workflows with an emphasis on automation and scalability.
• Knowledge of stream processing, API-based extraction, etc.
• Collaborate with cross-functional teams, including software engineers, architects, and project managers, to deliver high-quality, scalable, and secure solutions.
• Participate in and lead code reviews to ensure the quality and integrity of project deliverables.
• Excellent English communication skills to articulate technical concepts to diverse audiences.

REQUIREMENTS:

• Bachelors degree in Computer Science, Engineering, or a related field.
• Minimum of 5 years experience in data engineering with a focus on Databricks and Azure.
• Proficiency in Python, Scala, SQL, or another programming language.
• Strong understanding of ETL processes and data pipeline development.
• Hands-on experience with Databricks, Azure Data Factory, and other Azure data services.
• Familiarity with delta live tables and managing data quality.
• Experience with stream processing and API-based data extraction.
• Proven ability to optimize and troubleshoot data processing performance.
• Excellent documentation and communication skills.
• Ability to work effectively in a team-oriented, collaborative environment.
• Strong problem-solving skills and attention to detail.
• Experience with code reviews and maintaining high-quality project deliverables.

Marc Ellis