Design and develop scalable data pipelines using Microsoft Fabric and Azure ecosystem. Collaborate with BI developers, ensure data quality, and optimize data workflows.
Note: Only CV which are as per JD will be contacted.
Job Title: Data Engineer / Senior Data Engineer
Location : UAE - Onsite
Role Purpose
Job Title: Data Engineer / Senior Data Engineer
Location : UAE - Onsite
Role Purpose
Design, develop, and maintain scalable data pipelines and modern data platforms using Microsoft Fabric and Azure ecosystem. The role involves building efficient data solutions that support analytics, reporting, and advanced data use cases across the organization.
Key Responsibilities- Design, build, and maintain robust ETL/ELT pipelines using Azure Data Factory (ADF) and Microsoft Fabric
- Develop and optimize data workflows using PySpark and SQL
- Implement and manage Lakehouse architecture using Medallion (Bronze, Silver, Gold) layers
- Work with Azure Data Lake Storage (ADLS) and Delta Lake for efficient data storage and processing
- Integrate data from multiple sources ensuring data quality, consistency, and reliability
- Collaborate with BI developers and analysts to enable data models and reporting solutions
- Monitor, troubleshoot, and optimize data pipelines and performance issues
- Ensure data governance, security, and compliance best practices
- Lead the design and architecture of scalable data platforms and solutions
- Define data engineering standards, best practices, and frameworks
- Mentor junior data engineers and review code for quality and performance
- Optimize large-scale data processing and improve system efficiency
- Collaborate with stakeholders to translate business requirements into technical solutions
- Drive adoption of Microsoft Fabric and modern data engineering practices across teams
- Hands-on experience with Azure Data Factory (ADF) and ADLS
- Strong programming skills in PySpark and SQL
- Experience in building ETL pipelines and working with structured/unstructured data
- Basic understanding of Lakehouse and Medallion architecture
- Familiarity with Delta Lake concepts
- Good problem-solving and debugging skills
- Extensive experience in Azure data ecosystem and Microsoft Fabric
- Strong expertise in designing end-to-end data architectures
- Deep understanding of Lakehouse, Medallion architecture, and Delta Lake
- Proven experience in handling large-scale distributed data processing
- Strong performance tuning and optimization skills
- Experience leading projects and mentoring teams
- Experience in Azure Databricks
- Knowledge of data modeling and data warehousing concepts
- Exposure to data governance and data quality frameworks
- Experience working in Agile environments
- Strong communication and stakeholder management skills
- Analytical thinking and problem-solving mindset
- Ability to work independently and in a collaborative team environment
Top Skills
Azure Data Factory
Azure Data Lake Storage
Delta Lake
Lakehouse
Medallion Architecture
Microsoft Fabric
Pyspark
SQL
Similar Jobs
Big Data • Information Technology
The Senior Data Engineer will design and implement scalable data solutions, manage data pipelines, and collaborate with data science to integrate ML workflows.
Top Skills:
AirflowApache IcebergAWSAzureCi/CdCloudFormationData LakeDatabricksDelta LakeFastapiGCPKafkaKubernetesLakehouseMlflowNode.jsNoSQLPandasPolarsPostgresPulumiPythonReactScalaSparkTerraform
Software
Design, develop and maintain data pipelines, optimize data architectures, and collaborate with data scientists while ensuring data quality and governance.
Top Skills:
SparkApache SupersetAzureAzure Data FactoryAzure DevopsAzure SynapseC#Ci/CdCosmos DbDatabricksDelta LakeDockerEvent HubsGithub ActionsJavaKafkaKubernetesMongoDBMySQLPostgresPower BIPythonPyTorchScalaService BusSQLSQL ServerTableauTensorFlow
Information Technology • Consulting
The Senior Data Platform Engineer will architect and optimize Hexnode's data infrastructure for high-volume security telemetry and analytics, ensuring scalability, efficiency, and data security.
Top Skills:
Analytics PlatformsData EngineeringData Lake ArchitecturesDistributed SystemsLarge-Scale Distributed SearchReal-Time Processing
What you need to know about the Chennai Tech Scene
To locals, it's no secret that South India is leading the charge in big data infrastructure. While the environmental impact of data centers has long been a concern, emerging hubs like Chennai are favored by companies seeking ready access to renewable energy resources, which provide more sustainable and cost-effective solutions. As a result, Chennai, along with neighboring Bengaluru and Hyderabad, is poised for significant growth, with a projected 65 percent increase in data center capacity over the next decade.
