Designs, develops, and optimizes scalable data pipelines, ETL/ELT processes, integrations, and cloud data platforms. Ensures data quality, governance, reliability, monitoring, and operational efficiency across batch and real-time workflows. Supports Meridian platform implementation, troubleshooting, and deployment while collaborating with architects, analysts, product owners, and stakeholders. Provides technical guidance, participates in architecture and code reviews, mentors engineers, and promotes automation, documentation, and engineering best practices.
Position Summary
The Senior Data Engineer is responsible for designing, building, and optimizing scalable data pipelines, data integration solutions, and cloud-based data platforms that support enterprise analytics, reporting, and AI/ML initiatives.
This role serves as a technical expert in data engineering, ensuring reliable data ingestion, high-quality data assets, and efficient data processing while collaborating closely with architects, analysts, and business stakeholders.
Key Responsibilities
Data Engineering & Development
- Design, develop, and maintain scalable and high-performance data pipelines.
- Build and optimize ETL/ELT processes to ingest, transform, and deliver data across enterprise platforms.
- Develop and maintain reusable data engineering frameworks and components.
- Implement data integration solutions for structured and unstructured data sources.
- Support modern cloud-based data platform initiatives.
- Data Ingestion & Pipeline Operations Develop and manage batch and real-time data ingestion pipelines.
- Monitor and optimize pipeline performance, reliability, and scalability.
- Troubleshoot data processing issues and implement corrective actions.
- Automate data workflows and orchestration processes.
- Ensure timely and accurate data delivery to downstream consumers.
Data Quality & Governance
- Implement data validation, reconciliation, and quality controls.
- Support data governance standards, metadata management, and data lineage initiatives.
- Identify and resolve data quality issues and causes.
- Contribute to data observability and monitoring solutions.
- Ensure compliance with organizational data management standards.
- Meridian Platform Support & Implementation Support the implementation and enhancement of Meridian data solutions.
- Develop integrations and data pipelines supporting Meridian initiatives.
- Collaborate with stakeholders to translate business requirements into technical solutions.
- Ensure successful deployment, testing, and operational support of Meridian-related capabilities.
Technical Leadership & Collaboration
- Provide technical guidance and mentorship to junior and mid-level engineers.
- Participate in architecture reviews, design sessions, and code reviews.
- Collaborate with data architects, product owners, analysts, and business stakeholders.
- Promote engineering best practices, automation, and continuous improvement.
- Document technical designs, standards, and operational procedures.
Required Qualifications
- Bachelor's degree in Computer Science, Engineering, Information Systems, or a related field.
- 5+ years of experience in Data Engineering or related disciplines.
- Strong experience designing and developing enterprise data pipelines and integration solutions.
- Expertise in SQL and one or more programming languages such as Python, Scala, or Java.
- Experience with ETL/ELT frameworks, data warehousing, and data lake architectures.
- Experience with cloud platforms such as Azure, AWS, or Google Cloud. Knowledge of workflow orchestration tools such as Airflow, Azure Data Factory (ADF), or similar platforms.
- Strong understanding of data quality, data governance, and monitoring concepts.
- Experience troubleshooting and optimizing large-scale data processing solutions.
Requirements
- Experience with Meridian platform implementation or support.
- Experience with Data bricks, Snowflake, Azure Synapse, Microsoft Fabric, or similar modern data platforms.
- Knowledge of streaming technologies such as Kafka, Spark Streaming, or Event Hubs.
- Familiarity with DevOps practices, CI/CD pipelines, Infrastructure as Code (IaC), and automated testing.
- Experience supporting AI/ML data pipelines and feature engineering.
- Relevant cloud, data engineering, or platform certifications.
- Success Measures Reliable and scalable data pipelines with minimal operational issues.
- Improved data quality and reduced production defects. Timely delivery of data engineering solutions and enhancements.
- Successful implementation and support of Meridian data initiatives.
- Increased automation, operational efficiency, and platform stability.
- High-quality technical documentation and adherence to engineering standards.
Similar Jobs
Automotive
Develop and support Oracle HCM Cloud technical solutions for HR and Payroll operations. Responsibilities include building integrations, HCM Extracts, BI Publisher and OTBI reports, HDL/HSDL loads, Fast Formulas, data conversions, troubleshooting, testing, deployments, and production support. The role translates business requirements into technical solutions, supports Oracle Cloud updates, prepares documentation, and collaborates with functional teams, stakeholders, and vendors.
Top Skills:
Bi PublisherFast FormulasHcm ExtractsHdlHsdlOracle Hcm CloudOracle Integration CloudOtbiRest ApisSoap ApisSQL
Automotive
Designs, develops, and supports scalable SAP solutions using ABAP, RAP, CDS, OData, and S/4HANA extensibility. Responsibilities include API-led integrations, Fiori/UI5 backend services, BTP extensions, testing, debugging, performance optimization, code reviews, architecture discussions, and Agile delivery. The role requires maintaining custom SAP developments across S/4HANA and ECC while following clean core principles, enterprise security standards, and SAP best practices.
Top Skills:
Abap Development Tools (Adt)Api ManagementCloud Application Programming Model (Cap)Core Data Services (Cds)EclipseGitIdocsObject-Oriented AbapOdata V2Odata V4Rest ApisRestful Abap Programming Model (Rap)RfcsSap AbapSap Btp Abap EnvironmentSap Business Technology Platform (Btp)Sap EccSap FioriSap GatewaySap Integration SuiteSap S/4HanaSoapUi5Web Services
Automotive
Design, implement, maintain, and optimize Ford’s on-premises and GCP-based Dassault Systemes 3DEXPERIENCE infrastructure. Automate provisioning, configuration, and deployments with Terraform and Ansible; manage cloud resources, servers, networking, databases, and load balancing; troubleshoot Linux-based applications and platform incidents; implement security controls; optimize performance; document infrastructure; and collaborate with application, database, and security engineering teams.
Top Skills:
AnsibleApache Http ServerApache TomcatBashCi/CdCloud Load BalancingCloud SqlCloud StorageCompute EngineContainersGoogle Cloud Platform (Gcp)HaproxyInfrastructure As Code (Iac)Java Virtual Machine (Jvm)LinuxPythonServerless ComputingTerraformVpcWindows
What you need to know about the Chennai Tech Scene
To locals, it's no secret that South India is leading the charge in big data infrastructure. While the environmental impact of data centers has long been a concern, emerging hubs like Chennai are favored by companies seeking ready access to renewable energy resources, which provide more sustainable and cost-effective solutions. As a result, Chennai, along with neighboring Bengaluru and Hyderabad, is poised for significant growth, with a projected 65 percent increase in data center capacity over the next decade.
