Lead the migration of an Oracle-based data warehouse to Databricks on the Allianz Data Platform. Analyze existing Oracle, PostgreSQL, and SAS ETL structures; design target data models; build and optimize Spark SQL and PySpark ETL/ELT pipelines; translate PL/SQL logic; ensure data quality and performance; and document migration patterns and operating procedures while collaborating with architects, platform teams, and stakeholders.
Overall Objectives of job We are looking for a Senior Data Engineer with strong Databricks, SQL and data warehousing experience to lead the migration of an existing Oracle-based data warehouse to Databricks on the Allianz Data Platform (ADP). Experience with modern data engineering practices and AI-enabling data platforms is highly desirable. Duties and responsibilities • Analyse the current Oracle data warehouse (including PostgreSQL source feeds and SAS ETL–generated structures) and design the target model in Databricks. • Build and optimise ETL/ELT pipelines in Databricks (Spark SQL, PySpark) to migrate core data warehouse tables and datamarts from Oracle to Databricks. • Translate Oracle SQL/PL-SQL logic into Databricks Spark SQL and PySpark, ensuring functional equivalence and performance optimisation. • Ensure data quality, performance, and reliability of the new Databricks-based data warehouse, enabling downstream analytics and AI/ML use cases. • Collaborate closely with the Enterprise Architect, ADP platform team and business stakeholders to align with architecture, standards, and best practices. • Document migration patterns, data models, and operational procedures to support long-term maintainability and reusability. Qualification, Experience, Technical and Functional Skills Must Have: • Graduate or Postgraduate in Engineering or a Post graduate in non-engineering disciplines . • Experience of 5-9 years of experience in ITIL process/application support/Deployment activities.t. • Positive attitude, proactively provides solutions
Hexaware Technologies Chennai, Tamil Nadu, IND Office
MCN Nagar Ext, MCN Nagar Extension, Thoraipakkam, Chennai, Tamil Nadu, India, 600097
Similar Jobs
Information Technology • Consulting
Leads development of scalable big data applications and platforms using Snowflake, Spark, Python, Airflow, and Kubernetes. Responsibilities include data collection, storage, modeling, analysis, ETL, data warehousing, SQL programming, and data processing across numerous sources. The role also requires knowledge of CI/CD practices and tools including Git and Jenkins.
Top Skills:
Apache AirflowSparkCi/CdData WarehousingETLGitJenkinsKubernetesPythonSnowflakeSnowpipeSnowsqlSQL
Information Technology • Consulting
Lead migration of an Oracle-based data warehouse to Databricks on the Allianz Data Platform. Responsibilities include designing target data models, building and optimizing Spark SQL and PySpark ETL/ELT pipelines, translating Oracle SQL and PL/SQL logic, ensuring data quality and performance, enabling analytics and AI/ML use cases, and documenting migration patterns and operational procedures.
Top Skills:
Allianz Data PlatformData WarehousingDatabricksEltETLOraclePostgresPysparkSASSpark SqlSQL
Information Technology • Consulting
Lead development of big data solutions, focusing on text data processing, data pipelines, and cloud services, primarily in AWS and GCP.
Top Skills:
AWSDockerElasticsearchGCPHadoopHbaseKubernetesMongoDBPythonSpark
What you need to know about the Chennai Tech Scene
To locals, it's no secret that South India is leading the charge in big data infrastructure. While the environmental impact of data centers has long been a concern, emerging hubs like Chennai are favored by companies seeking ready access to renewable energy resources, which provide more sustainable and cost-effective solutions. As a result, Chennai, along with neighboring Bengaluru and Hyderabad, is poised for significant growth, with a projected 65 percent increase in data center capacity over the next decade.
