Senior Big Data Engineer

dunnhumby is the global leader in Customer Data Science, empowering businesses everywhere to compete and thrive in the modern data-driven economy. We always put the Customer First.  

Our mission: to enable businesses to grow and reimagine themselves by becoming advocates and champions for their Customers. With deep heritage and expertise in retail – one of the world’s most competitive markets, with a deluge of multi-dimensional data – dunnhumby today enables businesses all over the world, across industries, to be Customer First. 

dunnhumby employs nearly 2,500 experts in offices throughout Europe, Asia, Africa, and the Americas working for transformative, iconic brands such as Tesco, Coca-Cola, Meijer, Procter & Gamble and Metro.  

We’re looking for a Sr Big Data Engineer who expects more from their career. It’s chance to extend and improve dunnhumby’s Data Engineering Team.It’s an opportunity to work with a market-leading business to explore new opportunities for us and influence global retailers.    

Key Responsibilities 

• Design end-to-end data solutions, including data lakes, data warehouses, ETL/ELT pipelines, APIs, and analytics platforms. 

• Architect scalable and low-latency data pipelines using tools like Apache Kafka, Flink, or Spark Streaming to handle high-velocity data streams. 

• Design /Orchestrate end-to-end automation using orchestration frameworks such as Apache Airflow to manage complex workflows and dependencies. 

• Design intelligent systems that can detect anomalies, trigger alerts, and automatically reroute or restart processes to maintain data integrity and availability. 

• Develop scalable data architecture strategies that support advanced analytics, machine learning, and real-time data processing. 

• Define and implement data governance, metadata management, and data quality standards. 

• Lead architectural reviews and technical design sessions to guide solution development. 

• Partner with business and IT teams to translate business needs into data architecture requirements. 

• Explore appropriate tools, platforms, and technologies aligned with organizational standards. 

• Ensure security, compliance, and regulatory requirements are addressed in all data solutions. 

• Evaluate and recommend improvements to existing data architecture and processes. 
• Provide mentorship and guidance to data engineers and technical teams. 

Technical Expertise

• Bachelor's or master's degree in computer science, Information Systems, Data Science, or related field. 

• 7+ years of experience in data architecture, data engineering, or a related field. 

• Proficient in data pipeline tools such as Apache Spark, Kafka, Airflow, or similar. 

• Experience with data governance frameworks and tools (e.g., Collibra, Alation, OpenMetadata). 

• Strong knowledge of cloud platforms (Azure or Google Cloud), especially with cloud-native data services. 

• Strong understanding of API design and data security best practices. 

• Familiarity with data mesh, data fabric, or other emerging architectural patterns. 

• Experience working in Agile or DevOps environments. 

• Experience with modern data stack tools (e.g., dbt, Snowflake, Databricks). 

• Extensive experience with high level programming languages - Python, Java & Scala   

• Experience with Hive, Oozie, Airflow, HBase, MapReduce, Spark along with working knowledge of Hadoop/Spark Toolsets. 

• Extensive Experience working with Git and Process Automation  
• In depth understanding of relational database management systems (RDBMS) and Data Flow Development. 

Soft Skills 

• Problem-Solving: Strong analytical skills to troubleshoot and resolve complex data pipeline issues. 

• Communication: Ability to articulate technical concepts to non-technical stakeholders and document processes clearly. 

• Collaboration: Experience working in cross-functional teams and managing stakeholder expectations. 

• Adaptability: Willingness to learn new tools and technologies to stay ahead in the rapidly evolving data landscape. 

What you can expect from us  

We won’t just meet your expectations. We’ll defy them. So you’ll enjoy the comprehensive rewards package you’d expect from a leading technology company. But also, a degree of personal flexibility you might not expect.  Plus, thoughtful perks, like flexible working hours and your birthday off.  You’ll also benefit from an investment in cutting-edge technology that reflects our global ambition. But with a nimble, small-business feel that gives you the freedom to play, experiment and learn.  

And we don’t just talk about diversity and inclusion. We live it every day – with thriving networks including dh Gender Equality Network, dh Proud, dh Family, dh One and dh Thrive as the living proof. Everyone’s invited.  

Our approach to Flexible Working  

At dunnhumby, we value and respect difference and are committed to building an inclusive culture by creating an environment where you can balance a successful career with your commitments and interests outside of work.  

We believe that you will do your best at work if you have a work / life balance. Some roles lend themselves to flexible options more than others, so if this is important to you please raise this with your recruiter, as we are open to discussing agile working opportunities during the hiring process.