Lead Data Engineer

Our Purpose
Mastercard powers economies and empowers people in 200+ countries and territories worldwide. Together with our customers, we're helping build a sustainable economy where everyone can prosper. We support a wide range of digital payments choices, making transactions secure, simple, smart and accessible. Our technology and innovation, partnerships and networks combine to deliver a unique set of products and services that help people, businesses and governments realize their greatest potential.
Title and Summary
Lead Data Engineer
Job Description Summary
Who is Mastercard?
Mastercard is a global technology company in the payments industry. Our mission is to connect
and power an inclusive, digital economy that benefits everyone, everywhere by making
transactions safe, simple, smart, and accessible. Using secure data and networks,
partnerships and passion, our innovations and solutions help individuals, financial institutions,
governments, and businesses realize their greatest potential.
Our decency quotient, or DQ, drives our culture and everything we do inside and outside of our
company. With connections across more than 210 countries and territories, we are building a
sustainable world that unlocks priceless possibilities for all.
Job Overview:
As a Databricks Engineer, you will play a key role in building and operating high-quality data
platforms that support analytics, machine learning, and critical business decision-making. This
role requires strong end-to-end ownership of data pipelines, deep hands-on expertise with
Databricks, and close collaboration with engineering, data science, and product teams to
deliver reliable, scalable, and production-ready data solutions.
Role:
Data Engineering & Pipeline Ownership• Own the end-to-end lifecycle of data pipelines, including data ingestion, data cleansing,
transformation, and ML inference.• Design, build, and operate robust, scalable, and performant data pipelines using
Databricks.• Ensure data solutions meet high standards for data quality, reliability, scalability, and
operational excellence.
Databricks Platform Ownership• Take ownership of the team's Databricks workspace, including:
o Workspace configuration and optimization
o Cluster and job management
o Security, access controls, and governance
o Defining and promoting best practices and standards• Drive continuous improvement in Databricks usage, performance, and cost efficiency.
Software Development Lifecycle• Contribute across the full development lifecycle, including:
o Requirements analysis and solution design
o Implementation and test automation
o Deployment and production readiness
o Ongoing maintenance and support• Apply engineering best practices such as clean code, code reviews, CI/CD
integration, and documentation.
Collaboration & Delivery• Partner closely with software engineers, data scientists, and product teams to
deliver reliable, high-quality data solutions.• Translate analytical and business requirements into well-designed, production-grade
data pipelines.• Support troubleshooting, incident resolution, and continuous improvement in
production environments
All About you • Bachelor's degree in information technology, Computer Science or equivalent
education. • Hands-on experience with Apache Spark or Databricks, preferably using Python or Java.• Proven experience building large-scale data or ETL pipelines that handle high-volume
datasets.• Experience working in cloud-native environments, ideally on AWS.• Strong understanding of software craftsmanship, including: • Writing clean, maintainable, and well-tested code• Designing effective testing strategies• Working with CI/CD pipelines and modern source control practices• A strong mindset focused on reliability, scalability, and operational excellence, with
ownership for production systems.• Ability to collaborate effectively within cross-functional, Agile teams.• 6+ years of experience in the software engineering field. • Must have knowledge of AI Agents and hands on using AI Agents.• Strong problem-solving skills and the ability to work independently on complex
problems.• Experience working in Agile / Scrum environments.• Must be high-energy, detail-oriented, proactive, and able to function under pressure to
meet tight deadlines. • Strong communication skills -- both verbal and written - and able to quickly learn and
implement new technologies, application appropriate frameworks and tools. • Strong relationship, collaborative skills and organizational skills with a high degree of
initiative and self-motivation and Able to work as a member of matrix based diverse and
geographically distributed project team. • Willingness and ability to learn and take on challenging opportunities.• Knowledge of payments domain and Indian payment eco system is desirable.
Nice to Have• Working knowledge of Scala, especially in the context of Spark-based workloads. • Hands-on experience managing pipeline development, deployment stages, and usage
reporting within Databricks. • Experience handling regulated or sensitive data, with an understanding of data security,
privacy, and compliance requirements. • Familiarity with SQL and NoSQL databases and messaging or streaming systems, such
as Redis, ElastiCache, DynamoDB, Amazon S3, Kinesis, or Kafka. • Experience with monitoring, observability, and alerting tools, and supporting
high-traffic, customer-facing platforms, using tools such as Grafana or Prometheus. • Experience writing automated acceptance and integration tests that are fully integrated
into CI/CD pipelines. • Exposure to cloud-native tooling and infrastructure, including AWS, Docker,
Kubernetes, and Infrastructure-as-Code tools such as Terraform.
Corporate Security Responsibility
All activities involving access to Mastercard assets, information, and networks comes with an inherent risk to the organization and, therefore, it is expected that every person working for, or on behalf of, Mastercard is responsible for information security and must:

• Abide by Mastercard's security policies and practices;
• Ensure the confidentiality and integrity of the information being accessed;
• Report any suspected information security violation or breach, and
• Complete all periodic mandatory security trainings in accordance with Mastercard's guidelines.