Design, build, and maintain scalable batch and streaming data pipelines across hybrid cloud. Prepare and validate datasets for analytics and ML, implement data quality, lineage and governance, optimize processing jobs, troubleshoot production pipelines, and support deployment and monitoring in cloud environments while collaborating with data scientists and engineering teams.
Our Purpose
Mastercard powers economies and empowers people in 200+ countries and territories worldwide. Together with our customers, we're helping build a sustainable economy where everyone can prosper. We support a wide range of digital payments choices, making transactions secure, simple, smart and accessible. Our technology and innovation, partnerships and networks combine to deliver a unique set of products and services that help people, businesses and governments realize their greatest potential.
Title and Summary
Data Engineer II
Your Role
As a Data Engineer II, you will contribute to the design, development, and support of enterprise data products and platforms across private and public cloud environments. You will work closely with senior engineers, data scientists, and cross-functional teams to build reliable, scalable data pipelines and enable analytics and machine learning use cases.
Key Responsibilities• Contribute to the development and enhancement of enterprise data products and data platforms supporting analytics and data science use cases• Design, build, and maintain scalable data pipelines for batch, streaming, and API-based data ingestion across hybrid (private/public cloud) environments• Support machine learning and AI initiatives by preparing, transforming, and validating datasets used in predictive models• Collaborate with Data Scientists and senior engineers to ensure high-quality data inputs for algorithms and analytical models• Transform structured and unstructured data into usable formats (e.g., text processing, metadata tagging, basic feature engineering)• Assist in integrating new data sources and improving data availability for analytics and product use cases• Implement and monitor data quality checks, data lineage, and governance standards to ensure data reliability and compliance• Troubleshoot data issues in pipelines and production systems, partnering with upstream data providers and engineering teams to resolve root causes• Optimize data processing jobs and queries for performance and cost efficiency• Participate in the development and maintenance of data infrastructure supporting enterprise-scale analytics• Support deployment, testing, and monitoring of data pipelines in cloud environments• Work collaboratively in Agile teams to deliver incremental, production-ready solutions• Stay current with evolving data engineering tools, frameworks, and cloud technologies
Ideal Candidate Qualifications - Core Qualifications
• Bachelor's degree in Computer Science, Data Engineering, Information Systems, or a related field (or equivalent experience)• 2-5 years of hands-on experience in data engineering, data processing, or related roles• Experience building and maintaining data pipelines and ETL/ELT workflows in a distributed data environment• Proficiency in SQL and at least one programming language (Python, Scala, or Java)• Experience working with big data technologies such as Spark, Hadoop, or Kafka• Familiarity with data modeling concepts and data warehouse architecture• Experience debugging data issues and optimizing SQL queries and pipeline performance• Exposure to CI/CD practices for data pipelines• Understanding of data governance concepts such as data quality, lineage, and classification• Experience working in Agile development environments• Strong analytical and problem-solving skills with attention to detail• Good communication and collaboration skills
Preferred / Nice-to-Have Skills
• Experience with cloud platforms such as Azure, AWS, or GCP• Exposure to Databricks and cloud-native data processing frameworks• Familiarity with streaming and workflow orchestration tools (e.g., Kafka, Airflow, NiFi)• Experience supporting machine learning workflows or feature engineering pipelines• Experience with dashboarding/visualization tools (e.g., Power BI)• Prior experience in financial services or payments industry
Corporate Security Responsibility
All activities involving access to Mastercard assets, information, and networks comes with an inherent risk to the organization and, therefore, it is expected that every person working for, or on behalf of, Mastercard is responsible for information security and must:
Mastercard powers economies and empowers people in 200+ countries and territories worldwide. Together with our customers, we're helping build a sustainable economy where everyone can prosper. We support a wide range of digital payments choices, making transactions secure, simple, smart and accessible. Our technology and innovation, partnerships and networks combine to deliver a unique set of products and services that help people, businesses and governments realize their greatest potential.
Title and Summary
Data Engineer II
Your Role
As a Data Engineer II, you will contribute to the design, development, and support of enterprise data products and platforms across private and public cloud environments. You will work closely with senior engineers, data scientists, and cross-functional teams to build reliable, scalable data pipelines and enable analytics and machine learning use cases.
Key Responsibilities• Contribute to the development and enhancement of enterprise data products and data platforms supporting analytics and data science use cases• Design, build, and maintain scalable data pipelines for batch, streaming, and API-based data ingestion across hybrid (private/public cloud) environments• Support machine learning and AI initiatives by preparing, transforming, and validating datasets used in predictive models• Collaborate with Data Scientists and senior engineers to ensure high-quality data inputs for algorithms and analytical models• Transform structured and unstructured data into usable formats (e.g., text processing, metadata tagging, basic feature engineering)• Assist in integrating new data sources and improving data availability for analytics and product use cases• Implement and monitor data quality checks, data lineage, and governance standards to ensure data reliability and compliance• Troubleshoot data issues in pipelines and production systems, partnering with upstream data providers and engineering teams to resolve root causes• Optimize data processing jobs and queries for performance and cost efficiency• Participate in the development and maintenance of data infrastructure supporting enterprise-scale analytics• Support deployment, testing, and monitoring of data pipelines in cloud environments• Work collaboratively in Agile teams to deliver incremental, production-ready solutions• Stay current with evolving data engineering tools, frameworks, and cloud technologies
Ideal Candidate Qualifications - Core Qualifications
• Bachelor's degree in Computer Science, Data Engineering, Information Systems, or a related field (or equivalent experience)• 2-5 years of hands-on experience in data engineering, data processing, or related roles• Experience building and maintaining data pipelines and ETL/ELT workflows in a distributed data environment• Proficiency in SQL and at least one programming language (Python, Scala, or Java)• Experience working with big data technologies such as Spark, Hadoop, or Kafka• Familiarity with data modeling concepts and data warehouse architecture• Experience debugging data issues and optimizing SQL queries and pipeline performance• Exposure to CI/CD practices for data pipelines• Understanding of data governance concepts such as data quality, lineage, and classification• Experience working in Agile development environments• Strong analytical and problem-solving skills with attention to detail• Good communication and collaboration skills
Preferred / Nice-to-Have Skills
• Experience with cloud platforms such as Azure, AWS, or GCP• Exposure to Databricks and cloud-native data processing frameworks• Familiarity with streaming and workflow orchestration tools (e.g., Kafka, Airflow, NiFi)• Experience supporting machine learning workflows or feature engineering pipelines• Experience with dashboarding/visualization tools (e.g., Power BI)• Prior experience in financial services or payments industry
Corporate Security Responsibility
All activities involving access to Mastercard assets, information, and networks comes with an inherent risk to the organization and, therefore, it is expected that every person working for, or on behalf of, Mastercard is responsible for information security and must:
- Abide by Mastercard's security policies and practices;
- Ensure the confidentiality and integrity of the information being accessed;
- Report any suspected information security violation or breach, and
- Complete all periodic mandatory security trainings in accordance with Mastercard's guidelines.
Similar Jobs at Mastercard
Blockchain • Fintech • Payments • Consulting • Cryptocurrency • Cybersecurity • Quantum Computing
The role involves managing vendor contracts related to privacy and data protection, ensuring compliance with global regulations, and facilitating efficient business operations while collaborating with multiple stakeholders.
Top Skills:
CcpaGdprSaas Agreements
Blockchain • Fintech • Payments • Consulting • Cryptocurrency • Cybersecurity • Quantum Computing
The Senior Analyst provides operational and financial support for corporate meetings and events, focusing on budgeting, contract management, and event coordination.
Top Skills:
CventMicrosoft PowerpointMonday.Com
Blockchain • Fintech • Payments • Consulting • Cryptocurrency • Cybersecurity • Quantum Computing
Design, develop, test, and optimize scalable, secure software. Contribute to architecture, ensure high test coverage, perform performance tuning, enable observability, and troubleshoot distributed systems to maintain reliability and operability.
What you need to know about the Chennai Tech Scene
To locals, it's no secret that South India is leading the charge in big data infrastructure. While the environmental impact of data centers has long been a concern, emerging hubs like Chennai are favored by companies seeking ready access to renewable energy resources, which provide more sustainable and cost-effective solutions. As a result, Chennai, along with neighboring Bengaluru and Hyderabad, is poised for significant growth, with a projected 65 percent increase in data center capacity over the next decade.
