Seeking a Senior Data Engineer to design and maintain data pipelines for the Risk & Compliance domain using PySpark and Python, ensuring data integrity and security while collaborating with cross-functional teams.
We are seeking a highly skilled Senior Data Engineer to design, build, and maintain scalable data pipelines for enterprise-grade data platforms within the Risk & Compliance domain. The ideal candidate will have strong expertise in PySpark, Python, and data engineering best practices, with a focus on data quality, governance, and security.
Key Responsibilities- Design, develop, and optimize scalable data pipelines using PySpark and Python
- Build robust ETL/ELT workflows to process large volumes of structured and unstructured data
- Collaborate with data scientists, analysts, and business stakeholders to deliver high-quality datasets
- Ensure data integrity, accuracy, and reliability through validation frameworks and monitoring
- Implement data security and access control mechanisms aligned with compliance standards
- Work closely with Risk & Compliance teams to support regulatory and reporting requirements
- Optimize performance of data processing jobs and queries
- Maintain and enhance existing data architecture and pipelines
- 6+ years of experience in Data Engineering
- Strong hands-on experience with PySpark and Python
- Solid experience with SQL and Oracle databases
- Experience in building and maintaining large-scale data pipelines
- Good understanding of data warehousing concepts and ETL frameworks
- Experience with data validation, data quality, and governance frameworks
- Familiarity with cloud platforms (AWS/Azure/GCP) is a plus
- Exposure to banking, financial services, or risk & compliance domain is preferred
- Strong problem-solving and analytical skills
- Ability to work in a fast-paced, collaborative environment
- Excellent communication and stakeholder management skills
- Attention to detail with a focus on data quality and security
- Experience with Big Data ecosystems (Hadoop, Spark)
- Knowledge of data security and regulatory compliance frameworks
- Prior experience working with enterprise data platforms
Top Skills
AWS
Azure
GCP
Hadoop
Oracle
Pyspark
Python
Spark
SQL
Global Software Solutions Group Chennai, Tamil Nadu, IND Office
Third Cross Road,, SIPCOT IT Park, Siruseri, Chennai, Tamilnadu, India, 603103
Similar Jobs
Cloud • Information Technology • Internet of Things • Machine Learning • Software • Cybersecurity • Infrastructure as a Service (IaaS)
Join our software development team to design, develop and maintain applications focusing on modern technologies, including microservices, testing, and DevOps practices.
Top Skills:
CassandraDocker SwarmElasticsearchGitlab CiJpaJunitMavenMySQLPostmanRabbitMQSpring BootSpring CloudSpring DataWildfly
Cloud • Information Technology • Internet of Things • Machine Learning • Software • Cybersecurity • Infrastructure as a Service (IaaS)
The Senior Software Engineer will design, test, and automate products while ensuring test coverage, document progress, and participate in Agile methodologies.
Top Skills:
CkaCkadCloud Native ArchitecturesDockerGerritGitGoHelmJavaJenkinsKubernetesLinuxMavenPython
Big Data • Fintech • Information Technology • Business Intelligence • Financial Services • Cybersecurity • Big Data Analytics
Lead the Visualization Team to enhance executive storytelling through standardized formats, improve operational performance, and develop training for independent work.
Top Skills:
Power BI
What you need to know about the Chennai Tech Scene
To locals, it's no secret that South India is leading the charge in big data infrastructure. While the environmental impact of data centers has long been a concern, emerging hubs like Chennai are favored by companies seeking ready access to renewable energy resources, which provide more sustainable and cost-effective solutions. As a result, Chennai, along with neighboring Bengaluru and Hyderabad, is poised for significant growth, with a projected 65 percent increase in data center capacity over the next decade.
