Seeking a Senior Data Engineer to design and maintain data pipelines for the Risk & Compliance domain using PySpark and Python, ensuring data integrity and security while collaborating with cross-functional teams.
We are seeking a highly skilled Senior Data Engineer to design, build, and maintain scalable data pipelines for enterprise-grade data platforms within the Risk & Compliance domain. The ideal candidate will have strong expertise in PySpark, Python, and data engineering best practices, with a focus on data quality, governance, and security.
Key Responsibilities- Design, develop, and optimize scalable data pipelines using PySpark and Python
- Build robust ETL/ELT workflows to process large volumes of structured and unstructured data
- Collaborate with data scientists, analysts, and business stakeholders to deliver high-quality datasets
- Ensure data integrity, accuracy, and reliability through validation frameworks and monitoring
- Implement data security and access control mechanisms aligned with compliance standards
- Work closely with Risk & Compliance teams to support regulatory and reporting requirements
- Optimize performance of data processing jobs and queries
- Maintain and enhance existing data architecture and pipelines
- 6+ years of experience in Data Engineering
- Strong hands-on experience with PySpark and Python
- Solid experience with SQL and Oracle databases
- Experience in building and maintaining large-scale data pipelines
- Good understanding of data warehousing concepts and ETL frameworks
- Experience with data validation, data quality, and governance frameworks
- Familiarity with cloud platforms (AWS/Azure/GCP) is a plus
- Exposure to banking, financial services, or risk & compliance domain is preferred
- Strong problem-solving and analytical skills
- Ability to work in a fast-paced, collaborative environment
- Excellent communication and stakeholder management skills
- Attention to detail with a focus on data quality and security
- Experience with Big Data ecosystems (Hadoop, Spark)
- Knowledge of data security and regulatory compliance frameworks
- Prior experience working with enterprise data platforms
Global Software Solutions Group Chennai, Tamil Nadu, IND Office
Third Cross Road,, SIPCOT IT Park, Siruseri, Chennai, Tamilnadu, India, 603103
Similar Jobs
Big Data • Fintech • Information Technology • Business Intelligence • Financial Services • Cybersecurity • Big Data Analytics
Manage software and platform releases across the UK, ensuring operational integrity and stakeholder alignment. Oversee deployment through CI/CD pipelines, communication with teams, incident management, risk compliance, and continuous improvement processes.
Top Skills:
CheckmarxoneCloudflareDataprocGCPGkeGrafanaHarnessHashicorp VaultHelmKafkaKeycloakKongKubernetesOpentelemetryPingPostgresPrometheusRedisTerraformWiz
Big Data • Fintech • Information Technology • Business Intelligence • Financial Services • Cybersecurity • Big Data Analytics
Manage software and platform releases, ensuring operational integrity and compliance. Lead CI/CD processes, improve release management, and facilitate cross-functional coordination for UK platforms.
Top Skills:
CheckmarxoneCloudflareDataprocGCPGkeGrafanaHashicorp VaultHelmKafkaKeycloakKong ApimKubernetesOpentelemetryPingPostgresPrometheusRedisTerraformWiz
Cloud • Fintech • Food • Information Technology • Software • Hospitality
The Senior HRIS Analyst will manage Workday configurations, improve HR processes, and liaise with teams for employee experience enhancements. Responsible for system efficiency and compliance, particularly in the Benefits module.
Top Skills:
Benefits ModuleWorkday Hcm
What you need to know about the Chennai Tech Scene
To locals, it's no secret that South India is leading the charge in big data infrastructure. While the environmental impact of data centers has long been a concern, emerging hubs like Chennai are favored by companies seeking ready access to renewable energy resources, which provide more sustainable and cost-effective solutions. As a result, Chennai, along with neighboring Bengaluru and Hyderabad, is poised for significant growth, with a projected 65 percent increase in data center capacity over the next decade.
