Seeking a Senior Data Engineer to design and maintain data pipelines for the Risk & Compliance domain using PySpark and Python, ensuring data integrity and security while collaborating with cross-functional teams.
We are seeking a highly skilled Senior Data Engineer to design, build, and maintain scalable data pipelines for enterprise-grade data platforms within the Risk & Compliance domain. The ideal candidate will have strong expertise in PySpark, Python, and data engineering best practices, with a focus on data quality, governance, and security.
Key Responsibilities- Design, develop, and optimize scalable data pipelines using PySpark and Python
- Build robust ETL/ELT workflows to process large volumes of structured and unstructured data
- Collaborate with data scientists, analysts, and business stakeholders to deliver high-quality datasets
- Ensure data integrity, accuracy, and reliability through validation frameworks and monitoring
- Implement data security and access control mechanisms aligned with compliance standards
- Work closely with Risk & Compliance teams to support regulatory and reporting requirements
- Optimize performance of data processing jobs and queries
- Maintain and enhance existing data architecture and pipelines
- 6+ years of experience in Data Engineering
- Strong hands-on experience with PySpark and Python
- Solid experience with SQL and Oracle databases
- Experience in building and maintaining large-scale data pipelines
- Good understanding of data warehousing concepts and ETL frameworks
- Experience with data validation, data quality, and governance frameworks
- Familiarity with cloud platforms (AWS/Azure/GCP) is a plus
- Exposure to banking, financial services, or risk & compliance domain is preferred
- Strong problem-solving and analytical skills
- Ability to work in a fast-paced, collaborative environment
- Excellent communication and stakeholder management skills
- Attention to detail with a focus on data quality and security
- Experience with Big Data ecosystems (Hadoop, Spark)
- Knowledge of data security and regulatory compliance frameworks
- Prior experience working with enterprise data platforms
Global Software Solutions Group Chennai, Tamil Nadu, IND Office
Third Cross Road,, SIPCOT IT Park, Siruseri, Chennai, Tamilnadu, India, 603103
Similar Jobs
Digital Media • Information Technology • News + Entertainment
Design, develop, and execute ETL test strategies and automated validation for data ingestion, transformation, and loading. Validate data quality, perform reconciliation and lineage checks, develop SQL/Python validation scripts, monitor ETL workflows, collaborate with engineering and product teams, participate in code reviews and defect triage, and support data migrations and platform upgrades.
Top Skills:
AirflowAPIsAutomation Test FrameworkCi/CdData LakesData WarehousesDatabricksDbtETLJavaPythonSnowflakeSQL
Digital Media • Information Technology • News + Entertainment
Design, develop, test and integrate software and web applications. Produce technical documentation, train operations staff, mentor junior engineers, collaborate with QA, present architectural choices, and support software releases while applying project management and independent judgment.
Digital Media • Information Technology • News + Entertainment
Design, build, and optimize scalable data pipelines and platforms (Snowflake, Databricks, Spark). Ensure data quality, lineage, and observability. Mentor junior engineers, lead cross-team initiatives, implement CI/CD, and improve internal data engineering tools and architectures.
Top Skills:
AirflowSparkAWSAws S3AzureCi/CdDatabricksDbtGCPIcebergPythonSnowflakeSQL
What you need to know about the Chennai Tech Scene
To locals, it's no secret that South India is leading the charge in big data infrastructure. While the environmental impact of data centers has long been a concern, emerging hubs like Chennai are favored by companies seeking ready access to renewable energy resources, which provide more sustainable and cost-effective solutions. As a result, Chennai, along with neighboring Bengaluru and Hyderabad, is poised for significant growth, with a projected 65 percent increase in data center capacity over the next decade.
