Designs and maintains Python and SQL-based automation for ETL and data pipeline validation. Tests source-to-target mappings, transformations, schemas, data quality, incremental loads, and business rules across GCP platforms including BigQuery, Dataflow, Pub/Sub, and Cloud Composer. Develops reusable, metadata-driven test components for high-volume structured and semi-structured data, integrates suites with CI/CD, investigates defects, and supports API validation using tools such as Pytest, Requests, Postman, or REST Assured.
Job Description: Quality Assurance Engineer III– ETL & Data Test Automation
Role Overview
Role Overview
We are looking for a hands-on Quality Engineering Engineer with strong expertise in ETL/Data Testing Automation and working knowledge of API Automation. The primary responsibility is to build scalable automated validation solutions for data pipelines, data transformations, and cloud-based data platforms.
Key Responsibilities- Design, develop, and maintain Python and SQL-based ETL/Data Testing Automation frameworks.
- Automate source-to-target validation across complex ETL/ELT data pipelines.
- Validate data across Bronze/Raw, Silver/Curated, and Gold/Consumption layers.
- ETL & Data Validation: Validate source-to-target data, transformations, schema, record counts, nulls, duplicates, and business rules using BigQuery and SQL.
- Automation: Develop Python/dbt-based automation for ETL validation, data-quality checks, regression testing, and automated test reporting.
- GCP: Validate end-to-end data pipelines across BigQuery, GCS, Dataflow, Pub/Sub, and Cloud Composer, including full and incremental data loads.
- Build reusable automation for:
- Source-to-target reconciliation
- Row-count and aggregate validation
- Schema and data-type validation
- Null, duplicate, and mandatory-field checks
- Data completeness and integrity checks
- Transformation and business-rule validation
- Incremental, delta, and historical load validation
- Data freshness and pipeline validation
- Develop solutions for validating high-volume datasets using Python, SQL, Pandas, and cloud data platforms.
- Perform complex SQL validation and troubleshoot data discrepancies across source and target systems.
- Validate structured and semi-structured data, including JSON and nested datasets.
- Develop reusable, configuration-driven or metadata-driven automation components to reduce manual testing.
- Integrate automated ETL/data-quality test suites with CI/CD pipelines such as Jenkins, GitHub Actions, or Azure DevOps.
- Perform defect investigation and root-cause analysis by collaborating with developers, data engineers, and business teams.
- Maintain automation code using Git/GitHub, including appropriate branching and pull-request practices.
- Support API validation as a secondary responsibility, including REST API request/response, status code, JSON payload, authentication, schema, and basic integration validation.
- Develop and maintain basic API automation using Python Requests/Pytest, Postman/Newman, or REST Assured when required.
- Contribute to test strategy, regression coverage, release readiness, and overall quality engineering practices.
Similar Jobs
Digital Media • Information Technology • News + Entertainment
Oversee network security safeguards, enforce secure hardware and software configurations, maintain disaster recovery plans, develop incident response policies, lead penetration testing and vulnerability assessments, conduct security audits, evaluate security technologies, and coordinate with network operations, development, and support teams. The role requires independent judgment and availability for nights, weekends, and variable schedules.
Top Skills:
Disaster Recovery SystemsNetwork Security ProtocolsPenetration TestingVulnerability Assessment
Digital Media • Information Technology • News + Entertainment
Oversee network security safeguards, enforce secure hardware and software configurations, maintain disaster recovery plans, develop incident response policies, lead penetration tests and vulnerability assessments, conduct security audits, investigate security technologies, and coordinate with development, support, and network operations teams. The role requires independent judgment and availability for nights, weekends, and variable schedules.
Top Skills:
Disaster Recovery SystemsNetwork Security ProtocolsNetwork Security SafeguardsPenetration TestingVulnerability Assessment
Digital Media • Information Technology • News + Entertainment
Oversee network security safeguards, enforce secure hardware and software configurations, maintain disaster recovery plans, develop incident response policies, evaluate security technologies, lead penetration tests and vulnerability assessments, and conduct internal and external security audits. The role also coordinates with network operations, development, and support teams and requires independent judgment, punctual attendance, and flexibility to work nights and weekends.
Top Skills:
Disaster Recovery SystemsNetwork Security ControlsNetwork Security ProtocolsPenetration TestingVulnerability Assessments
What you need to know about the Chennai Tech Scene
To locals, it's no secret that South India is leading the charge in big data infrastructure. While the environmental impact of data centers has long been a concern, emerging hubs like Chennai are favored by companies seeking ready access to renewable energy resources, which provide more sustainable and cost-effective solutions. As a result, Chennai, along with neighboring Bengaluru and Hyderabad, is poised for significant growth, with a projected 65 percent increase in data center capacity over the next decade.

