2COMS Consulting Pvt. Ltd.
Hiring || Senior ETL Testing/ Data Engineer - Remote
Be an Early Applicant
Designs, builds, tests, and optimizes scalable ETL and ELT pipelines in GCP. Develops data warehousing and transformation solutions using BigQuery, Cloud Dataflow, Cloud Composer, Snowflake, Oracle, and other databases. Writes Java, Python, and advanced SQL; establishes reconciliation, schema, regression, and data-driven testing. Integrates pipelines with CI/CD tools and collaborates in Agile teams using Jira and Confluence.
Senior ETL / Data Engineer
Experience: 5-10 Years
Location: Remote
SummaryWe are seeking an experienced Senior ETL and Data Engineer to architect, build, and optimize robust data pipelines within a cloud-native environment. This pivotal role is responsible for designing scalable data integration solutions that ensure high availability, accuracy, and performance across our data infrastructure. The ideal candidate will bridge the gap between complex data sources and actionable insights, leveraging advanced cloud technologies and automation frameworks to drive data reliability and efficiency for the organization.
Responsibilities
- Design and implement scalable ETL and ELT processes utilizing GCP services such as BigQuery, Cloud Dataflow, and Cloud Data Fusion.
- Manage and orchestrate data workflows using Cloud Composer (Apache Airflow) to ensure seamless data movement and processing.
- Develop and maintain data warehousing solutions across diverse platforms including Snowflake, Oracle, Teradata, and various SQL databases.
- Write efficient, high-performance code in Java and Python to handle complex data transformations and logic.
- Construct advanced SQL queries involving window functions, CTEs, and complex joins to extract and manipulate data.
- Establish and enforce rigorous testing protocols, including source-to-target reconciliation, schema validation, and regression testing.
- Integrate data pipelines into CI/CD workflows using Jenkins, Git, and GCP Cloud Build to streamline deployment and version control.
- Collaborate within an Agile environment using Jira and Confluence to track progress, manage defects, and document technical specifications.
- Utilize tools like Informatica PowerCenter and Talend for enterprise-level data integration tasks.
Requirements
1. Experience: 5 to 10 years of professional experience in data engineering, ETL testing, or related fields.
2. Cloud Expertise: Proven proficiency with Google Cloud Platform (GCP) services, specifically BigQuery, Cloud Storage, and Cloud Composer.
3. Database Skills: Strong command of relational and NoSQL databases, including PostgreSQL, MySQL, Cloud SQL, Cloud Spanner, Oracle, and Snowflake.
4. Programming: Advanced skills in Java (JDBC, Apache Beam SDK) and Python (Pandas, PyTest).
5. SQL Mastery: Expert-level knowledge of SQL, including complex joins, window functions, and PL/SQL.
6. Testing & QA: Hands-on experience with testing frameworks like TestNG, JUnit, and BDD Cucumber, along with data-driven testing methodologies.
7. DevOps & Tools: Familiarity with CI/CD pipelines, Git, GitHub, Maven, and defect tracking systems like Jira and HP ALM.
8. Location: Must be available for remote work
Similar Jobs
Artificial Intelligence • Cloud • Computer Vision • Hardware • Internet of Things • Software
Manage enterprise customer accounts as a trusted technical advisor, driving platform adoption, measurable business value, and technical ROI. Develop success plans, troubleshoot complex hardware, software, and API issues, manage escalations, conduct root-cause analysis, and lead technical reviews. Collaborate with Sales, Support, Product, and Engineering while communicating effectively with technical and executive stakeholders. Use AI tools to improve customer insights and outcomes, and contribute to knowledge sharing and team development.
Top Skills:
AIAPIsGainsightGongInternet Of Things (Iot)JIRAPaasPythonSaaSSalesforceTableauZendesk
Fintech • Professional Services • Consulting • Energy • Financial Services • Cybersecurity • Generative AI
Tests liquidity risk and regulatory reporting releases across PRA, EBA, HKMA, and US Federal Reserve requirements. Responsibilities include SQL-based data validation, test strategy and execution, defect management, UAT dashboards, requirements gathering, data-gap analysis, issue resolution, and stakeholder communication. The role also supports finance data quality initiatives, regulatory compliance, operating-model transitions, change implementation, and coordination across Finance, Risk, Business, and IT teams.
Top Skills:
ExcelMicrosoft PowerpointMicrosoft ProjectMicrosoft VisioMicrosoft WordQuality CentreSQL
Big Data • Food • Hardware • Machine Learning • Retail • Automation • Manufacturing
Leads commercial finance partnering for East India, Nepal, and Bangladesh. Responsibilities include budgeting, forecasting, performance management, revenue and pricing analysis, trade investment and profitability management, financial governance, reporting, team leadership, stakeholder influence, and finance process transformation through standardization and automation.
What you need to know about the Chennai Tech Scene
To locals, it's no secret that South India is leading the charge in big data infrastructure. While the environmental impact of data centers has long been a concern, emerging hubs like Chennai are favored by companies seeking ready access to renewable energy resources, which provide more sustainable and cost-effective solutions. As a result, Chennai, along with neighboring Bengaluru and Hyderabad, is poised for significant growth, with a projected 65 percent increase in data center capacity over the next decade.
