Yadimen Consulting Limited Jobs

Data Engineer — PySpark + AWS Glue

Yadimen Consulting Limited

Data Engineer — PySpark + AWS Glue

Posted 2 Months Ago

In-Office

Chennai, Tamil Nadu, IND

Senior level

In-Office

Chennai, Tamil Nadu, IND

Senior level

Design, develop, optimize, and maintain scalable ETL pipelines using PySpark and AWS Glue. Build data ingestion and transformation workflows, orchestrate processes with AWS Step Functions, and develop Python-based AWS Lambda functions. Support AWS data lake architectures, analytical use cases, monitoring, reliability, and performance optimization. The role also involves SQL and may include Java microservices, REST APIs, and backend integrations.

The summary above was generated by AI

Data Engineer — PySpark + AWS Glue

ETL & Data Pipelines | 2 Openings

Location: Remote (UK)

Employment Type: Contract / Permanent

Experience Level: 4–8 Years

Openings: 2

About the Role

We are looking for a skilled Data Engineer with solid hands-on experience in PySpark, AWS Glue, and ETL development to build and maintain scalable, production-grade data pipelines on AWS. You will be part of a delivery-focused team working on complex data engineering challenges, contributing to data lake architecture and end-to-end pipeline development.

Requirements

Key Responsibilities

• Design, develop, and maintain scalable ETL pipelines using PySpark and AWS Glue

• Build and optimise data ingestion, transformation, and loading workflows

• Orchestrate data workflows using AWS Step Functions

• Develop serverless functions using AWS Lambda (Python)

• Work with data lake architectures on AWS to support analytical use cases

• Ensure pipeline reliability, monitoring, and performance optimisation

Required Skills & Experience

• Strong hands-on experience with PySpark and AWS Glue

• Proven track record in ETL pipeline development and optimisation

• Experience orchestrating workflows with AWS Step Functions

• Proficiency in serverless development using AWS Lambda (Python)

• Good SQL skills and a solid understanding of data processing principles

Nice to Have

• Exposure to Java-based microservices

• Understanding of REST APIs and backend service integrations

• Experience working with AWS-based data lakes

Similar Jobs

Optum

Senior Software Engineer

3 Hours Ago

In-Office

Senior level

Artificial Intelligence • Big Data • Healthtech • Information Technology • Machine Learning • Software • Analytics

Develop and maintain scalable fullstack web applications using ReactJS and related technologies. Collaborate with designers, backend developers, product owners, and DevOps teams to deliver features, integrate frontend and backend systems, optimize performance, troubleshoot defects, conduct code reviews, implement automated testing, and support Agile development. The role also includes documentation, production deployment collaboration, mentoring junior developers, and maintaining current knowledge of frontend and fullstack best practices.

Top Skills: AgileBabelCSSCss PreprocessorsEnzymeExpress.JsGitHTMLJavaJavaScriptJestMongoDBMySQLNode.jsPostgresReact RouterReactReduxRestful ApisWebpack

Tufin

Assistant Controller

9 Hours Ago

Remote or Hybrid

India

Mid level

Security • Cybersecurity

Own global revenue recognition under ASC 606 and accounting processes including commissions, COGS, and leases. Review tax provisions, returns, transfer pricing, and subsidiary filings; support external audits and cross-functional reporting. Lead accounting projects, process improvements, and AI-driven workflow automation. Requires a CA, Big 4 high-tech experience, strong US GAAP and revenue recognition knowledge, ERP experience, advanced Excel skills, and excellent communication.

Top Skills: Ai Automation ToolsAsc 340-40Asc 606ExcelMS OfficeNetSuiteUs Gaap

CrowdStrike

Analyst, Alliances Shared Services

9 Hours Ago

Hybrid

Mid level

Cloud • Computer Vision • Information Technology • Sales • Security • Cybersecurity

Supports daily operations for CrowdStrike’s partner ecosystem by creating opportunities and CPQ quotes, maintaining Salesforce data integrity, enforcing pricing and attribution policies, producing reports and KPIs, monitoring SLAs, documenting processes, identifying anomalies, and coordinating effectively with alliance managers and stakeholders.

Top Skills: Cpq SoftwareExcelPowerPointSalesforce

What you need to know about the Chennai Tech Scene

To locals, it's no secret that South India is leading the charge in big data infrastructure. While the environmental impact of data centers has long been a concern, emerging hubs like Chennai are favored by companies seeking ready access to renewable energy resources, which provide more sustainable and cost-effective solutions. As a result, Chennai, along with neighboring Bengaluru and Hyderabad, is poised for significant growth, with a projected 65 percent increase in data center capacity over the next decade.