PDI Technologies Logo

PDI Technologies

Data Engineer II

Posted 2 Days Ago
Be an Early Applicant
Hybrid
Chennai, Tamil Nadu, IND
Mid level
Hybrid
Chennai, Tamil Nadu, IND
Mid level
Design, build, and maintain AWS-native cloud data pipelines and ETL/ELT processes (S3, Glue, Redshift, AppFlow, Step Functions). Integrate enterprise apps (Salesforce, NetSuite), develop scalable data models, monitor pipeline performance, ensure data quality and governance, support CI/CD and IaC (Terraform), apply AI/ML for data quality/monitoring, and collaborate with stakeholders and peers.
The summary above was generated by AI
At PDI Technologies, we empower some of the world's leading convenience retail and petroleum brands with cutting-edge technology solutions that drive growth and operational efficiency.
By “Connecting Convenience” across the globe, we empower businesses to increase productivity, make more informed decisions, and engage faster with customers through loyalty programs, shopper insights, and unmatched real-time market intelligence via mobile applications, such as GasBuddy.  We’re a global team committed to excellence, collaboration, and driving real impact. Explore our opportunities and become part of a company that values diversity, integrity, and growth.

Role Overview:

As a Data Engineer II, you will design, develop, and support scalable cloud-based data pipelines that enable analytics, reporting, and business intelligence across the organization. Working within AWS-native technologies, you will integrate enterprise applications into centralized data platforms while ensuring data quality, reliability, and performance. You will collaborate with business stakeholders and technical teams to translate reporting requirements into scalable data solutions that support informed business decisions.

 

Key Responsibilities

    • Design, develop, and maintain cloud-based data pipelines supporting enterprise reporting and analytics.

    • Build and optimize ETL/ELT processes using AWS-native data services and modern data engineering practices.

    • Integrate enterprise source systems, including Salesforce, NetSuite, and other business applications, into centralized data platforms.

    • Develop and maintain scalable data models supporting reporting and business intelligence initiatives.

    • Monitor pipeline performance, troubleshoot data issues, and implement improvements to ensure data accuracy and reliability.

    • Partner with business stakeholders to understand reporting requirements and translate them into technical solutions.

    • Support data governance initiatives by maintaining data quality, metadata, and documentation.

    • Contribute to automation, continuous integration, and infrastructure-as-code practices to improve development efficiency.

    • Participate in code reviews and promote engineering best practices across the team.

Qualifications

    • Bachelor's degree in Computer Science, Information Systems, Engineering, or related field, or equivalent experience.

    • 2-4 years of experience in data engineering or cloud data platform development.

    • Experience developing data pipelines using AWS services such as S3, Glue, Redshift, AppFlow, and Step Functions.

    • Experience working with SQL, Python, and PySpark.

    • Understanding of dimensional modeling, data warehousing concepts, and modern data lake technologies.

    • Experience integrating enterprise applications such as Salesforce, NetSuite, or similar SaaS platforms.

    • Familiarity with Git, CI/CD pipelines, and Infrastructure as Code (Terraform preferred).

    • Practical experience with applying AI/ML techniques to data engineering problems, such as automated data quality checks, anomaly detection, or intelligent pipeline monitoring

    • Strong analytical and problem-solving skills.

    • Excellent collaboration and communication skills.

Behavioral Competencies

  • Ensures Accountability

  • Manages Complexity

  • Communicates Effectively

  • Balances Stakeholders

  • Collaborates Effectively

Similar Jobs

9 Days Ago
In-Office
Chennai, Tamil Nadu, IND
Senior level
Senior level
Fintech • Payments • Software • Financial Services
Design and maintain scalable GCP-based data pipelines and MLOps workflows. Productionize, deploy, monitor, and optimize ML models (Vertex AI, Ray). Manage Kubernetes (GKE) clusters, build CI/CD for ML with Docker, implement real-time streaming (Apache Flink), and apply advanced techniques like Data Sketches for performant financial analytics.
Top Skills: Apache FlinkData SketchesDockerGCPKubernetes (Gke)PythonRaySQLVertex AiVertex Ai EndpointsVertex Ai Feature StoreVertex Ai Model RegistryVertex Ai Pipelines
10 Days Ago
Hybrid
Chennai, Tamil Nadu, IND
Senior level
Senior level
Fintech • Financial Services
Lead and contribute to design, architecture, and hands-on development of scalable data platforms and ETL pipelines on cloud (GCP). Drive best practices, CI/CD, integrations, API specs, code reviews, and collaborate with product and stakeholders to deliver high-availability, big-data solutions and BI reporting.
Top Skills: ApacheBig Data LakeBigQueryCi/CdDaxETLGitGoogle Cloud PlatformGoogle Cloud StorageHive SqlHTTPJenkinsMavenPower BIPower QuerySaltScalaSecure File Transfer (Sft)SparkSQLUnix/Linux
11 Days Ago
Hybrid
Chennai, Tamil Nadu, IND
Mid level
Mid level
Fintech • Financial Services
Design, develop, and maintain data platforms and ETL processes on cloud-based big data systems. Own architecture, write code and APIs, build automation and CI/CD pipelines, collaborate with product managers, perform code reviews, and support data integration, modeling, and BI solutions for global customers.
Top Skills: ApacheSparkBig Data LakeBigQueryCi/CdDaxETLGitGoogle Cloud PlatformGoogle Cloud StorageHive SqlHTTPJenkinsMavenPower BIPower QuerySaltScalaSecure File Transfer (Sft)SQLUnix/Linux

What you need to know about the Chennai Tech Scene

To locals, it's no secret that South India is leading the charge in big data infrastructure. While the environmental impact of data centers has long been a concern, emerging hubs like Chennai are favored by companies seeking ready access to renewable energy resources, which provide more sustainable and cost-effective solutions. As a result, Chennai, along with neighboring Bengaluru and Hyderabad, is poised for significant growth, with a projected 65 percent increase in data center capacity over the next decade.

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account