Seeking an experienced Data Engineer to design and maintain data pipelines, ensure data quality, and optimize real-time data processing. Requires expertise in SQL, Python, big data tools, and cloud platforms.
About the Role
RequirementsKey Responsibilities
We are looking for an experienced Data Engineer with strong expertise in building and maintaining robust data pipelines, optimizing data workflows, and ensuring the availability of high-quality data for business insights and real-time systems (RTS). The ideal candidate will have hands-on experience with modern data engineering tools, cloud platforms, and big data ecosystems.
RequirementsKey Responsibilities
- Design, build, and maintain scalable data pipelines and ETL/ELT processes.
- Work with structured and unstructured datasets to ensure data quality, accuracy, and availability.
- Collaborate with data scientists, analysts, and business stakeholders to deliver reliable datasets for analytics and reporting.
- Optimize query performance and ensure real-time/near-real-time data processing for RTS.
- Implement data security, governance, and compliance standards.
- Troubleshoot data-related issues and ensure system reliability and scalability.
- Stay updated with emerging tools and best practices in big data and cloud-based data engineering.
- 5–8 years of proven experience as a Data Engineer or in a similar role.
- Strong proficiency in SQL, Python, or Scala.
- Hands-on experience with big data technologies (Hadoop, Spark, Kafka, Hive, etc.).
- Experience with cloud platforms (AWS, Azure, or GCP) and their data services (e.g., Redshift, Snowflake, BigQuery, Data Lake).
- Strong understanding of ETL/ELT frameworks, data warehousing, and data modeling.
- Experience with real-time data streaming and processing frameworks.
- Good knowledge of CI/CD, version control (Git), and DevOps practices.
- Excellent problem-solving and communication skills.
- Experience in containerization and orchestration (Docker, Kubernetes, Airflow, etc.).
- Knowledge of data governance and security best practices.
- Familiarity with machine learning pipelines.
- Bachelor’s or Master’s degree in Computer Science, Information Technology, Engineering, or a related field.
Top Skills
Airflow
AWS
Azure
BigQuery
Ci/Cd
Data Lake
Docker
GCP
Git
Hadoop
Hive
Kafka
Kubernetes
Python
Redshift
Scala
Snowflake
Spark
SQL
Global Software Solutions Group Chennai, Tamil Nadu, IND Office
Third Cross Road,, SIPCOT IT Park, Siruseri, Chennai, Tamilnadu, India, 603103
Similar Jobs
Artificial Intelligence • Consumer Web • Edtech • Enterprise Web • HR Tech • Social Impact • Generative AI
The Trust & Safety Associate will investigate potential fraud, manage cases, improve detection systems, and collaborate with cross-functional teams to protect the platform.
Top Skills:
ExcelLookerSQL
Artificial Intelligence • Big Data • Logistics • Machine Learning • Software • Transportation
As a Staff Software Engineer, you'll develop AI/ML-integrated applications, lead technical architecture, mentor teams, and ensure system quality.
Top Skills:
AngularAWSAzureCi/CdDockerGCPJavaKafkaKubernetesMlflowNode.jsPythonReactSagemakerTensorFlow
Artificial Intelligence • Healthtech • Professional Services • Analytics • Consulting
Develop and apply advanced statistical models, use data to guide decisions, and communicate results to clients while mentoring Associates.
Top Skills:
ExcelHadoopJavaPythonRSASTableauVisual Basic
What you need to know about the Chennai Tech Scene
To locals, it's no secret that South India is leading the charge in big data infrastructure. While the environmental impact of data centers has long been a concern, emerging hubs like Chennai are favored by companies seeking ready access to renewable energy resources, which provide more sustainable and cost-effective solutions. As a result, Chennai, along with neighboring Bengaluru and Hyderabad, is poised for significant growth, with a projected 65 percent increase in data center capacity over the next decade.
