Crayon Data Logo

Crayon Data

Lead Data Engineer

Posted 4 Days Ago
Be an Early Applicant
In-Office
Chennai, Tamil Nadu, IND
Senior level
In-Office
Chennai, Tamil Nadu, IND
Senior level
Design, build, and scale cloud-native data platforms and pipelines for AI and analytics. Lead data engineering projects, implement big data processing (Spark/Hadoop/Hive/HBase), ensure data quality/governance, and collaborate with cross-functional teams to deliver production-ready datasets and data-driven product features.
The summary above was generated by AI
Lead Data Engineer – Chennai (Data Platform & Big Data Focus)
Build the foundation of AI — engineer the data systems that power enterprise intelligence
Location: Chennai, India
Experience: 5+ years in Data Engineering / Software Engineering

Problem solvers who don’t just move data but build scalable, reliable data platforms that drive intelligent business decisions.

Role Overview

As a Lead Data Engineer, you will design, build, and scale data platforms that power AI-driven products and analytics solutions. You will work closely with data scientists, product managers, and engineering teams to enable seamless data flow, storage, and processing for enterprise use cases.

What makes you right for this role

Data Platform Development – Design and build scalable data pipelines and data lake architectures for high-performance analytics.

End-to-End Engineering – Develop robust data ingestion, transformation, and processing systems from raw data to production-ready datasets.

Cloud-Native Development – Architect and deploy data solutions on AWS, Azure, or GCP with strong DevOps and automation practices.

Big Data Expertise – Work with distributed data processing frameworks like Spark, Hadoop, and Hive to handle large-scale data workloads.

Collaboration – Partner with cross-functional teams to deliver reliable and impactful data solutions.

Best Practices – Ensure data quality, governance, security, and performance through standardization and continuous improvement.

The person you’re looking for

Strong experience in data engineering with proficiency in Python, Scala, or Java.

Hands-on expertise in building scalable data pipelines and distributed systems.

Good understanding of data lake architecture, data warehousing, and relational databases (Oracle, SQL, DB2, Teradata).

Experience with Big Data technologies such as Hadoop, Spark, Hive, and HBase.

Familiarity with workflow orchestration tools like Airflow.

Experience with cloud platforms (AWS, Azure, or GCP).

Strong problem-solving mindset with the ability to take ownership and lead initiatives.

Passionate about data, innovation, and solving real-world enterprise challenges.

Chennai-based with flexibility to collaborate with global teams and stakeholders.

What you will do

Lead and deliver data engineering and analytics projects that create business impact.

Build and optimize data pipelines and infrastructure for scalability and efficiency.

Collaborate with teams to understand requirements and translate them into technical solutions.

Communicate complex data concepts to both technical and business stakeholders.

Drive data strategy and support analytics-driven decision-making.

Design and develop data-driven features and components for product platforms.

Travel to client locations when required.



Crayon Data Chennai, Tamil Nadu, IND Office

5th Floor, # 33-B, Software Block, ELNET SOFTWARE CITY, Old Mahabalipuram Road, Taramani, Chennai, Tamil Nadu, India, 600 113

Similar Jobs

5 Days Ago
In-Office
Chennai, Tamil Nadu, IND
Senior level
Senior level
Financial Services
Lead test engineering teams to design and execute system integration and automated tests across UI, services, and APIs. Develop and maintain reusable automation frameworks, test plans, and procedures; partner with product, development, and DevOps; track execution, risks, and environment/data dependencies; ensure process compliance and mentor junior engineers.
Top Skills: AlmAWSBitbucketCloudbeesCucumberDb2GitJavaJdbcJenkinsJIRAJpaJunitKarateOraclePythonRest AssuredSeleniumSeleniumSnowflakeSQLUnix
4 Days Ago
In-Office
Chennai, Tamil Nadu, IND
Senior level
Senior level
Information Technology
Lead Azure Data Engineer to design, build, and operate petabyte-scale data systems on Azure. Lead a data engineering team to develop ETL pipelines (ADF, Synapse, Databricks), perform data modelling, migrations, monitoring, security, and collaborate with BI/ML/Product teams while mentoring junior engineers and improving analytics and dashboarding.
Top Skills: AdfAzure Analysis ServicesAzure Data FactoryAzure Data LakeAzure DatabricksAzure Dedicated Sql PoolAzure SqlAzure Synapse AnalyticsData ModellingPower BIPysparkSparksqlSQLSsisSynapse NotebookSynapse PipelineTableau
8 Days Ago
In-Office or Remote
Senior level
Senior level
Artificial Intelligence • Big Data • Cloud • Generative AI
Lead design and delivery of AWS Data Lakehouse solutions using Medallion architecture. Architect and automate cloud infrastructure, optimize Redshift/Glue performance and costs, enforce data governance and quality, implement CDC/ETL pipelines, and liaise with Data Science/BI and business stakeholders. Mentor engineers and drive technical roadmap and migrations.
Top Skills: Amazon AthenaAmazon RedshiftAmazon S3Apache AirflowAws DmsAws GlueAws Glue Data QualityAws Glue DatabrewAws Lake FormationAws Step FunctionsDbtDockerGithub ActionsIamMicrosoft FabricOpenlineageParquetPower BIPower Bi EmbeddedPysparkPythonRedshift ServerlessSnowflakeSQLTerraform

What you need to know about the Chennai Tech Scene

To locals, it's no secret that South India is leading the charge in big data infrastructure. While the environmental impact of data centers has long been a concern, emerging hubs like Chennai are favored by companies seeking ready access to renewable energy resources, which provide more sustainable and cost-effective solutions. As a result, Chennai, along with neighboring Bengaluru and Hyderabad, is poised for significant growth, with a projected 65 percent increase in data center capacity over the next decade.

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account