Design, develop, and optimize data pipelines and analytics solutions using Databricks and PySpark, ensuring data quality and compliance.
Senior Data Engineer (Data Bricks)
Primary Skills
- ETL Fundamentals, SQL, SQL (Basic + Advanced), Python, Data Warehousing, PLSQL, T-SQL, Stored Procedures
- Modern Data Platform Fundamentals, Data Modelling Fundamentals, Data Ware Housing
- Databricks
- Cloud
Specialization
- Databricks Engineering: Senior Data Engineer
Job requirements
- Role Overview:
- We are seeking a highly skilled Databricks Engineer to design, develop, and optimize data pipelines and analytics solutions. The ideal candidate will have strong expertise in Databricks, PySpark, and SQL, with experience in building scalable data solutions across cloud platforms.
- Key Responsibilities:
- • Design and implement data pipelines and ETL processes using Databricks and PySpark.
- • Develop and optimize SQL queries for data transformation and analytics.
- • Collaborate with data architects, analysts, and business stakeholders to deliver high-quality data solutions.
- • Ensure data quality, governance, and compliance across all processes.
- • Integrate data from multiple sources and manage large-scale datasets efficiently.
- • Troubleshoot and optimize performance for data workflows and queries.
- Primary Skills (Must-Have):
- • Databricks (including Delta Lake and ML capabilities)
- • PySpark for distributed data processing
- • SQL for data manipulation and analytics
- Good-to-Have Skills:
- • Azure Services: ADF, Synapse, Purview, Fabric
- • AWS Services: Glue, Lambda, Step Functions
- • Workflow Orchestration: Airflow
- • Data Transformation & Integration: DBT, Fivetran, Informatica
- • Streaming & Messaging: Kafka
- • BI & Visualization: Power BI
- • Data Governance Tools: Collibra, Alation
- • GCP: BigQuery
- Qualifications:
- • Bachelor’s or Master’s degree in Computer Science, Data Engineering, or related field.
- • Proven experience in building and managing data pipelines in cloud environments.
- • Strong problem-solving and analytical skills.
- • Excellent communication and collaboration abilities.
- Nice to Have:
- • Experience with multi-cloud environments (Azure, AWS, GCP).
- • Familiarity with modern data stack and data governance frameworks.
- Promoting Women Professionals
Top Skills
Airflow
Alation
Aws Services
Azure Services
BigQuery
Collibra
Databricks
Dbt
Fivetran
GCP
Informatica
Kafka
Power BI
Pyspark
SQL
Brillio Chennai, Tamil Nadu, IND Office
Chennai, India
Similar Jobs
Cloud • Computer Vision • Information Technology • Sales • Security • Cybersecurity
Drive enterprise-wide automation initiatives by designing and implementing solutions using N8N and Tray.ai. Collaborate with teams to automate processes, integrate systems, and ensure high availability of workflows while mentoring others and reporting on impact.
Top Skills:
Gemini EnterpriseJSONMcpN8NPythonRest ApisSQLTray.AiXML
Cloud • Computer Vision • Information Technology • Sales • Security • Cybersecurity
The role involves designing and implementing automation solutions using N8N and Tray.ai, developing AI-powered workflows, and integrating various applications while collaborating with cross-functional teams.
Top Skills:
Ci/CdCoupaDevOpsGemini EnterpriseJSONMcpN8NNetSuitePythonRest ApisSalesforceSAPSnowflakeSQLTray.IoWorkdayXML
Big Data • Fintech • Information Technology • Business Intelligence • Financial Services • Cybersecurity • Big Data Analytics
The Consultant Database Administrator will manage and optimize DB2 and PostgreSQL databases, writing complex SQL queries, ensuring data integrity, and implementing security measures.
Top Skills:
Db2PostgresSQL
What you need to know about the Chennai Tech Scene
To locals, it's no secret that South India is leading the charge in big data infrastructure. While the environmental impact of data centers has long been a concern, emerging hubs like Chennai are favored by companies seeking ready access to renewable energy resources, which provide more sustainable and cost-effective solutions. As a result, Chennai, along with neighboring Bengaluru and Hyderabad, is poised for significant growth, with a projected 65 percent increase in data center capacity over the next decade.
