The Senior Data Engineer will work with Databricks and PySpark, focusing on system design, performance monitoring, and integrating solutions while mentoring junior engineers.
Job Summary: Looking for a developer who has hands on experience with Databricks/Pyspark , Java/ python, Webservices and SQL. Position is long term and will likely be renewed annually.
Essential Job Functions:
- Understands the full picture and helps operations with monitoring, dashboards etc.
- work with architect and tech lead on design
- constantly keeps an eye on cost and performance of the system
- work with BA on the integration requirements
- Ability to trouble shoot performance issue and other functional issues
- Attention to details and documents the key decisions made
- Ability to handle fast paced projects and willing to look into production issues
- Should be a quick/self learner and be able to adopt to new technologies
- Prompt response /communication needed
Minimum Qualifications and Job Requirements:
- Must be a team player.
- Must have at least 5 years of IT development experience.
- Must have strong analytical and problem-solving skills.
- Must have experience in designing solutions, performing code reviews, mentoring junior engineers.
- Must have strong SQL and backend experience, and working on data driven projects.
Must have following experience:
- Python/PySpark
- SQL/ PL/SQL
- Databricks
- Java/C#
Preferred to have following experience:
- Azure
- Kafka
- Node JS
- Azure Data Factory
Similar Jobs
Artificial Intelligence • Big Data • Healthtech • Information Technology • Machine Learning • Software • Analytics
Support and operate SQL Server and MongoDB Atlas platforms, ensuring performance and reliability in a 24/7 production environment, including troubleshooting and incident management.
Top Skills:
AWSAzureGCPGitKubernetesMongodb AtlasPowershellPythonShellSQL ServerTerraform
Artificial Intelligence • Big Data • Healthtech • Information Technology • Machine Learning • Software • Analytics
The Principal Data Engineer supports and operates Oracle and MongoDB Atlas platforms, ensuring operational stability, troubleshooting, and performance optimization in cloud-based environments.
Top Skills:
AWSAzureGCPGitKubernetesMongodb AtlasOraclePowershellPythonShellTerraform
Big Data • Fintech • Information Technology • Business Intelligence • Financial Services • Cybersecurity • Big Data Analytics
This role develops credit risk management solutions and business intelligence analytics by engaging with clients and performing statistical analysis. It requires both strong programming skills and project management abilities.
Top Skills:
C/C++ExcelHadoopHiveJavaPigPythonRSparkSQLTableau
What you need to know about the Chennai Tech Scene
To locals, it's no secret that South India is leading the charge in big data infrastructure. While the environmental impact of data centers has long been a concern, emerging hubs like Chennai are favored by companies seeking ready access to renewable energy resources, which provide more sustainable and cost-effective solutions. As a result, Chennai, along with neighboring Bengaluru and Hyderabad, is poised for significant growth, with a projected 65 percent increase in data center capacity over the next decade.
