The Senior Data Engineer will work with Databricks and PySpark, focusing on system design, performance monitoring, and integrating solutions while mentoring junior engineers.
Job Summary: Looking for a developer who has hands on experience with Databricks/Pyspark , Java/ python, Webservices and SQL. Position is long term and will likely be renewed annually.
Essential Job Functions:
- Understands the full picture and helps operations with monitoring, dashboards etc.
- work with architect and tech lead on design
- constantly keeps an eye on cost and performance of the system
- work with BA on the integration requirements
- Ability to trouble shoot performance issue and other functional issues
- Attention to details and documents the key decisions made
- Ability to handle fast paced projects and willing to look into production issues
- Should be a quick/self learner and be able to adopt to new technologies
- Prompt response /communication needed
Minimum Qualifications and Job Requirements:
- Must be a team player.
- Must have at least 5 years of IT development experience.
- Must have strong analytical and problem-solving skills.
- Must have experience in designing solutions, performing code reviews, mentoring junior engineers.
- Must have strong SQL and backend experience, and working on data driven projects.
Must have following experience:
- Python/PySpark
- SQL/ PL/SQL
- Databricks
- Java/C#
Preferred to have following experience:
- Azure
- Kafka
- Node JS
- Azure Data Factory
Top Skills
Azure
Azure Data Factory
Databricks
Java
Kafka
Node Js
Pl/Sql
Pyspark
Python
SQL
Similar Jobs
Fintech • Professional Services • Consulting • Energy • Financial Services • Cybersecurity • Generative AI
The Liquidity Risk Tester will execute tests, manage test data, and develop test strategies within cloud-native banking applications, utilizing Agile and DevOps methodologies.
Top Skills:
AWSAzureAzure DevopsGCPGitlab CiHp AlmJenkinsJIRAQuality CentreSQL
Big Data • Cloud • Software • Database
Develop automated frameworks for application and database modernization, ensuring functional equivalence and performance validation, while directing a team of engineers.
Top Skills:
AngularJavaJdbcJpaLlmsMongoDBMySQLPostgresReactSpringSQL
Artificial Intelligence • Big Data • Healthtech • Information Technology • Machine Learning • Software • Analytics
The ITSM Analyst manages ITSM practices, monitors operations, drives improvements, and facilitates collaboration across teams to enhance service delivery governance and efficiency.
Top Skills:
ItilItsmServicenow
What you need to know about the Chennai Tech Scene
To locals, it's no secret that South India is leading the charge in big data infrastructure. While the environmental impact of data centers has long been a concern, emerging hubs like Chennai are favored by companies seeking ready access to renewable energy resources, which provide more sustainable and cost-effective solutions. As a result, Chennai, along with neighboring Bengaluru and Hyderabad, is poised for significant growth, with a projected 65 percent increase in data center capacity over the next decade.
