Lead the design and implementation of real-time data pipelines and scalable data streaming solutions for healthcare data engineering projects.
Role Overview:
We are looking for a highly skilled with expertise in Kafka, Python, and Azure Databricks (preferred) to drive our healthcare data engineering projects. The ideal candidate will have deep experience in real-time data streaming, cloud-based data platforms, and large-scale data processing. This role requires strong technical leadership, problem-solving abilities, and the ability to collaborate with cross-functional teams.
Key Responsibilities:
- Lead the design, development, and implementation of real-time data pipelines using Kafka, Python, and Azure Databricks.
- Architect scalable data streaming and processing solutions to support healthcare data workflows.
- Develop, optimize, and maintain ETL/ELT pipelines for structured and unstructured healthcare data.
- Ensure data integrity, security, and compliance with healthcare regulations (HIPAA, HITRUST, etc.).
- Collaborate with data engineers, analysts, and business stakeholders to understand requirements and translate them into technical solutions.
- Troubleshoot and optimize Kafka streaming applications, Python scripts, and Databricks workflows.
- Mentor junior engineers, conduct code reviews, and ensure best practices in data engineering.
- Stay updated with the latest cloud technologies, big data frameworks, and industry trends.
Required Skills & Qualifications:
- 4+ years of experience in data engineering, with strong proficiency in Kafka and Python.
- Expertise in Kafka Streams, Kafka Connect, and Schema Registry for real-time data processing.
- Experience with Azure Databricks (or willingness to learn and adopt it quickly).
- Hands-on experience with cloud platforms (Azure preferred, AWS or GCP is a plus).
- Proficiency in SQL, NoSQL databases, and data modeling for big data processing.
- Knowledge of containerization (Docker, Kubernetes) and CI/CD pipelines for data applications.
- Experience working with healthcare data (EHR, claims, HL7, FHIR, etc.) is a plus.
- Strong analytical skills, problem-solving mindset, and ability to lead complex data projects.
- Excellent communication and stakeholder management skills.
Top Skills
Azure Databricks
Docker
Kafka
Kubernetes
NoSQL
Python
SQL
Similar Jobs
Healthtech • Information Technology • Other • Software • Analytics
Seeking a Senior Backend Engineer with expertise in Scala, Clojure, Java, and Oracle databases for a migration initiative to AlloyDB on GCP.
Top Skills:
AlloydbClojureGCPJavaOracleScala
Healthtech • Information Technology • Other • Software • Analytics
The Senior Consultant will utilize Edifecs tools for healthcare integration, manage EDI transactions, and leverage SQL database skills for data transformation and mapping in a DevOps environment.
Top Skills:
Edifecs Healthcare Transaction RepositoryEdifecs Integration ConsoleEdifecs SpecbuilderEdifecs Transaction ManagerEdifecs Xengine ServerGitGroovyJavaJavaScriptJSONMySQLOracleRdbmsRestSoapSpecbuilderSQLSQL ServerXMLXslt
Healthtech • Information Technology • Other • Software • Analytics
The role involves designing and developing scalable web applications, utilizing both backend and frontend technologies, and providing mentorship.
Top Skills:
AngularJavaPostgres
What you need to know about the Chennai Tech Scene
To locals, it's no secret that South India is leading the charge in big data infrastructure. While the environmental impact of data centers has long been a concern, emerging hubs like Chennai are favored by companies seeking ready access to renewable energy resources, which provide more sustainable and cost-effective solutions. As a result, Chennai, along with neighboring Bengaluru and Hyderabad, is poised for significant growth, with a projected 65 percent increase in data center capacity over the next decade.
