emids Logo

emids

Senior Consultant

Posted 13 Days Ago
Be an Early Applicant
In-Office or Remote
Hiring Remotely in India
Senior level
In-Office or Remote
Hiring Remotely in India
Senior level
Design and build scalable data pipelines and a modern data platform on AWS using Databricks and Lakehouse architecture. Implement batch and streaming processing, cluster management, vacuuming, and pipeline cost modeling. Establish and drive engineering standards including CI/CD, unit testing, security scanning, and practical AI integration. Operate at an architect level to provide technical guidance and ownership.
The summary above was generated by AI

Databricks Engineer – Data Engineering (AWS) will build build scalable data pipelines and contribute to modern data platform development using AWS and Lakehouse architecture.

The candidate has solid hands-on Databricks experience and showed good engagement, passion, and familiarity with batch/streaming concepts, clustering, vacuuming, and cost-modeling considerations.

However, we do not recommend moving forward. The main concern is that the candidate frequently gave lengthy, indirect answers that did not address the specific questions asked, even after redirection. There were also notable gaps in modern data engineering practices, especially around CI/CD, unit testing, security scanning, and practical use of AI in development workflows.

Given the architect-level expectations for this role, the candidate may struggle to operate independently, provide clear guidance, and drive engineering standards effectively.

Similar Jobs

4 Days Ago
In-Office or Remote
Senior level
Senior level
Insurance
Designs, develops, and maintains complex application modules using Java and Spring Boot in a microservices architecture. Participates in design and code reviews, creates test plans, resolves defects, supports production deployments, documents solutions, estimates work, and collaborates with business partners to deliver tactical solutions and improve operational efficiency.
Top Skills: Java 17Java 8MicroservicesRest ApisSpring Boot
13 Days Ago
In-Office or Remote
India
Senior level
Senior level
Healthtech • Information Technology • Other • Software • Analytics
Design and build scalable data pipelines and modern data platform components on AWS Lakehouse using Databricks. Implement batch and streaming solutions, manage clusters and Delta Lake operations (vacuuming), optimize costs, and establish CI/CD, unit testing, security scanning, and AI-assisted development practices. Provide architect-level guidance and drive engineering standards.
Top Skills: AISparkAWSCi/CdDatabricksDelta LakeLakehouseSecurity ScanningUnit Testing
14 Days Ago
In-Office or Remote
India
Senior level
Senior level
Healthtech • Information Technology • Other • Software • Analytics
Design, develop, and maintain Splunk dashboards, reports, alerts and integrations. Administer Splunk Enterprise and forwarders, implement custom scripts (Python/PowerShell), analyze and optimize performance, collaborate with stakeholders in Agile ceremonies, mentor developers, and maintain documentation.
Top Skills: Data MiningPowershellPythonRegular Expressions (Regex)SplunkSplunk Dashboard DevelopmentSplunk EnterpriseSplunk ForwarderSQL

What you need to know about the Chennai Tech Scene

To locals, it's no secret that South India is leading the charge in big data infrastructure. While the environmental impact of data centers has long been a concern, emerging hubs like Chennai are favored by companies seeking ready access to renewable energy resources, which provide more sustainable and cost-effective solutions. As a result, Chennai, along with neighboring Bengaluru and Hyderabad, is poised for significant growth, with a projected 65 percent increase in data center capacity over the next decade.

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account