The Senior Kafka Developer will design and manage Kafka-based solutions, develop data pipelines for real-time processing, integrate data sources, optimize data storage on AWS, monitor Kafka clusters, and collaborate with teams for defining data processing requirements.
ProArch is seeking a skilled Senior Kafka Developer with AWS experience to join our innovative team. In this role, you will be responsible for designing, developing, and managing Kafka-based solutions that improve our data streaming capabilities and support our cloud infrastructure on AWS.
Key Responsibilities:
- Design, develop, and implement Kafka streams and data pipelines to facilitate real-time data processing and analytics.
- Integrate Kafka with various data sources and sinks, ensuring data flow across different applications.
- Work alongside data architects and engineers to optimize data flow and storage patterns in AWS.
- Monitor and troubleshoot Kafka clusters, ensuring high availability and reliability of services.
- Implement security measures and manage data governance to protect sensitive information.
- Collaborate with cross-functional teams to define and refine requirements for data processing functionalities.
- Document data pipeline designs, workflows, and operational processes for reference and knowledge sharing.
To be successful in this role, candidates should possess the following qualifications:
- 4+ years of experience in developing applications using Kafka.
- Strong understanding of AWS services such as Kinesis, S3, EC2, Lambda, and CloudFormation.
- Proficient in programming languages like Java, Scala, or Python.
- Experience with data modeling and schema design using Avro or JSON.
- Familiarity with monitoring tools for Kafka, such as Confluent Control Center or Grafana.
- Strong analytical and problem-solving skills.
- Excellent communication skills and ability to work in a team-oriented environment.
- Knowledge of data governance, security, and compliance frameworks is a plus.
Top Skills
Java
Python
Scala
Similar Jobs
As a Senior Software Engineer, you will develop and implement web-based applications and new features, troubleshoot issues across the stack, and mentor team members. You'll work in an agile environment, ensuring high-quality code while keeping up with industry trends.
The Senior Software Engineer will design, develop, and maintain microservices and distributed systems using Java and Spring Cloud, while integrating Pub/Sub messaging systems. Responsibilities include optimizing system performance, ensuring seamless communication between services, and collaborating with cross-functional teams. The role also involves conducting code reviews and mentoring junior developers.
As a Senior Software Engineer, you will design and develop scalable software applications, work with cross-functional teams to gather requirements, write clean code, maintain software, conduct code reviews, and implement security measures. You will also engage in troubleshooting and problem-solving using data structures and algorithms.
What you need to know about the Chennai Tech Scene
To locals, it's no secret that South India is leading the charge in big data infrastructure. While the environmental impact of data centers has long been a concern, emerging hubs like Chennai are favored by companies seeking ready access to renewable energy resources, which provide more sustainable and cost-effective solutions. As a result, Chennai, along with neighboring Bengaluru and Hyderabad, is poised for significant growth, with a projected 65 percent increase in data center capacity over the next decade.
