The RTS Kafka Engineer will design and maintain scalable real-time streaming platforms, optimizing Kafka-based solutions, and collaborate with teams for efficient data processing in cloud environments.
Job Summary
We are looking for a highly skilled RTS Kafka Engineer to design, build, and maintain scalable real-time streaming platforms. The ideal candidate will have strong hands-on experience with Apache Kafka, cloud-native architectures, and event-driven systems operating in high-throughput environments. This role involves building reliable streaming pipelines, enabling real-time data processing, and supporting cloud-based distributed systems.
Key Responsibilities- Design, implement, and manage Kafka-based real-time streaming solutions in high-throughput environments.
- Perform Kafka setup, configuration, tuning, and performance optimization.
- Develop and maintain scalable event-driven and streaming architectures.
- Build and manage streaming data pipelines processing large-scale datasets.
- Implement CI/CD pipelines using tools such as Git, Jenkins, ArgoCD, or similar platforms.
- Work with cloud-native applications deployed across AWS, Azure, or OCI environments.
- Automate infrastructure and deployment using cloud automation toolsets.
- Apply GitOps practices for deployment and environment management.
- Ensure system reliability, scalability, and high availability.
- Implement automated unit testing and integration testing for streaming applications.
- Collaborate with DevOps, platform engineering, and application teams to deliver end-to-end real-time solutions.
- Apply software design patterns and best practices in distributed systems development.
- Strong hands-on experience with Apache Kafka setup, configuration, and administration.
- Experience working with high-volume streaming data platforms.
- Solid understanding of event-driven architecture and distributed systems.
- Experience with source control systems (Git).
- Hands-on exposure to CI/CD pipelines (Jenkins, ArgoCD, or similar).
- Experience with at least one public cloud platform:
- AWS
- Microsoft Azure
- Oracle Cloud Infrastructure (OCI)
- Knowledge of cloud automation tools and infrastructure automation.
- Understanding of GitOps methodology.
- Strong knowledge of software design patterns and implementation practices.
- Experience implementing automated testing (unit & integration testing).
- Experience with Kafka ecosystem tools (Kafka Connect, Schema Registry, ksqlDB).
- Exposure to containerization and orchestration platforms (Docker, Kubernetes, OpenShift).
- Experience with monitoring and observability tools (Prometheus, Grafana, ELK, etc.).
- Background in real-time systems (RTS) or financial transaction platforms is a plus.
Global Software Solutions Group Chennai, Tamil Nadu, IND Office
Third Cross Road,, SIPCOT IT Park, Siruseri, Chennai, Tamilnadu, India, 603103
Similar Jobs
Digital Media • Information Technology • News + Entertainment
Design, develop, and execute ETL test strategies and automated validation for data ingestion, transformation, and loading. Validate data quality, perform reconciliation and lineage checks, develop SQL/Python validation scripts, monitor ETL workflows, collaborate with engineering and product teams, participate in code reviews and defect triage, and support data migrations and platform upgrades.
Top Skills:
AirflowAPIsAutomation Test FrameworkCi/CdData LakesData WarehousesDatabricksDbtETLJavaPythonSnowflakeSQL
Digital Media • Information Technology • News + Entertainment
Design, develop, test and integrate software and web applications. Produce technical documentation, train operations staff, mentor junior engineers, collaborate with QA, present architectural choices, and support software releases while applying project management and independent judgment.
Digital Media • Information Technology • News + Entertainment
Design, build, and optimize scalable data pipelines and platforms (Snowflake, Databricks, Spark). Ensure data quality, lineage, and observability. Mentor junior engineers, lead cross-team initiatives, implement CI/CD, and improve internal data engineering tools and architectures.
Top Skills:
AirflowSparkAWSAws S3AzureCi/CdDatabricksDbtGCPIcebergPythonSnowflakeSQL
What you need to know about the Chennai Tech Scene
To locals, it's no secret that South India is leading the charge in big data infrastructure. While the environmental impact of data centers has long been a concern, emerging hubs like Chennai are favored by companies seeking ready access to renewable energy resources, which provide more sustainable and cost-effective solutions. As a result, Chennai, along with neighboring Bengaluru and Hyderabad, is poised for significant growth, with a projected 65 percent increase in data center capacity over the next decade.
