Design, build, and optimize large-scale ETL pipelines and distributed data applications (batch and streaming) using Spark and Scala. Ensure data quality and performance, deploy and manage workloads in cloud/container environments, troubleshoot production issues, and collaborate with product, engineering, and analytics teams to deliver scalable AdTech data solutions.
We are looking for Senior Data Engineers to join our AdTech team and build scalable, high-performance data platforms that power advertising insights and analytics. The ideal candidate will have strong experience in distributed data processing, ETL pipelines, and Big Data technologies, with hands-on expertise in Spark and Scala.
You will work on designing, developing, and optimizing large-scale data pipelines while collaborating closely with cross-functional engineering teams to build reliable, production-grade data solutions.
Key Responsibilities
- Design, develop, and maintain scalable ETL pipelines for large-scale data processing.
- Build and optimize distributed data applications using Spark and Scala.
- Develop reliable, high-performance data pipelines for batch and streaming workloads.
- Work with large datasets to ensure data quality, consistency, and performance.
- Collaborate with engineering, product, and analytics teams to deliver robust data solutions.
- Optimize data workflows for scalability, reliability, and cost efficiency.
- Deploy and manage data workloads in cloud and containerized environments.
- Troubleshoot production issues and continuously improve platform performance.
Requirements
Candidates who demonstrate:
- 5+ years of experience in Data Engineering or Big Data Engineering.
- Strong hands-on experience with Apache Spark and Scala.
- Experience building and maintaining ETL pipelines.
- Familiarity with Google Cloud Storage (GCS).
- Experience with Kubernetes (K8s).
- Strong SQL skills and understanding of distributed data processing.
- Excellent debugging, problem-solving, and performance optimization skills.
- Strong communication and collaboration skills.
Good to Have
- Experience with AWS and cloud-native data services.
- Familiarity with streaming technologies such as Kafka.
- Experience working on large-scale data platforms or AdTech systems.
- Exposure to orchestration tools such as Airflow.
Benefits
- Best-in-class salary: We hire strong talent and compensate accordingly.
- Proximity Talks: Meet and learn from designers, engineers, product leaders, and AI practitioners.
- Continuous learning: Work with a world-class team and stay close to the latest in AI, engineering, and product development.
- High-impact work: Build AI-first systems and products used at scale by global clients.
About Us
Proximity is the trusted technology, design, and consulting partner for some of the biggest Sports, Media, and Entertainment companies in the world. We’re headquartered in San Francisco and have offices in Palo Alto, Dubai, Mumbai, and Bangalore.
Since 2019, Proximity has built high-impact, scalable products used by millions of users every day. Today, we are a global team of engineers, designers, product managers, and experts solving complex problems and building cutting-edge technology at scale.
Similar Jobs
Artificial Intelligence • Cloud • Computer Vision • Hardware • Internet of Things • Software
Manage enterprise customer accounts as a trusted technical advisor, driving platform adoption, measurable business value, and technical ROI. Develop success plans, troubleshoot complex hardware, software, and API issues, manage escalations, conduct root-cause analysis, and lead technical reviews. Collaborate with Sales, Support, Product, and Engineering while communicating effectively with technical and executive stakeholders. Use AI tools to improve customer insights and outcomes, and contribute to knowledge sharing and team development.
Top Skills:
AIAPIsGainsightGongInternet Of Things (Iot)JIRAPaasPythonSaaSSalesforceTableauZendesk
Blockchain • Fintech • Payments • Consulting • Cryptocurrency • Cybersecurity • Quantum Computing
Lead product strategy and delivery for Mastercard's In Control virtual card platform. Translate business requirements into technical specs, work with engineering scrum teams, engage regional partners and customers, define platform vision for tokenized and virtual card payments, validate technical delivery, and drive go-to-market and product enhancements to grow B2B and T&E payment adoption.
Top Skills:
AgilePayment ProcessingPme FrameworkScrumTokenizationUx/UiVirtual Card Platform
Fintech • Professional Services • Consulting • Energy • Financial Services • Cybersecurity • Generative AI
Tests liquidity risk and regulatory reporting releases across PRA, EBA, HKMA, and US Federal Reserve requirements. Responsibilities include SQL-based data validation, test strategy and execution, defect management, UAT dashboards, requirements gathering, data-gap analysis, issue resolution, and stakeholder communication. The role also supports finance data quality initiatives, regulatory compliance, operating-model transitions, change implementation, and coordination across Finance, Risk, Business, and IT teams.
Top Skills:
ExcelMicrosoft PowerpointMicrosoft ProjectMicrosoft VisioMicrosoft WordQuality CentreSQL
What you need to know about the Chennai Tech Scene
To locals, it's no secret that South India is leading the charge in big data infrastructure. While the environmental impact of data centers has long been a concern, emerging hubs like Chennai are favored by companies seeking ready access to renewable energy resources, which provide more sustainable and cost-effective solutions. As a result, Chennai, along with neighboring Bengaluru and Hyderabad, is poised for significant growth, with a projected 65 percent increase in data center capacity over the next decade.
