Design, build, and optimize large-scale ETL pipelines and distributed data applications (batch and streaming) using Spark and Scala. Ensure data quality and performance, deploy and manage workloads in cloud/container environments, troubleshoot production issues, and collaborate with product, engineering, and analytics teams to deliver scalable AdTech data solutions.
We are looking for Senior Data Engineers to join our AdTech team and build scalable, high-performance data platforms that power advertising insights and analytics. The ideal candidate will have strong experience in distributed data processing, ETL pipelines, and Big Data technologies, with hands-on expertise in Spark and Scala.
You will work on designing, developing, and optimizing large-scale data pipelines while collaborating closely with cross-functional engineering teams to build reliable, production-grade data solutions.
Key Responsibilities
- Design, develop, and maintain scalable ETL pipelines for large-scale data processing.
- Build and optimize distributed data applications using Spark and Scala.
- Develop reliable, high-performance data pipelines for batch and streaming workloads.
- Work with large datasets to ensure data quality, consistency, and performance.
- Collaborate with engineering, product, and analytics teams to deliver robust data solutions.
- Optimize data workflows for scalability, reliability, and cost efficiency.
- Deploy and manage data workloads in cloud and containerized environments.
- Troubleshoot production issues and continuously improve platform performance.
Requirements
Candidates who demonstrate:
- 5+ years of experience in Data Engineering or Big Data Engineering.
- Strong hands-on experience with Apache Spark and Scala.
- Experience building and maintaining ETL pipelines.
- Familiarity with Google Cloud Storage (GCS).
- Experience with Kubernetes (K8s).
- Strong SQL skills and understanding of distributed data processing.
- Excellent debugging, problem-solving, and performance optimization skills.
- Strong communication and collaboration skills.
Good to Have
- Experience with AWS and cloud-native data services.
- Familiarity with streaming technologies such as Kafka.
- Experience working on large-scale data platforms or AdTech systems.
- Exposure to orchestration tools such as Airflow.
Benefits
- Best-in-class salary: We hire strong talent and compensate accordingly.
- Proximity Talks: Meet and learn from designers, engineers, product leaders, and AI practitioners.
- Continuous learning: Work with a world-class team and stay close to the latest in AI, engineering, and product development.
- High-impact work: Build AI-first systems and products used at scale by global clients.
About Us
Proximity is the trusted technology, design, and consulting partner for some of the biggest Sports, Media, and Entertainment companies in the world. We’re headquartered in San Francisco and have offices in Palo Alto, Dubai, Mumbai, and Bangalore.
Since 2019, Proximity has built high-impact, scalable products used by millions of users every day. Today, we are a global team of engineers, designers, product managers, and experts solving complex problems and building cutting-edge technology at scale.
Similar Jobs
Artificial Intelligence • Healthtech • Machine Learning • Natural Language Processing • Biotech • Pharmaceutical
Leads incentive compensation strategy, quota-setting, recognition programs, analytics, governance, implementation, and continuous improvement across Pfizer’s international markets. Partners with business leaders and cross-functional teams to design compliant, motivating, and financially responsible programs, deliver data-driven recommendations, oversee payouts and operational timelines, and maintain audit readiness. The role requires strong quantitative analysis, commercial operations expertise, project management, stakeholder influence, and experience supporting pharmaceutical or healthcare organizations.
Artificial Intelligence • Healthtech • Machine Learning • Natural Language Processing • Biotech • Pharmaceutical
Leads marketing channel delivery and medical samples services across Malaysia, Indonesia, Singapore, and the Philippines. Oversees sample planning, logistics, compliance, campaigns, HCP portal strategy, social communications, ePermission governance, and omnichannel reporting. Partners with marketing, digital, commercial, legal, compliance, and delivery teams to implement services, manage performance, train stakeholders, maintain audit readiness, and drive process improvement and AI-enabled innovation.
Top Skills:
AIEmail MarketingOmnichannel Reporting DashboardsPfizerpro
Automotive
Designs, develops, and supports scalable SAP solutions using ABAP, RAP, CDS, OData, and S/4HANA extensibility. Responsibilities include API-led integrations, Fiori/UI5 backend services, BTP extensions, testing, debugging, performance optimization, code reviews, architecture discussions, and Agile delivery. The role requires maintaining custom SAP developments across S/4HANA and ECC while following clean core principles, enterprise security standards, and SAP best practices.
Top Skills:
Abap Development Tools (Adt)Api ManagementCloud Application Programming Model (Cap)Core Data Services (Cds)EclipseGitIdocsObject-Oriented AbapOdata V2Odata V4Rest ApisRestful Abap Programming Model (Rap)RfcsSap AbapSap Btp Abap EnvironmentSap Business Technology Platform (Btp)Sap EccSap FioriSap GatewaySap Integration SuiteSap S/4HanaSoapUi5Web Services
What you need to know about the Chennai Tech Scene
To locals, it's no secret that South India is leading the charge in big data infrastructure. While the environmental impact of data centers has long been a concern, emerging hubs like Chennai are favored by companies seeking ready access to renewable energy resources, which provide more sustainable and cost-effective solutions. As a result, Chennai, along with neighboring Bengaluru and Hyderabad, is poised for significant growth, with a projected 65 percent increase in data center capacity over the next decade.
