Apache Spark Module Lead

Posted Yesterday
Be an Early Applicant
Chennai, Tamil Nadu
Junior
Information Technology • Consulting
The Role
The Apache Spark Module Lead will leverage strong programming skills in Python and Scala to build and optimize data pipelines, implement data ingestion pipelines, and work with big data tools like Spark and Azure. Responsibilities also include team leadership, user training, and maintaining effective communication with customers.
Summary Generated by Built In

Company Description

About Sopra Steria
Sopra Steria, a major Tech player in Europe with 56,000 employees in nearly 30 countries, is recognized for its consulting, digital services and software development. It helps its clients drive their digital transformation and obtain tangible and sustainable benefits. The Group provides end-to-end solutions to make large companies and organizations more competitive by combining in-depth knowledge of a wide range of business sectors and innovative technologies with a fully collaborative approach. Sopra Steria places people at the heart of everything it does and is committed to putting digital to work for its clients in order to build a positive future for all. In 2023, the Group generated revenues of €5.8 billion.
The world is how we shape it.

Job Description


Data Engineer with Big Data development experience - (4-5 yrs. experience)


Required Skills:

  • Strong expertise of programming languages – Python/Scala/others
  • Hands-on experience in Spark with Scala and Python(PySpark) including Data frame core functions, SparkSQL, Spark Streaming and with HIVE, Hadoop, Kafka, YARN.
  • Good Experience in Big data cloud platform, preferably Azure.
  • Databricks and coding with notebooks.
  • Hands-on experience with data orchestration tools such as Airflow, Apache NiFi
  • Processing and manipulating data using SQL and Python code.
  • Professional experience implementing data ingestion pipelines using Data Factory.
  • Building and optimizing data pipelines, architectures, and data sets.
  • Working knowledge of queueing, stream processing, and highly scalable data stores
  • User training, customer support, and coordination with cross-functional teams.
  • Writing database-heavy services or APIs.
  • Strong understanding of structuring code for testability.
  • Should be able to perform estimations, lead the team technically, Monitoring & Tracking of activities.
  • Should be able to coordinate and communicate well with customer.


Good to have:

  • Professional Data Engineer Certifications in Data Bricks.
  • Familiarity with GIT workflow, CI/CD Pipelines.
  • Working Knowledge of Azure DevOps environment.
  • Experience of Agile practices & methodologies.
  • Experience on Migration projects (Scala/PySpark).
  • Flexibility to work in both support and development projects


Total Experience Expected: 04-06 years

Qualifications


  • Education: B.E./ B.Tech./ MCA

Additional Information

At our organization, we are committed to fighting against all forms of discrimination. We foster a work environment that is inclusive and respectful of all differences.

All of our positions are open to people with disabilities.

Top Skills

Python
Scala
The Company
Chennai, Tamil Nadu
49,329 Employees
On-site Workplace

What We Do

Sopra Steria, a major Tech player in Europe with 56,000 employees in nearly 30 countries, is recognised for its consulting, digital services and software development. It helps its clients drive their digital transformation and obtain tangible and sustainable benefits. The Group provides end-to-end solutions to make large companies and organisations more competitive by combining in-depth knowledge of a wide range of business sectors and innovative technologies with a fully collaborative approach. Sopra Steria places people at the heart of everything it does and is committed to putting digital to work for its clients in order to build a positive future for all. In 2023, the Group generated revenues of €5.8 billion.

The world is how we shape it

Similar Jobs

Effectv Logo Effectv

AD Operations Coordinator 1

AdTech • Digital Media • Marketing Tech
Hybrid
Chennai, Tamil Nadu, IND
2157 Employees

Bounteous Logo Bounteous

Associate Director, Delivery (Technical)

Agency • Digital Media • eCommerce • Professional Services • Software • Analytics • Consulting
Chennai, Tamil Nadu, IND
5000 Employees

FourKites Logo FourKites

Senior Director/ VP , Cloud Operations

Artificial Intelligence • Big Data • Logistics • Software • Transportation
Easy Apply
Chennai, Tamil Nadu, IND
550 Employees

FourKites Logo FourKites

Operations Associate

Artificial Intelligence • Big Data • Logistics • Software • Transportation
Easy Apply
Chennai, Tamil Nadu, IND
550 Employees

Similar Companies Hiring

Bounteous Thumbnail
Software • Professional Services • eCommerce • Digital Media • Consulting • Analytics • Agency
Chicago, Illinois
5000 Employees
Toast Thumbnail
Software • Information Technology • Hospitality • Food • Fintech • Cloud
Boston, MA
4500 Employees
Intelsat Thumbnail
Software • Mobile • Internet of Things • Information Technology • Digital Media • Aerospace
McLean, VA
2100 Employees

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account