Orion Innovation Logo

Orion Innovation

Senior Data Engineer

Job Posted 7 Days Ago Posted 7 Days Ago
Be an Early Applicant
India
Senior level
India
Senior level
The Senior Data Engineer will develop and maintain data infrastructure, design data pipelines, ensure data quality, and collaborate with stakeholders to meet data needs.
The summary above was generated by AI

Orion Innovation is a premier, award-winning, global business and technology services firm.  Orion delivers game-changing business transformation and product development rooted in digital strategy, experience design, and engineering, with a unique combination of agility, scale, and maturity.  We work with a wide range of clients across many industries including financial services, professional services, telecommunications and media, consumer products, automotive, industrial automation, professional sports and entertainment, life sciences, ecommerce, and education.

Responsibilities:

  •  Build and maintain the infrastructure for data generation, collection, storage, and processing.
  • Design, build, and maintain scalable data pipelines to support data flows from various sources to data warehouses and analytics platforms.
  • Develop and manage ETL (Extract, Transform, Load) processes to ensure data is accurately transformed and loaded into the target systems.
  • Design and optimize databases, ensuring performance, security, and scalability of data storage solutions.
  • Integrate data from various internal and external sources into unified data systems for analysis.
  • Work with big data technologies (e.g., Hadoop, Spark) to process and manage large volumes of structured and unstructured data.
  • Implement and manage cloud-based data solutions using Azure & Fabric platforms.
  • Ensure data quality by developing validation processes and monitoring for anomalies and inconsistencies.
  • Work closely with data scientists, analysts, and other stakeholders to meet their data needs and ensure smooth data operations.
  • Automate repetitive data processes and workflows to improve efficiency and reduce manual effort.
  • Implement and enforce data security protocols, ensuring compliance with industry standards and regulations.
  • Optimize data queries and system performance to handle large data sets efficiently.
  • Create and maintain clear documentation of data pipelines, infrastructure, and processes for transparency and training.
  • Set up monitoring tools to ensure data systems are functioning smoothly and troubleshoot any issues that arise.
  • Stay updated with emerging trends and tools in data engineering and continuously improve data infrastructure.

Qualifications:


  • Azure Solution Architect certification preferred
  • Microsoft Fabric Analytics Engineer Associate certification preferred
  • 5+ years of architecture experience in the technology Operations/Development using Azure technologies.
  • Strong experience in Python and PySpark required
  • Strong understanding and experience in building lake houses, data lakes and data warehouses
  • Strong experience in Microsoft Fabric technologies.
  • Good understanding of the Scrum Agile methodology
  • Strong experience with Azure Cloud technologies
  • Solid knowledge of SQL, and non-relational (NoSQL) databases
  • Solid knowledge of networking, firewalls, load balancers etc.
  • Exceptional communication skills and the ability to communicate appropriately with technical teams
  • Familiarity with at least one of the following code build/deploy tools Azure DevOps or GitHub Actions


Orion is an equal opportunity employer, and all qualified applicants will receive consideration for employment without regard to race, color, creed, religion, sex, sexual orientation, gender identity or expression, pregnancy, age, national origin, citizenship status, disability status, genetic information, protected veteran status, or any other characteristic protected by law.

Candidate Privacy Policy

Orion Systems Integrators, LLC and its subsidiaries and its affiliates (collectively, “Orion,” “we” or “us”) are committed to protecting your privacy. This Candidate Privacy Policy (orioninc.com) (“Notice”) explains:

  • What information we collect during our application and recruitment process and why we collect it;
  • How we handle that information; and
  • How to access and update that information.

Your use of Orion services is governed by any applicable terms in this notice and our general Privacy Policy.


Top Skills

Azure
Azure Devops
Github Actions
Hadoop
Microsoft Fabric
NoSQL
Pyspark
Python
Spark
SQL

Orion Innovation Chennai, Tamil Nadu, IND Office

Ambit IT Park, Ambit Park Road Ambattur Industrial Estate, Chennai, India, 600 058

Similar Jobs

4 Days Ago
Hybrid
Bengaluru, Bengaluru Urban, Karnataka, IND
Senior level
Senior level
Healthtech • Software • Analytics • Biotech • Pharmaceutical • Manufacturing
As a Senior Data Engineer, you will build and maintain data systems, develop scalable data models and pipelines using AWS, and collaborate with various teams to improve data quality and support digital products.
Top Skills: AWSDatabricksDmsGraphQLLambdaNo-SqlPysparkPythonSQLSqsStep FunctionsStreaming Technologies
8 Days Ago
In-Office or Remote
Bengaluru, Karnataka, IND
Senior level
Senior level
Cloud • Information Technology • Productivity • Security • Software • App development • Automation
Responsible for designing data solutions, enhancing quality, and building scalable datasets while collaborating with stakeholders to meet business needs.
Top Skills: AirflowAWSDatabricksPythonSparkSparksqlSQL
12 Days Ago
Hybrid
Bengaluru, Bengaluru Urban, Karnataka, IND
Senior level
Senior level
Healthtech • Software • Analytics • Biotech • Pharmaceutical • Manufacturing
As a Senior Data Engineer, you will lead complex data structures and pipelines, ensuring data integrity and utilizing advanced techniques for data integration and modeling.
Top Skills: AirflowAnsibleAthenaAws RedshiftBigQueryChefDatabricksEltETLGitlabKafkaRdsSagemakerSparkTerraformTidal

What you need to know about the Chennai Tech Scene

To locals, it's no secret that South India is leading the charge in big data infrastructure. While the environmental impact of data centers has long been a concern, emerging hubs like Chennai are favored by companies seeking ready access to renewable energy resources, which provide more sustainable and cost-effective solutions. As a result, Chennai, along with neighboring Bengaluru and Hyderabad, is poised for significant growth, with a projected 65 percent increase in data center capacity over the next decade.
By clicking Apply you agree to share your profile information with the hiring company.

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account