The Data Engineer role involves designing and deploying ETL processes, managing data integrations from GA4 into BigQuery/Snowflake, optimizing data queries, and collaborating with stakeholders on data strategies.
Orion Innovation is a premier, award-winning, global business and technology services firm. Orion delivers game-changing business transformation and product development rooted in digital strategy, experience design, and engineering, with a unique combination of agility, scale, and maturity. We work with a wide range of clients across many industries including financial services, professional services, telecommunications and media, consumer products, automotive, industrial automation, professional sports and entertainment, life sciences, ecommerce, and education.
What you'll be doing:
- Design, build, and deploy robust ETL and data management processes specifically for ingesting, transforming, and loading high-volume digital analytics data from Google Analytics 4 (GA4) into BigQuery/Snowflake.
- Develop and optimize BigQuery/Snowflake datasets, tables, and views to support various analytical needs, ensuring efficient querying and data integrity.
- Design, build, and deploy ETL job workflows with reliable error/exception handling and rollback frameworks. This includes designing and implementing data pipelines to feed data models for subsequent consumption.
- Monitoring and optimizing data processing and storage resources, with a focus on performance and cost efficiency.
- Troubleshooting and resolving data pipeline issues and performance bottlenecks, particularly those related to large-scale digital data processing.
- Write complex, customized SQL queries to manipulate data, generate automatic periodic reports, and support ad-hoc analytical requests.
- Build applications and scripts using Python to automate data processes, integrate systems, and enhance data quality.
- Develop strategies for data ingestion (from multiple sources) into data platforms, using various extract and load techniques through scripting and/or tooling, streaming, API consumption, and replication.
- Document data engineering processes, best practices, and technical specifications, especially for GA4 data models and BigQuery/Snowflake schemas.
- Work collaboratively with Business Partner Teams and Business Stakeholders during project scoping and feasibility phases as a Subject Matter Expert (SME) for concept investigation/viability and technical impact; identify and communicate technical risks and mitigations.
- Conform to agile development practices – Evolutionary design, refactoring, continuous integration/delivery, test-driven development.
- Provide production support for data load jobs, ensuring the continuous flow of critical digital data. Actively monitor and resolve user support issues, working closely with your functional squad and other squads.
- Maintain or upgrade existing data applications and pipelines.
- Clearly communicate technical terms to non-technical people and help them understand why change might be required to achieve a specific goal or to complete a project.
- Attend key design meetings and provide support.
- Perform research of viable technical and/or non-technical solutions.
- Other ad hoc duties as required.
What you’ll need:
- Education: Minimum of a Bachelor's degree in Computer Science, Engineering, Mathematics, or a related technical field preferred.
- Experience: 8+ years of relevant experience in data warehousing, data modeling, and building data integration pipelines, including ingestion, integration, and ETL.
Engineer's Core Skills:
- Proven experience in working with GA4 data, including understanding its data model, event-based tracking, and integration with BigQuery.
- 6+ years of hands-on experience with Google BigQuery or Snowflake including advanced SQL techniques (CTEs, window functions, aggregate functions), schema design, and performance optimization.
- 6+ years of hands-on experience with advanced SQL techniques (CTEs, window functions, aggregate functions).
- 5+ years of strong experience in Python programming (object-oriented/functional programming, Pandas, PySpark).
- 5+ years working with cloud platforms.
- Experience designing star schemas, analyzing data warehouses, and applying data warehouse methodologies, particularly in the context of digital analytics data.
- Essential experience in loading data into warehouse/ODS environments from diverse sources and formats. Design strategies for new data ingestion requests, including logical and physical data modeling.
- Hands-on experience with version control and CI/CD pipelines for data engineering workflows.
- Proven ability to perform unit testing (SQL scripts, ETL modules), integration testing, and performance/load/stress testing.
Tools & Technologies:
- Cloud Platforms: GCP (BigQuery, DataProc, DataFlow, CloudRun, Cloud Functions, Pub/Sub, etc.) or equivalent cloud platforms like AWS or Azure.
- Data Platforms: Snowflake.
- ETL/Integration Tools: AWS Glue, Fivetran.
- Orchestration/Transformation Tools: Alteryx or DBT (data build tool).
- Version Control: GitHub or similar repositories.
- BI Tools (Bonus): Looker or Power BI tools for data consumption.
- Other Web Analytics (Bonus): Experience with other web analytics platforms or marketing data sources.
- Infrastructure as Code (Bonus): Terraform.
Orion is an equal opportunity employer, and all qualified applicants will receive consideration for employment without regard to race, color, creed, religion, sex, sexual orientation, gender identity or expression, pregnancy, age, national origin, citizenship status, disability status, genetic information, protected veteran status, or any other characteristic protected by law.
Candidate Privacy Policy
Orion Systems Integrators, LLC and its subsidiaries and its affiliates (collectively, “Orion,” “we” or “us”) are committed to protecting your privacy. This Candidate Privacy Policy (orioninc.com) (“Notice”) explains:
- What information we collect during our application and recruitment process and why we collect it;
- How we handle that information; and
- How to access and update that information.
Your use of Orion services is governed by any applicable terms in this notice and our general Privacy Policy.
Top Skills
Alteryx
Aws Glue
BigQuery
Cloud Functions
Cloudrun
Dataflow
Dataproc
Dbt
Fivetran
GCP
Git
Google Analytics 4
Looker
Power BI
Pub/Sub
Python
Snowflake
SQL
Terraform
Orion Innovation Chennai, Tamil Nadu, IND Office
Ambit IT Park, Ambit Park Road Ambattur Industrial Estate, Chennai, India, 600 058
Similar Jobs
Other • Security
The IT ServiceMax Data Developer will design and manage data integration processes, build data pipelines, ensure data quality, and collaborate with teams to enhance field service operations through effective data management.
Top Skills:
Dm AmpHadoopInformaticaMySQLSalesforceServicemaxSnowflakeSparkSQLSQL Server
Artificial Intelligence • Big Data • Cloud • Machine Learning • Software • Database • Analytics
Design, build, and launch data models and pipelines, implement data governance, optimize data ingestion, and align with business needs while collaborating with stakeholders.
Top Skills:
Apache AirflowAWSAzureDbtGCPPythonSnowflakeSQL
AdTech • Sales • Automation
Lead the migration of data analytics to Looker, design data models, develop dashboards, optimize SQL, and ensure data governance.
Top Skills:
AWSAzureBigQueryDbtGCPGitLookerLookmlRedshiftSnowflakeSQL
What you need to know about the Chennai Tech Scene
To locals, it's no secret that South India is leading the charge in big data infrastructure. While the environmental impact of data centers has long been a concern, emerging hubs like Chennai are favored by companies seeking ready access to renewable energy resources, which provide more sustainable and cost-effective solutions. As a result, Chennai, along with neighboring Bengaluru and Hyderabad, is poised for significant growth, with a projected 65 percent increase in data center capacity over the next decade.
