TaskUs Logo

TaskUs

Senior Data Engineer

Reposted 2 Months Ago
Be an Early Applicant
In-Office
Chennai, Tamil Nadu, IND
Senior level
In-Office
Chennai, Tamil Nadu, IND
Senior level
Build and maintain resilient ETL/ELT pipelines for lakehouse environments, using Python, SQL, PySpark, dbt, and orchestration tools. Develop data observability, monitoring, alerting, testing, and CI/CD practices. Optimize Spark and SQL performance, create feature-ready datasets, and implement cost-efficient hybrid compute patterns using DuckDB and distributed platforms. Manage infrastructure with Kubernetes and IaC tools while supporting streaming, cloud storage, and embedded analytics use cases.
The summary above was generated by AI

About TaskUs: TaskUs is a provider of outsourced digital services and next-generation customer experience to fast-growing technology companies, helping its clients represent, protect and grow their brands. Leveraging a cloud-based infrastructure, TaskUs serves clients in the fastest-growing sectors, including social media, e-commerce, gaming, streaming media, food delivery, ride-sharing, HiTech, FinTech, and HealthTech. 


The People First culture at TaskUs has enabled the company to expand its workforce to approximately 60,000 employees globally. Presently, we have a presence in twenty-three locations across twelve countries, which include the Philippines, India, and the United States.


It started with one ridiculously good idea to create a different breed of Business Processing Outsourcing (BPO)! We at TaskUs understand that achieving growth for our partners requires a culture of constant motion, exploring new technologies, being ready to handle any challenge at a moment’s notice, and mastering consistency in an ever-changing world.


What We Offer: At TaskUs, we prioritize our employees' well-being by offering competitive industry salaries and comprehensive benefits packages. Our commitment to a People First culture is reflected in the various departments we have established, including Total Rewards, Wellness, HR, and Diversity. We take pride in our inclusive environment and positive impact on the community. Moreover, we actively encourage internal mobility and professional growth at all stages of an employee's career within TaskUs. Join our team today and experience firsthand our dedication to supporting People First.

The Mission: As a Senior Data Engineer, you are the engine room of our data strategy. You don't just "move data"; you build resilient, self-healing systems that transform raw, into high-fidelity data products. You will take the Architect’s blueprints and turn them into production-grade code, ensuring there is clean and reliable data.

Work mode : Remote

What the Senior Data Engineer will own:

  • Pipeline Engineering: Build and maintain high-throughput ETL/ELT pipelines that ingest data from various sources into our Lakehouse.

  • Code Quality & Tooling: Drive the adoption of dbt for transformation and PySpark for heavy lifting. You will be responsible for writing modular, reusable code that follows strict CI/CD practices.

  • Observability & Reliability: Implement "Data SLAs." You will build the monitoring and alerting systems that notify the team of data drift or pipeline failures before the business notices.

  • Data Productization: Work closely with the BI team and Data Scientists to prepare "feature-ready" datasets.

  • Performance Tuning: Deep-dive into SQL and Spark query plans to optimize slow-running jobs, reducing cost and latency.

  • Local Development Advocacy: Implementing a workflow where engineers can develop and test complex SQL transformations locally using DuckDB, drastically reducing "waiting-for-cluster" time and lowering development costs.

  • Cost-Efficient Micro-Pipelines: Building specialized pipelines for specific client reporting or data-quality checks that run on single-node containers (e.g., Lambda or small ECS tasks) using DuckDB, avoiding the minimum billing cycles of larger warehouses.

  • Embedded Analytics: Exploring ways to use DuckDB as an embedded engine for internal tools or "edge" processing within the BPO’s local site offices where bandwidth to the cloud might be a constraint.

Technical Skills & Experience:

  • 5+ Years in Data Engineering: You’ve lived through the "on-call" life and know how to build systems that don't break at 3 AM.

  • The Power Trio: Expert-level proficiency in Python, SQL, and PySpark.

  • Modern Lakehouse Stack: Hands-on experience with Databricks (Delta Lake) or Snowflake. You understand the nuances of the Medallion Architecture (Bronze/Silver/Gold).

  • Transformation & Modeling: Advanced experience with dbt (Data Build Tool). You treat data models like software, including version control, testing, and documentation.

  • Orchestration: Experience with Apache Airflow or Prefect, specifically building complex, idempotent DAGs.

  • Streaming Experience: Familiarity with Kafka, Kinesis, or Spark Streaming is a huge plus—our BPO operations move in real-time.

  • Infrastructure as Code (IaC): Comfort with Terraform or CloudFormation to manage your own data infrastructure.

  • In-Process Analytics (DuckDB): Proven experience using DuckDB for high-speed local development, unit testing data transformations, or as a query engine for "small-to-medium" datasets (up to 100GB) without the overhead of a distributed cluster.

  • Hybrid Execution Patterns: Ability to identify when to use heavyweight compute (PySpark/Databricks) versus lightweight compute (DuckDB/Python) to minimize cloud costs and reduce job latency.

  • Parquet/Iceberg Interaction: Experience using DuckDB to directly query data stored in S3/Azure Blob (via Parquet or Iceberg files) for rapid ad-hoc analysis or local dashboarding.

  • Orchestration: Kubernetes (K8s). Deep understanding of Pods, Deployments, Services, ConfigMaps, and Secrets management. 

How We Partner To Protect You: TaskUs will neither solicit money from you during your application process nor require any form of payment in order to proceed with your application. Kindly ensure that you are always in communication with only authorized recruiters of TaskUs.

DEI: In TaskUs we believe that innovation and higher performance are brought by people from all walks of life. We welcome applicants of different backgrounds, demographics, and circumstances. Inclusive and equitable practices are our responsibility as a business. TaskUs is committed to providing equal access to opportunities. If you need reasonable accommodations in any part of the hiring process, please let us know.

We invite you to explore all TaskUs career opportunities and apply through the provided URL https://www.taskus.com/careers/.

Similar Jobs

5 Days Ago
In-Office
Chennai, Tamil Nadu, IND
Senior level
Senior level
Information Technology • Consulting
Designs and maintains scalable Azure Databricks data pipelines using Python, Scala, PySpark, Delta Lake, and SQL. Develops ETL/ELT workflows, optimizes processing performance, ensures data quality and governance, manages Azure Data Lake and Unity Catalog integrations, and supports cloud architecture, migration, modernization, troubleshooting, documentation, and cross-functional delivery.
Top Skills: Apache AirflowAzure Data FactoryAzure Data LakeAzure DatabricksCi/CdDelta LakeEltETLInformaticaOraclePl/SqlPysparkPythonRdbmsScalaSQLUnity Catalog
9 Days Ago
In-Office
Senior level
Senior level
Information Technology • Software • Analytics
Designs and maintains scalable Azure-based ETL/ELT pipelines using Data Factory, Databricks, Python, SQL, PySpark, and Spark. Integrates diverse data sources, develops data models and warehouse structures, implements CDC and batch processing, and optimizes queries and distributed workloads. Manages orchestration, monitoring, error handling, CI/CD, documentation, and production support using Airflow, Git, GitHub, and Azure DevOps. Collaborates with cross-functional teams to troubleshoot issues and deliver reliable data solutions.
Top Skills: Apache AirflowSparkAzure Blob StorageAzure Data FactoryAzure Data Lake Storage Gen2Azure DatabricksAzure DevopsAzure Key VaultAzure MonitorAzure Synapse AnalyticsCi/CdCsvDelta LakeGitGitJSONMicrosoft Sql ServerMySQLNumpyOraclePandasParquetPostgresPysparkPythonSQLXML
12 Days Ago
Hybrid
Senior level
Senior level
Blockchain • Database • Analytics
Designs and maintains scalable Databricks ETL/ELT pipelines using Python, PySpark, SQL, Spark, and Delta Lake. Manages Databricks infrastructure, Jobs, Workflows, clusters, access controls, and performance optimization. Implements data governance through Unity Catalog, data quality checks, lineage, metadata management, cataloging, and governed access. Integrates diverse data sources, troubleshoots pipeline issues, supports data security and compliance, and collaborates with teams on enterprise data management practices.
Top Skills: Amazon S3SparkCi/CdDatabricksDelta LakeGitPysparkPythonRest ApisSQLUnity Catalog

What you need to know about the Chennai Tech Scene

To locals, it's no secret that South India is leading the charge in big data infrastructure. While the environmental impact of data centers has long been a concern, emerging hubs like Chennai are favored by companies seeking ready access to renewable energy resources, which provide more sustainable and cost-effective solutions. As a result, Chennai, along with neighboring Bengaluru and Hyderabad, is poised for significant growth, with a projected 65 percent increase in data center capacity over the next decade.

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account