Principal Data Engineer responsible for building big-data solutions using PySpark and Databricks, writing and optimizing SQL, developing Python pipelines, and working with Hadoop/Hive, Git, Airflow/Dagster, and Azure.
Requirement 1: Principal Data Engineer
Location: Remote
Experience: 6 Years
Scadea Full Time, End Client – CX Data Labs
Exp in pyspark, writng and optimizing sql queries, exp in python, hands on exp in databricks for building big data solutions, hadoop/hive, exp with data versioning tools like GIT, Airflow , Dagster, exp with azure.
Similar Jobs
Artificial Intelligence • Healthtech • Machine Learning • Natural Language Processing • Biotech • Pharmaceutical
Perform clinical data monitoring and management activities: data review, query management, investigate logic checks, and ensure database design quality through documentation, testing, validation, and implementation of data collection tools. Act as first point of contact for CTMS issues, follow SOPs, collaborate to resolve TMF/document discrepancies, and contribute to process improvements and project work.
Top Skills:
ChatgptCtmsElectronic Documentation Management SystemsMicrosoft CopilotMicrosoft Office SuiteWeb-Based Data Management Systems
Digital Media • Information Technology • News + Entertainment
Responsible for the reliability, scalability, monitoring, performance, automation, and security of large-scale data platforms. Duties include incident response, troubleshooting data pipelines and storage, capacity planning, infrastructure deployment, disaster recovery, performance optimization, documentation, and cross-team collaboration. The role requires developing automation tools, managing cloud and distributed data systems, and supporting high-availability data products.
Top Skills:
AerospikeAnsibleAWSAws S3AzureCassandraCi/CdDockerElk StackGCPGoGrafanaHadoopHdfsJavaKafkaKubernetesMicroservicesMySQLNoSQLPostgresPrometheusPythonScalaSnowflakeSparkTerraform
Digital Media • Information Technology • News + Entertainment
Designs, implements, automates, and supports secure AWS cloud infrastructure and Kubernetes platforms. Builds CI/CD pipelines and Infrastructure as Code, integrates DevSecOps controls, manages monitoring and observability, automates processes, and supports production incidents. Collaborates with engineering, QA, MLOps, and security teams across cloud and hybrid environments while driving reliability, vulnerability remediation, governance, and operational excellence independently.
Top Skills:
Amazon CloudwatchAmazon Ec2Amazon EksAmazon S3Amazon VpcAnsibleAWSAws CloudformationAws IamAws LambdaAzure DevopsBashDatadogDevsecopsDockerGithub ActionsGitlab Ci/CdGrafanaHelmJenkinsKubernetesPrismaPrometheusPythonSnykSplunkTerraform
What you need to know about the Chennai Tech Scene
To locals, it's no secret that South India is leading the charge in big data infrastructure. While the environmental impact of data centers has long been a concern, emerging hubs like Chennai are favored by companies seeking ready access to renewable energy resources, which provide more sustainable and cost-effective solutions. As a result, Chennai, along with neighboring Bengaluru and Hyderabad, is poised for significant growth, with a projected 65 percent increase in data center capacity over the next decade.
