The Big Data Engineer will support existing applications and work on new design and implementation requirements, utilizing big data technologies and tools.
This is a remote position.
Role: Big Data Engineer/Developer
Experience: 3-6 Years in Big Data
Immediate Joiners: Preferred
Location: Remote( for now)
Job Description:
- Data Engineer required with experience working in Hadoop in technologies including Hive, Hdfs, Sqoop etc.
- Must have - Python, Pyspark, Spark, AWS/Azure, SQL.
- Candidate will perform the role of senior developer where he needs to support existing applications as well as work on new requirements from a design and implementation perspectives.
- GitLab Hadoop Hive Jenkins MAVEN Nexus
- Scala Shell Scripting YARN Oracle Scrum/Agile Spring.
- Sound communication skills - both written and verbal
- Good to have - Knowledge of Teradata, Airflow, GCP
Similar Jobs
Fintech • Financial Services
Designs and delivers scalable data architectures and end-to-end solutions. Builds ETL pipelines using AWS, Spark, Scala, and big data technologies; contributes to technical design and implementation; collaborates with business and engineering teams; supports Agile delivery; promotes engineering best practices; and ensures secure, highly available solutions meet governance and risk standards.
Top Skills:
SparkAWSHadoopHiveHugging FaceLangchainLlamaOracle SqlPl/SqlPythonScalaTeradata
Cloud • eCommerce • HR Tech • Information Technology • Analytics
Manage and support AKS-based enterprise data platform infrastructure, administer Apache NiFi, Azure Event Hub and Functions, implement CI/CD with Azure DevOps, automate infrastructure with Terraform/Bicep/ARM, monitor with Azure Monitor and Log Analytics, troubleshoot production incidents, ensure security, capacity planning, and collaborate with data engineering teams.
Top Skills:
Apache NifiArgocdArm TemplatesAzure Application GatewayAzure Application InsightsAzure Container RegistryAzure Data FactoryAzure DevopsAzure Event HubAzure FirewallAzure FunctionsAzure Kubernetes Service (Aks)Azure MonitorAzure Synapse AnalyticsBashBicepCluster AutoscalerDockerFluxGitGrafanaHpaIstioKqlKubernetesLog AnalyticsMicrosoft Defender For CloudNetwork Security GroupNginxNifi RegistryOpentelemetryPeeringPowershellPrivate EndpointsPrometheusPythonRbacTerraformVirtual NetworkVpaVpnYaml Pipelines
Software
Design, implement, and lead large-scale generative AI and ML systems (LLM fine-tuning, RAG, agents, Graph ML). Build REST APIs, deploy with Docker/Kubernetes, optimize big-data workflows, mentor engineers, and operate ML services on cloud platforms.
Top Skills:
Anomaly DetectionAWSAzureBig DataCi/CdData MiningDockerDocument DatabasesFastapiGenerative AiGCPGraph MlKnowledge BasesKubernetesLlmsNoSQLPythonPyTorchRagRestful ApisSagemakerTensorrtTriton
What you need to know about the Chennai Tech Scene
To locals, it's no secret that South India is leading the charge in big data infrastructure. While the environmental impact of data centers has long been a concern, emerging hubs like Chennai are favored by companies seeking ready access to renewable energy resources, which provide more sustainable and cost-effective solutions. As a result, Chennai, along with neighboring Bengaluru and Hyderabad, is poised for significant growth, with a projected 65 percent increase in data center capacity over the next decade.
