We are looking for a Data Engineer with skills in Python, SQL, and Azure technologies to develop Big Data applications. Responsibilities include working with Spark and Docker to handle large scale data processing and analytics.
This is a remote position.
- Must have - Python, SQL, Spark, Databricks, Azure Data Factory, Azure Synapse, Docker
- Good to have - Knowledge of Teradata, Airflow, AWS, GC
- experience in developing Big Data applications
- Experience in large-scale Big Data & Analytics applications
- Hands on experience in using technologies like Spark, Scala,
- Experience in Azure & Docker are added advantage.
- Strong problem-solving skills, data structures and algorithms
- Experience with distributed systems handling large amounts of data.
Requirements
- Immediate Joiner
- Remote location
- Data Engineer with 2 to 6 Yrs of experience
Similar Jobs
Cloud • Information Technology • Productivity • Security • Software • App development • Automation
Lead data engineering architecture and execution for large-scale, low-latency analytics. Own technical direction, ensure operational data quality, design streaming and batch pipelines, mentor engineers, coordinate cross-functional teams, and deliver scalable solutions using Spark, Airflow, and AWS or equivalent data platforms.
Top Skills:
AirflowAthenaColumn StoresDatabricksDatabricks ApisEmrFlinkHiveKafkaKappa ArchitectureLambda ArchitectureMaster Data Management (Mdm)Microservices ArchitectureRedshiftSparkSQLStreaming Pipelines
Cloud • Information Technology • Productivity • Security • Software • App development • Automation
Lead and execute data engineering initiatives: define metrics, instrument logging, ingest and model data, build ETL/ELT pipelines, ensure data quality and governance, optimize processing performance, enable analytics/visualization, and lead projects that drive product and business strategy.
Top Skills:
Anomaly DetectionDashboardingData ModelingData QualityData VisualizationEltETLLoggingProduct AnalyticsPythonRelational DatabasesSQL
Cloud • Information Technology • Professional Services • Software
Build production-grade agentic AI systems that reason, search, and execute workflows. Develop RAG pipelines, AI agents, semantic search layers, data processing solutions, dimensional data models, and scalable AI workloads. Work with structured and unstructured data, LLMs, vector databases, cloud infrastructure, CI/CD, and ML observability. Responsibilities include fine-tuning models, prompt engineering, optimizing context windows, and deploying reliable AI applications.
Top Skills:
Ai AgentsAmazon RedshiftAws BedrockAws LambdaCi/CdClaudeEmbedding ModelsGeminiGoogle IndexingGptLangchainLlamaindexMl ObservabilityPandasPgvectorPysparkPythonRagScalaSnowflakeSQLTransformer Architectures
What you need to know about the Chennai Tech Scene
To locals, it's no secret that South India is leading the charge in big data infrastructure. While the environmental impact of data centers has long been a concern, emerging hubs like Chennai are favored by companies seeking ready access to renewable energy resources, which provide more sustainable and cost-effective solutions. As a result, Chennai, along with neighboring Bengaluru and Hyderabad, is poised for significant growth, with a projected 65 percent increase in data center capacity over the next decade.
