As a Senior Data Engineer at 3Pillar, you will leverage advanced technologies to enhance data-driven decision-making for clients. Your expertise will involve developing data analytics solutions, managing data pipelines, and utilizing cloud data warehouses. You will also be responsible for guiding fellow data engineers and collaborating within a dynamic team.
At 3Pillar, our focus is on leveraging cutting-edge technologies that revolutionize industries by enabling data driven decision making. As a Senior Data Engineer, you will hold a crucial position within our dynamic team, actively contributing to thrilling projects that reshape data analytics for our clients, providing them with a competitive advantage in their respective industries.
Minimum Qualifications
- Demonstrated expertise with a minimum of 5+ years of experience as data engineer or similar role
- Advanced SQL skills and experience with relational databases and database design.
- Experience working with cloud Data Warehouse solutions (e.g., Snowflake, Redshift, BigQuery, Azure Synapse, etc.).
- Strong Python skills with hands-on experience on Pandas, NumPy and other data related libraries
- Experience with Big Data technologies like Spark, Map Reduce, Hadoop, Hive etc
- Proficiencient in data pipeline and workflow management tools e.g. Airflow
- Experience with data visualization tools like PowerBI, Tableau, AWS QuickSight etc
- Experience with the AWS/Azure/GCP data engineering services
- Knowledge of AWS services viz., S3, Lambda, EMR, GLUE ETL, Athena, RDS, Redshift, EC2, IAM
- OR
- Knowledge of Azure services viz., ADF, Azure Synapse Analytics, ADLS Gen2, Azure SQL DB
- Very good exposure of working on Data Lakes & Data Warehouses solutions
- Excellent problem-solving, communication, and organizational skills.
- Proven ability to work independently and with a team.
- Ability to guide other data engineers.
Top Skills
Python
SQL
Similar Jobs
As a Senior AWS Data Engineer, you will develop and maintain data pipelines, create data models, and orchestrate workflows using tools like Amazon EMR, AWS Glue, Airflow, and SQL. Collaborating to understand data needs, you will optimize existing systems and document solutions for varied audiences while staying updated with new AWS technologies.
The Senior Data Scientist will collaborate with cross-functional teams to apply technical expertise in quantitative analysis and experimentation to develop business strategies. They will create insights to enhance products, measure success through metrics, and influence strategy decisions based on data-driven recommendations.
This role requires analyzing data, documenting business processes, and participating in solution design and change management activities. The individual will collaborate on implementing solutions and track progress while managing status reporting. Strong interpersonal skills and banking domain knowledge are essential.
What you need to know about the Chennai Tech Scene
To locals, it's no secret that South India is leading the charge in big data infrastructure. While the environmental impact of data centers has long been a concern, emerging hubs like Chennai are favored by companies seeking ready access to renewable energy resources, which provide more sustainable and cost-effective solutions. As a result, Chennai, along with neighboring Bengaluru and Hyderabad, is poised for significant growth, with a projected 65 percent increase in data center capacity over the next decade.
