The Bigdata Engineer will design and develop ETL/Hadoop and analytics components, ensuring performance optimization and adherence to architectural standards, while managing distributed computing tasks and data ingestion workflows.
This is a remote position.
Responsibilities:
- As a developer, possess excellent Knowledge of distributed computing architecture, core hadoop component (HDFS, Spark, Yarn, Map-Reduce, H base, HIVE, Impala) and related technologies.
- Technical Design and development of ETL/Hadoop and Analytics services /components
- Contribute in end to end architecture and process flow
- Understand Business requirement and publish reusable designs
- Result oriented approach with ability to provide apt solutions.
- Proficient in performance improvement & fine-tuning ETL and Hadoop implementations
- Conduct code reviews across projects. Takes responsibility for ensuring that build and code adhere to architectural and quality standards and policies.
- Can work independently with minimum supervision.
- Strong analytical and problem solving skills
- Experience/Exposure to SQL, advanced SQL skills
Requirements
Skills Set:
- Strong understanding of distributed computing architecture, core hadoop component (HDFS, Spark, Yarn, Map-Reuduce, H base, HIVE, Impala) and related technologies.
- Hands on experience with batch data ingestion (Sqoop)
- Expert level understanding of relational data structure and RDBMS as well as NoSQL databases (Cassandra, MongoDB, Elasticsearch)
- Experience with automation/Scheduling of workflows/jobs (via shell-scripting, Tivoli)
- Solid Grasp of data storage formats (Parquet, Avro, HBase, Cassandra)
- Understanding of Agile methodologies as well as SDLC life-cycles and processes.
- Strong Understanding of Data warehousing and lakes
Similar Jobs
Fintech • Professional Services • Consulting • Energy • Financial Services • Cybersecurity • Generative AI
Lead business analysis for Risk Weighted Assets (RWA) and Counterparty Credit Risk (CCR) initiatives. Translate Basel III/IV and SA-CCR/IMM/EAD regulatory requirements into BRDs, FRDs, user stories, process flows and data mappings. Liaise with Market Risk, CCR, Regulatory Reporting, Finance, Quant and Technology teams to support capital calculations, RWA optimization and regulatory reporting for transformation programs.
Fintech • Professional Services • Consulting • Energy • Financial Services • Cybersecurity • Generative AI
Support regulatory liquidity reporting by applying predefined rules and mappings, performing variance analysis and reconciliations, documenting business/functional requirements, coordinating BA/IT/testing streams, and contributing to governance, controls and audit readiness within regulatory change programmes.
Fintech • Professional Services • Consulting • Energy • Financial Services • Cybersecurity • Generative AI
Apply predefined regulatory rules and mappings for regulatory submissions, perform variance analysis and reconciliations, and conduct regulatory impact assessments. Document business and functional requirements, support project governance and controls, and assist audit readiness and remediation. Familiarity with collateral, intraday liquidity, and reporting remediation within regulatory change programs is expected.
What you need to know about the Chennai Tech Scene
To locals, it's no secret that South India is leading the charge in big data infrastructure. While the environmental impact of data centers has long been a concern, emerging hubs like Chennai are favored by companies seeking ready access to renewable energy resources, which provide more sustainable and cost-effective solutions. As a result, Chennai, along with neighboring Bengaluru and Hyderabad, is poised for significant growth, with a projected 65 percent increase in data center capacity over the next decade.
