Design, develop, and support scalable ETL pipelines and large-scale data platforms using Python, PySpark, SSIS, and SQL Server. Optimize database performance, build data models, troubleshoot production incidents, and implement data-quality controls. Support Hadoop-based processing, enterprise scheduling, CI/CD, and secure development practices. Collaborate across engineering, infrastructure, and business teams while working independently on complex assignments and mentoring other developers.
We are seeking a highly skilled and motivated Senior Data Platform Engineer, AVP (Python/PySpark, Microsoft SSIS, SQL Server, ETL & Database Development) to design, develop, enhance, and support robust, scalable, and high-performance data processing solutions.
The ideal candidate will combine strong hands-on expertise in Python/PySpark, SQL Server, ETL/data pipelines (Microsoft SSIS ), and large-scale data platforms, with experience troubleshooting production data applications, optimizing database performance, and working across enterprise data environments.
The candidate will be working independently on complex assignments while collaborating with engineering , infrastructure, and business teams.
Responsibilities
- Data Pipeline & ETL Development: Design, develop, test, and support scalable ETL/data-processing pipelines using Python/PySpark, Microsoft BI/SSIS, SQL, and related technologies.
- SQL Server Development: Develop and optimize complex SQL queries, stored procedures, functions, data transformations, and database objects for high-volume applications.
- Database Performance: Identify and resolve query and database performance issues through execution-plan analysis, indexing, query optimization, statistics, and appropriate database design techniques.
- Large-Scale Data Processing: Develop and support solutions processing large data volumes across SQL Server and distributed data platforms.
- Data Architecture & Modeling: Apply relational, dimensional, and data-warehouse modeling principles to build scalable and maintainable data solutions.
- Production Support & Troubleshooting: Analyze failed jobs, data-quality issues, performance problems, and production incidents; perform root-cause analysis and implement sustainable resolutions.
- Job Scheduling & Automation: Develop and support enterprise batch-processing workflows using scheduling/orchestration platforms such as AutoSys.
- Big Data Platforms: Work with Hadoop ecosystem technologies including HDFS, Hive and related distributed-processing technologies.
- Data Quality & Reconciliation: Implement validation, reconciliation, exception handling, logging, monitoring, and data-quality controls within data pipelines.
- Code Quality: Participate in code reviews and enforce development standards for SQL, Python/PySpark, ETL, and data-processing components.
- DevOps: Use Git-based source control and participate in CI/CD processes; work with container technologies such as Docker/Kubernetes where applicable.
- Security: Follow secure data-development practices, access-control requirements, encryption standards, and organizational security policies.
- Technical Collaboration: Work closely with application developers, infrastructure teams, business teams, and other technology groups to design and deliver solutions.
- Continuous Improvement & AI: Evaluate new technologies and leverage approved AI-assisted development tools to improve code quality, troubleshooting, refactoring, and developer productivity.
- Appropriately assess risk when business decisions are made, demonstrating consideration for the firm’s reputation and safeguarding the firm, its clients, and assets while complying with applicable laws, rules, regulations, and policies.
Recommended Qualifications
- 8+ years of relevant experience in data engineering, application development, database development, or related technology roles.
- Strong hands-on experience with Python and PySpark for large-scale data processing and ETL development.
- Strong proficiency in SQL/T-SQL and relational database technologies, preferably Microsoft SQL Server.
- Experience developing and supporting complex SQL queries, stored procedures, database objects, and high-volume data-processing solutions.
- Strong understanding of SQL Server performance tuning, including query execution plans, indexing, statistics, joins, locking/blocking, and query optimization.
- Experience with Microsoft BI/SSIS or comparable enterprise ETL technologies.
- Experience working with large datasets and distributed data platforms such as Hadoop/HDFS/Hive.
- Strong understanding of ETL architecture, data warehousing, dimensional modeling, and relational/3NF modeling.
- Experience implementing data-quality checks, reconciliation, audit controls, exception handling, and production monitoring.
- Experience with enterprise scheduling/orchestration tools such as AutoSys.
- Experience with Git, CI/CD, and modern development practices.
- Familiarity with Docker/Kubernetes is preferred.
- Familiarity with NoSQL technologies such as MongoDB is beneficial.
- Strong analytical, troubleshooting, communication, and problem-solving skills.
- Ability to work independently on complex technical assignments while collaborating effectively across teams.
- Experience mentoring or providing technical guidance to other developers
Education
- Bachelor’s degree/University degree or equivalent experience.
------------------------------------------------------
Job Family Group: Technology------------------------------------------------------
Job Family:Applications Development------------------------------------------------------
Time Type:Full time------------------------------------------------------
Most Relevant Skills Please see the requirements listed above.------------------------------------------------------
Other Relevant Skills For complementary skills, please see above and/or contact the recruiter.------------------------------------------------------
Citi is an equal opportunity employer, and qualified candidates will receive consideration without regard to their race, color, religion, sex, sexual orientation, gender identity, national origin, disability, status as a protected veteran, or any other characteristic protected by law.
If you are a person with a disability and need a reasonable accommodation to use our search tools and/or apply for a career opportunity review Accessibility at Citi.
View Citi’s EEO Policy Statement and the Know Your Rights poster.
Citi Chennai, Tamil Nadu, IND Office
C P Ramaswamy Road, Chennai, Tamil Nadu, India, 600018
Similar Jobs
Aerospace • Artificial Intelligence • Cloud • Machine Learning • Software • Cybersecurity • Defense
Lead aircraft engine and power-system certification and qualification activities. Prepare and review certification plans, reports, and related artifacts; coordinate with FAA, EASA, and other aviation regulators; and support international validation efforts. Improve certification processes by standardizing workflows, reducing cycle time and cost, and applying automation and artificial intelligence. Provide technical guidance, coaching, and support to aerospace programs.
Top Skills:
Artificial Intelligence (Ai)DoorsDoors NgEasaFaaModel-Based Systems Engineering (Mbse)RpeTechnical Standard Order (Tso)
Aerospace • Artificial Intelligence • Cloud • Machine Learning • Software • Cybersecurity • Defense
Provides technical leadership for aerospace navigation database production and customer support. Engages internal and external stakeholders to resolve technical issues, delivers tailored solutions, and supports production lifecycle activities. Develops quality assurance processes aligned with DO-200 standards, conducts reviews and audits, coordinates with source suppliers, improves workflows, and mentors team members. Requires extensive aerospace and flight operations knowledge, including FMS, ARINC 424, navigation procedures, flight planning, and aeronautical charts.
Top Skills:
Aerospace Navigation DatabasesArinc 424Do-200Flight Management Systems (Fms)Honeywell Navigation Database Formats
Aerospace • Artificial Intelligence • Cloud • Machine Learning • Software • Cybersecurity • Defense
Designs, develops, integrates, and validates aerospace engine, power, avionics, and actuation control systems. Responsibilities include requirements engineering, system architecture, control-system analysis, MATLAB/Simulink modeling, embedded C development, sensor and actuator integration, SIL/HIL testing, verification, validation, and aerospace certification compliance. The role supports model-based development and real-time embedded applications using aerospace standards and communication protocols.
Top Skills:
AfdxArinc 429Arm CortexArp4754AArp4761C++CanDo-160Do-178CDo-254DoorsDoors NgEmbedded CHilMatlabMbsePowerpcRs232Rs422/485RtosSilSimulinkStateflowSysmlTi C2000
What you need to know about the Chennai Tech Scene
To locals, it's no secret that South India is leading the charge in big data infrastructure. While the environmental impact of data centers has long been a concern, emerging hubs like Chennai are favored by companies seeking ready access to renewable energy resources, which provide more sustainable and cost-effective solutions. As a result, Chennai, along with neighboring Bengaluru and Hyderabad, is poised for significant growth, with a projected 65 percent increase in data center capacity over the next decade.
