Deputy Manager - Data Engineering

About Company

Job Description

Roles and Responsibilities

• Architect, build, and optimize scalable data pipelines using Python, SQL, and distributed processing frameworks such as Spark.
• Design and maintain robust ETL/ELT workflows to integrate structured and semi-structured data from multiple internal and external systems.
• Ensure data quality, lineage, validation, and monitoring across all ingestion and transformation layers.
• Partner with analytics and product teams to operationalize analytical models, KPI dashboards, and metric frameworks.
• Build reusable datasets, analytics marts, and query-optimized schemas to support diverse use cases including reporting, forecasting, personalization, and operational automation.
• Translate raw data into accessible, well-governed, analysis-ready formats.
• Develop SOPs, quality checks, and review workflows to ensure accurate, consistent, and reliable data delivery
• Establish coding standards, documentation requirements, and review mechanisms to ensure high engineering rigor
• Implement monitoring and error-handling frameworks to improve system reliability and reduce failures.
• Lead and mentor a team of data engineers, analysts, and technical contributors.
• Define project scope, timelines, and milestones; drive execution using Agile methodologies or equivalent project management approaches.
• Coordinate with cross-functional teams (engineering, product, business) to ensure alignment and timely delivery of data initiatives.
• Oversee vendor partners, outsourced contributors, or external data service providers where relevant.
• Work closely with senior stakeholders to understand business requirements, translate them into scalable technical solutions, and communicate progress effectively.
• Provide data-driven insights and recommendations to support operational efficiency and strategic decision-making.
• Lead development of data-driven systems such as rules-based or ML-powered recommendation engines, user segmentation pipelines, scoring models, or personalization algorithms.
• Implement frameworks that track user behavior, performance metrics, or system outputs to continuously enhance relevance and accuracy.
• Oversee creation of dashboards and automated reporting solutions using tools like Plotly or equivalent BI tools.
• Collaborate with analytics teams to identify trends, anomalies, or optimization opportunities across operational and customer-facing data.

Required Skills & Experience

• 7–12 years of experience in data engineering, data platforms, or large-scale data projects.
• Strong programming skills in Python, SQL, and experience with Pandas, NumPy, Spark or similar distributed systems.
• Proven track record of building end-to-end data pipelines and delivering production-grade data solutions.
• Experience designing data models, quality frameworks, governance standards, and automated monitoring.
• Demonstrated ability to lead technical teams, manage projects, and collaborate across cross-functional units.
• Experience using data to derive insights, optimize systems, and improve business outcomes.
• Ability to work in fast-paced environments and manage multiple priorities simultaneously
• Excellent communication, stakeholder management, and documentation abilities
• Knowledge in using leading cloud platforms preferably Azure.
• Ability to work in a team in an agile setting, familiarity with JIRA and clear understanding of how Git works