1. Technical Leadership:

Provide technical leadership and mentorship to the data engineering team, promoting best practices, code quality, and professional growth.

Act as a subject matter expert in data engineering, staying updated with industry trends and advancements, and sharing knowledge with the team.

2. Architectural Design and Development:

Lead the design and implementation of complex data architectures, ensuring scalability, reliability, and performance to meet business objectives and analytical needs.

Collaborate with stakeholders while demonstrating strong business context to define data architecture standards and guidelines.

3. Data Pipeline Development and Optimization:

Design and develop efficient and scalable data pipelines, ensuring data ingestion, processing, and transformation align with business requirements.

Optimize data pipelines for performance, resilience, and cost-effectiveness.

4. Database Management and Optimization:

Oversee the design and management of databases, data warehouses, and data lakes, optimizing for performance, security, and scalability.

Implement and optimize ETL processes to maintain data integrity and efficiency in data storage solutions.

5. Technical Ownership and Execution:

Lead and manage complex data engineering projects, ensuring successful project delivery within specified timelines and budget.

Collaborate with cross-functional teams to align project goals and priorities, providing technical expertise to meet project requirements.

6. Data Quality and Governance:

Establish and enforce data quality standards, governance policies, and compliance with relevant regulations, ensuring data accuracy, consistency, and security.

Implement data quality monitoring and improvement processes.

7. Collaboration and Stakeholder Management:

Collaborate with various stakeholders to understand their data needs and provide data solutions that drive business value.

Communicate effectively with both technical and non-technical stakeholders, translating technical concepts into actionable insights.

8.Common Software Engineering Requirements

You lead the end-to-end delivery of complex software applications, ensuring adherence to best practices and high overall quality standards.

You create frameworks and abstractions that lessen the workload of other engineers. Your documentation and tutorials enable other teams to leverage your work independently. You identify and resolve improvements in test coverage for end-to-end business flows across multiple systems (and teams). You introduce new tools, techniques, and coding patterns to other engineers, some of which may be developed by yourself, and drive this change to other teams.

You drive technical decision-making, contribute to, and sometimes lead, architectural discussions, and propose innovative solutions to address complex technical and/or systemic architectural challenges. You collaborate across teams and build systems to address architectural gaps.

You take ownership of critical projects and initiatives, providing project leadership, and ensuring successful delivery through effective project management and communication. You engage with stakeholders across the group, understanding their needs and working through the complexity and conflicting goals.

You have the ability to provide a clear and concise explanation of business strategy, technical concepts, designs or implementation to a non-technical audience. You collaborate closely with product managers, designers, and fellow engineers to gather and translate business requirements into efficient and scalable software solutions. You have a track record of leading cross-team efforts that deliver meaningful results across engineering, product, and/or design. You work through technical, organizational and scheduling complexity to align teams and deliver results in a positive way for the people and teams involved. Your interactions build trust and speed at scale into our organization.

You are the technical thought leader of the team, and provide necessary guidance and mentorship to senior team members whenever required.

You play the key technical counterpart to an engineering manager with a larger team; you can provide effective feedback to engineering managers at all levels to identify and help address gaps in their teams.

You successfully advocate for technologies and processes that make multiple teams more efficient.

You conduct rigorous code reviews, offering valuable feedback to maintain code quality, performance, and maintainability.

You continuously identify opportunities for process improvements and efficiency gains within the software development lifecycle.

You stay informed about emerging technologies, industry trends, and best practices, and proactively introduce relevant advancements to enhance our products and services.

Qualifications:

• Bachelor's or Master's degree in Computer Science, Data Science, or a related field.

• 10+ years of experience in data engineering, with a focus on data architecture, ETL,

  and database management.

• Ability to identify opportunities for optimisation and efficiency gains within the data pipelines/architecture/infrastructure.

• Work like a technical thought leader of the team and provide necessary guidance and mentorship to senior team members.

• Ability to take ownership of critical projects and initiatives, providing project leadership, and ensuring successful delivery through effective project management and communication.

• Involve in rigorous code and data quality reviews, offering valuable feedback to maintain best practices, quality, performance, and maintainability.

• Lead the team in designing and building complex end-to-end data pipelines.

• Proficiency in programming languages like Python/PySpark and Java /Scala

• Expertise in big data technologies such as Hadoop, Spark, Kafka, etc.

• In-depth knowledge of SQL and experience with various database technologies (e.g., PostgreSQL, MySQL, NoSQL databases).

• Guide the team in orchestrating and designing job schedules using the CICD tools like Jenkins and Airflow.

• Ability to work in an Agile environment (Scrum, Lean, Kanban, etc)

• Familiarity with cloud platforms (e.g., AWS, Azure, GCP) and their data services

  (e.g., AWS Redshift, S3, Azure SQL Data Warehouse).

• Strong leadership, problem-solving, and decision-making skills.

• Excellent communication and collaboration abilities.