Responsibilities
• Design, build, and maintain scalable and efficient data platforms using data engineering technologies such as Glue, EMR, Athena, Redshift, Lake Formation, Apache Spark, Hive, HDFS, and Trino.
• Build/manage data pipelines, and common data-related cross-cutting concerns like data catalog, data lineage, data quality, data profiling, data discovery, metadata management
• Develop and maintain web applications using AngularJS and Python.
• Build/manage BI/Analytical dashboard reducing time to insight for the business stakeholders.
• Implement CI/CD pipelines using Terraform, Jenkins, Github actions, and Gitflow.
• Collaborate with cross-functional teams to develop and implement new features.
• Write clean, reusable, and efficient code.
• Participate in code reviews and ensure code quality.
• Develop and maintain APIs using Python and ensure API security and best practices are implemented.
• Implement SSO integration with Microsoft Azure AD using oAuth, OIDC, and SAML.
• Implement integration with AWS Cognito for user authentication and authorization.
• Ensure the application is optimized for maximum speed and scalability.
• Troubleshoot and debug issues as they arise.
• Implement DevOps best practices to ensure efficient application deployment and management.
• Collaborate with data scientists and analysts to integrate data analytics solutions with web applications.
• Stay up to date with emerging trends and technologies.
Skills
Must have
• 8+ years of experience in similar positions;
• Hands-on Experience with data engineering technologies such as AWS Glue, EMR, Athena, Redshift, Lake Formation, Apache Spark, Apache Hive, Apache Airflow, S3FS, Apache Hudi, and Trino.
• Extensive experience in building data pipelines using orchestration tools like Apache Airflow. Hands-on experience in building cross-cutting concerns like data catalog, data lineage, data quality, data profiling, data discovery, metadata management
• Proven experience as a Full Stack Developer with AngularJS and Python.
• Strong understanding of web development technologies including HTML, CSS, and JavaScript.
• Experience working with RESTful APIs and JSON. Familiarity with microservices architecture.
• Experience with core AWS technologies such as EC2, ELB, Auto Scaling, S3, EFS, Lambda, API
Gateway, Step Functions, Cloudwatch, VPC, Route 53, ACM
• Hands on experience with SQL and NoSQL databases.
• Hands experience with BI tools like Tableau, AWS QuickSight
• Experience with Git or other version control systems.
• Understanding of agile development methodologies.
• Strong problem-solving skills.
• Excellent written and verbal communication skills.
• Ability to work independently and collaboratively in a team environment.
• Experience with cloud platforms such as AWS.
• Bachelor's degree in Computer Science, Engineering, or related field
Nice to have
• Agile, Scrum framework 2+ years' experience on past projects
• Knowledge of containerization technologies like Docker or Kubernetes.
• Experience with front-end frameworks like React or Vue.js.
• Experience with DevOps and CI/CD best practices.
• Experience with Terraform and the AWS provider.
• Experience with API development, security, and best practices.
• Experience with SSO integration with Microsoft Azure AD using oAuth, OIDC, and SAML.
• Experience with integration with AWS Cognito for user authentication and authorization