What you will do:

• Write production code to implement fast, reliable, scalable data pipelines in Python (and Pandas and Spark) and SQL to process billions of historical data points collected from tens of thousands of retail stores across the US
• Build out, scale up, and deploy our compute platform (running on Azure cloud infrastructure using Kubernetes) and data platform (powered by a combination of traditional databases, data warehouses, and data lakes) that power our recommendation engine and ordering system
• Implement reusable libraries and abstractions for ensuring that we can scale our services to more, bigger customers and different departments and domains
• Work closely with Afresh internal stakeholders, solution architects, and our customers to understand how to transform customer data
• Collaborate with an interdisciplinary team of experts in machine learning, data science, design, software engineering, and business operations to triage data or ETL issues that are causing anomalies in our system
• Independently analyze and find issues in the data—including those unknown to the customer themselves—and work with internal stakeholders to either clearly communicate high-priority issues to the customer or find novel workarounds to extract the signal we need from customer data
• Build out tests and health checks to verify the quality of our code and data, ensure any assumptions about the data continue to hold, and avoid future regressions in the code or data
• Monitor, analyze, and understand the data flowing through our system by adding the necessary visualizations and dashboards

What skills and experience do you need?