Senior Data Engineer, Data Insights

Senior Data Engineer, Data Insights

Who we are

As a Senior Software Engineer on the Data Insights team, you will be responsible for developing, owning, and scaling a monitoring and insights pipeline ingesting data from every data platform owned by Wayfair. Your contributions will ensure the continued growth, health, and availability of a dataset that powers every infrastructure decision at Wayfair. This pipeline integrates streaming technology, batch processing, and ML to provide forecasting, anomaly detection, cost insights, and value reporting.

  • 70 kafka clusters and over 30k kafka topics across the world
  • Thousands of cloud assets, including database tables, storage buckets
  • Thousands of dashboards, applications, and user tools
  • Millions of user and service interactions with data systems

Candidates that are a fit for this profile are familiar with data engineering or business intelligence, are experienced with Python, systems and configuration as code, and are passionate about building highly resilient, scalable solutions, and would be interested in the following specifics for this team:

What You’ll Do

  • Work with partner teams to ingest data from their platforms into a centralized monitoring platform
  • Design and implement self-service offerings for frictionless onboarding based on stakeholder requirements
  • Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability
  • Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources
  • Design, build, and expose value added services on top of this pipeline, such as cost attribution, anomaly detection, automated cost-reduction tools, and more
  • Architect and implement a monitoring and alerting approach to enable exposure of Service Level Indicators (SLIs) and exposure/adherence to Service Level Objectives (SLOs) 

What You’ll Need

  • 5+ years of experience in Data Engineering role
  • Applied experience with Python and at least one OLAP database (Redshift, bigquery, snowflake or equivalent) at scale
  • Familiarity with Puppet, Chef, Terraform (or similar technologies)
  • Experience building and optimizing ‘big data’ data pipelines, architectures and data sets.

About Wayfair Inc.

Wayfair is one of the world’s largest online destinations for the home. Whether you work in our global headquarters in Boston or Berlin, or in our warehouses or offices throughout the world, we’re reinventing the way people shop for their homes. Through our commitment to industry-leading technology and creative problem-solving, we are confident that Wayfair will be home to the most rewarding work of your career. If you’re looking for rapid growth, constant learning, and dynamic challenges, then you’ll find that amazing career opportunities are knocking.

No matter who you are, Wayfair is a place you can call home. We’re a community of innovators, risk-takers, and trailblazers who celebrate our differences, and know that our unique perspectives make us stronger, smarter, and well-positioned for success. We value and rely on the collective voices of our employees, customers, community, and suppliers to help guide us as we build a better Wayfair – and world – for all. Every voice, every perspective matters. That’s why we’re proud to be an equal opportunity employer. We do not discriminate on the basis of race, color, ethnicity, ancestry, religion, sex, national origin, sexual orientation, age, citizenship status, marital status, disability, gender identity, gender expression, veteran status, or genetic information.

Logos/outerjoin logo full

Outer Join is the premier job board for remote jobs in data science, analytics, and engineering.