Company Description
As an Etsy employee, you can do the work you love, be yourself, and make an impact in the lives of millions. Our commitments to diversity and inclusion, team culture and the spaces where we work all reflect our mission to keep commerce human.
Job Description
Etsy's Data Engineering department builds systems and infrastructure and writes data pipelines for collecting, storing, and analyzing huge sets of data in batch and streaming contexts. As a member of the Data Pipeliners team, you'll help us identify, extract, and write shared data pipelines that power hundreds to thousands of downstream data processing jobs; you'll be responsible for designing new methods and tooling for ensuring the validity and quality of our data; and you will help other teams write robust data pipelines in Scala Spark, BigQuery SQL, and Kafka Streams. We value curiosity, enthusiasm, responsibility and generosity of spirit.
We're looking for a flexible engineer who cares about the data that flows through pipelines and the internal customers who will use that data. We care more about being excited to pivot to the appropriate tool for the job than having deep expertise in single systems. We work in Scala, Python, Java, and SQL, and we work with technologies like Spark, Kafka, Scalding, Airflow, Looker, Dataproc, and BigQuery. Experience with any of these is helpful but not required.
The technical staff at Etsy believes that code is craft, and that the work we do is part of a larger creative culture that includes the hundreds of thousands of inspired artists and designers who make Etsy such a unique marketplace. We believe that small, empowered, self-motivated teams can do big things. We believe in measuring everything, taking advantage of our continuous deployment system to ship code early and often, and keeping up a blameless culture based on trust and a commitment to learning.
This is a full time role. Our team is headquartered in Brooklyn, but we support remote work and accept remote applicants.
Qualifications
ABOUT THE ROLE
Our team is responsible for the daily delivery of hundreds of business-critical datasets.
We help define, maintain, and monitor the ETL pipelines and data models that power our core business analytics datasets and the upstream feature data that feeds all of Etsy's machine learning systems.
We build and maintain internal tools for monitoring and validating high-quality data.
Our team is leading the company's migration from Scalding to Spark by developing data onboarding materials, defining best practices, and adding tooling to ensure that pipelines are well-written.
ABOUT YOU
You understand that being an effective software engineer is about communicating with people as much as it is about writing code.
You are willing to work with and improve code you did not originally write.
You are generous with your time and experience, and can mentor and learn from other engineers.
You can tackle unconstrained problems and know when to seek help.
You are flexible with languages and tools and are willing to learn whatever is necessary to get the job done.
You have familiarity with a few of the following: writing and scheduling ETL pipelines, writing SQL queries for exploration and analysis, tuning mapreduce jobs and cluster resources, building and monitoring cloud services, stream processing systems like Kafka Streams, Spark, or Dataflow.
Additional Information
At Etsy, we believe that a diverse, equitable and inclusive workplace makes us a more relevant, more competitive, and more resilient company. We welcome people from all backgrounds, ethnicities, cultures, and experiences. Etsy is an equal opportunity employer. We do not discriminate on the basis of race, color, ancestry, religion, national origin, sexual orientation, age, citizenship, marital or family status, disability, gender identity or expression, veteran status, or any other legally protected status. We will ensure that individuals with disabilities are provided a reasonable accommodation to participate in the job application or interview process, to perform essential job functions, and to receive other benefits and privileges of employment. While Etsy supports visa sponsorship, sponsorship opportunities may be limited to certain roles and skillsets.