As a Staff Software Engineer - Data Ingest Platform, you will work alongside cross functional teams of product engineers, scientists and designers to bring the latest machine learning innovations into production. You will focus on our data ingest pipelines and capabilities, ensuring performance, data quality, and core capabilities meet downstream team’s needs.

From your first day, you might…

• ...architect and extend scalable ingest pipelines to ingest, analyze, and understand large volumes of data
• ...partner with feature teams to design an information retrieval architecture to analyze and conduct search on documents and graphs
• ...develop internal tools to help manage the pipelines and data stores, including facilitating adding new sources, parsing new document types, integrating with third party APIs and backfilling data
• …work on massive datasets, integrating with features such as event and topic detection, relation extraction, summarization, entity recognition, document classification, multilingual NLP and knowledge base generation

Experiences we like

• Experience leading commercial software technical projects end-to-end in an agile environment on any major cloud provider (AWS, GCP, Azure, etc)
• Experience with data ingestion and processing systems at scale
• Experience with working with real-time and distributed systems
• 8+ years of shipping production grade software
• Expertise in Python
• Expertise in ElasticSearch, PostgreSQL, Kafka and Kubernetes

Added Bonus

• Data-driven engineering; using metrics to drive decisions and inform performance and cost tradeoffs
• Hands-on DevOps experience with cloud infrastructure
• Experience with Golang, Redis
• Work experience in AI/ML focused companies