THE EARNEST RESEARCH COMPANY

Earnest Research is a VC-backed data innovation startup driven to change the way professionals understand consumer and business behavior. Working with world-class data partners, we transform raw data into a source for business and investment professionals to ask better questions so they can make better decisions. We believe, in the right hands, data has the power to change the way we work. 

We look for the following characteristics in all of our employees:

• Creative problem solving
• A high level of enthusiasm and proactivity
• Attention to detail
• The ability to succinctly communicate ideas
• The ability to produce high-quality work under tight timelines
• A willingness to take ownership of work
• The ability to work as part of a team and effectively with others

Earnest Research is headquartered in New York, NY and we fully support remote working.

DATA ENGINEER, DATASETS TEAM

Earnest Research is seeking a Data Engineer to join our newest product and fastest growing dataset. The Datasets Team is responsible for the ingestion, transformation, and productization of dozens (and growing!) disparate datasets. As a Data Engineer in our Datasets Team, you will be instrumental in creating the next generation of Earnest’s products and play a leading role in building our internal and client-facing data pipelines, infrastructure, and tooling. This is a chance to work on a cross-functional team with modern managed cloud services, functional programming, and lots of data. The work we do will be directly attributable to the next level of growth for Earnest.

RESPONSIBILITIES

• Collaborate with product owners and data analysts in the development and delivery of new product features across a multitude of datasets
• Build and maintain integrated data pipelines, systems, and internal tooling in functional Scala, Python, and SQL to power the company’s products
• Define ETL/ELT logic for processing terabytes of raw data, including writing scripts, calling APIs, writing BigQuery SQL, Dataflow (Apache Beam) and Spark
• Ensure high data duality and pipeline stability
• Work with the engineering organization to build Earnest’s data platform, in particular interfacing with our data science group
• Assist analysts with troubleshooting data issues and leverage technology to increase their productivity

QUALIFICATIONS

Required:

• Experience processing large amounts of structured and semi-structured data
• Programming experience in Python, a JVM language, SQL, and Bash
• 2+ years writing and maintaining ETL at a terabyte level scale 
• 1+ years experience working with Hadoop applications (Spark/Scalding) or Dataflow (Apache Beam)
• Experience with version control systems (Git)
• Substantial SQL and data modeling experience, particularly focussed on efficient transformations
• Industrious and conscientious with the ability to work both independently and in a collaborative environment
• Effective interpersonal, written and verbal communication with engineers and non-engineers

Preferred:

• Code-based data transformation orchestration / scheduling with Apache Airflow, Dagster, Luigi, Flyte or similar
• Knowledge of Amazon Web Services (AWS), especially EMR, and columnar storage-style databases including Snowflake
• BigQuery and GCP experience, including use of or knowledge of Pub/Sub
• Scala experience, either with microservices or distributed big data transformation tools like Spark / Scalding
• Experience with Docker containerization and CI/CD toolchains
• Knowledge of statistics and analytics
• Data warehouse modeling experience
• Experience with or willingness to learn functional programming paradigms
• Experience with unit testing, property checking, and type-driven development
• Experience automating data quality checks through Data Build Tool (DBT), Great Expectations or other company tools

 BENEFITS & PERKS:

• Flexible and generous time off
• 100% company paid medical plan options (additional medical, dental and vision plans available too!)
• 401K retirement plans
• Generous Parental Leave Policies
• Pre-tax savings plans for public transportation and parking expenses
• Regular company happy hours, lunches & events

Earnest Research is an equal opportunity employer, and we encourage people with a diverse range of backgrounds to apply.