Platform Data Engineer (Remote)
Engineering | Full-time, Remote
Astronomer is the commercial developer of Apache Airflow, a community-driven open-source tool that’s leading the market in data orchestration. We’re a globally-distributed and rapidly growing venture-backed team of learners, innovators and collaborators. Our mission is to build an Enterprise-grade product that makes it easy for data teams at Fortune 500’s and startups alike to adopt Apache Airflow. As a member of our team, you will be at the forefront of the industry as we strive to make Apache Airflow the de-facto standard in data orchestration.
Astronomer's Data Team drives the business and product direction by leveraging analytics via our Data Platform, which is built around Apache Airflow. Your role will be to develop data pipelines and tools within this Data Platform, contributing to the development of Astronomer’s products and open-source technologies.
We are looking for a full-time Platform Data Engineer who will design and develop data pipelines and machine learning workflows. These workflows will serve two purposes: to provide insights, reports, and operational models that will power Astronomer’s business; and (of equal importance) to act as a reference point and ‘laboratory’ for using Airflow. You will work within an agile, sprint-based team, using Airflow to build and orchestrate the data pipelines that power our business, and you will also use open-source and other data technologies to build our data ops and machine learning environment.
What you get to do
Use Airflow to orchestrate pipelines from source data to operational analytics
Work with a huge variety of different data sets — as part of real-world analytics projects that impact all of Astronomer’s operational systems.
Collaborate with stakeholders at all organizational levels to translate mission-critical business requirements into actionable analytics
Help drive the development of a data orchestration and ETL/ELT framework for internal business operations and analytics
As an internal user of Airflow (as well as open-source and cloud technologies like Snowflake, Databricks, Preset, Great Expectations, dbt, etc.) provide regular feedback to the product and engineering teams
Develop tools and Airflow extensions to assist internal teams and the greater Apache Airflow community
What you bring to the role
Professional experience with supporting data pipelining needs of users, including writing Python, creating data pipelines, and building tools to make this easier for other users
Familiarity with DevOps best practices
Data engineering and ETL/ELT experience in a production environment (2 years minimum)
SQL experience (1 year minimum, 2+ years ideal)
Python fluency
Strong analytical and problem-solving skills
Excellent interpersonal and communication skills
Enthusiasm for collaborating in a team-oriented environment
Bonus points if you have
Proficiency with using modern data engineering tools (Apache Airflow, dbt, Snowflake, Spark, Databricks, BigQuery, etc.)
Familiarity with data analysis, statistics, or machine learning
Bachelor's degree in computer science, information technology, information systems, or a related field OR equivalent experience
Applicable professional certification/qualifications
At Astronomer, we value diversity. We are an equal opportunity employer: we do not discriminate on the basis of race, religion, color, national origin, gender, sexual orientation, age, marital status, veteran status, or disability status. Astronomer is a remote-first company.