To help our teams work together effectively, this role requires you to be located in the Australia / New Zealand. Your future team Atlassian is looking for a Principal Data Engineer to join our Product Data Engineering (PDE) team and build world-class data solutions and applications that power crucial decisions throughout the organisation. We are looking for someone who is passionate about building data services that scale. Our team typically works with data scientists, business analysts, and other partners to ensure that data is available, accurate, and usable for decision-making purposes. In this role as a Principal Data Engineer, you will report into the Senior Engineering Manager for Product Data Engineering.
On your first day, we'll expect you to have

• 7+ years of experience working with varied forms of data infrastructure, including relational databases (e.g. SQL), Spark, and column stores (e.g. Redshift)
• 5+ years of experience building scalable data pipelines using Spark using Airflow scheduler/executor framework or similar scheduling tools.
• 5+ years of experience working in a technical environment with the latest technologies like AWS data services (Redshift, Athena, EMR) or similar Apache projects (Spark, Flink, Hive, or Kafka).
• Understanding of Data Engineering tools/frameworks and standards to improve the productivity and quality of output for Data Engineers across the team.
• Industry experience working with large-scale, high-performance data processing systems (batch and streaming) with a "Streaming First" mindset to lead Atlassian's business growth and improve the product experience.

We'd be super excited if you have

• Designed and implemented OLAP (Online Analytical Processing) to power real-time analytics.
• Built real-time data processing pipelines using Apache Spark Streaming/ Databricks and is well-versed with their APIs.
• Develop and implement data governance procedures to ensure data security, privacy, and compliance.
• Above all else, as a Principal Data Engineer, you will lead large programs, driving architectural decisions, and incorporating business and technology strategies. You will mentor junior members of the team and help onboard and grow the team.