Summary:

The Principal Data Engineer for the Data Engineering team will work closely with the Data Architect helping to implement and evangelize the data vision for the business.

Are you an innovative and collaborative Engineering Leader that is passionate about enterprise level big data solutions and enablement of Data Engineering, Analytics, and Machine Learning projects Join Edmentum’s Data Platform team as a Principal Data Engineer and lead exciting enablement work that focuses on delivering Cloud-based Enterprise Data Platform solutions.

As a Principal Data Engineer, you will be the Engineering Leader focused on driving the development of a highly collaborative team that solves Edmentum’s big data challenges. You will lead and perform development activities for our data platform, which serves as the foundational data architecture for Data Science, Data Analytics, and Research & Learning Engineering teams. You will be directly involved in the design, development and implementation of data infrastructure, data pipelines and scalable data architecture. You’ll work to ingest and process both streaming and batch data using leading-edge cloud tools from AWS and Databricks that represent purpose-built enterprise architecture. We invite you to bring your passion for data and join us in operationalizing exciting data projects aligned to Edmentum’s strategic goals.

ESSENTIAL DUTIES AND RESPONSIBILITIES:

• Define and own development and release processes for the Data Engineering team. Including coding standards, CI/CD processes, and release management.
• Lead technical implementation and solutioning discussions and drive decisions for data platform engineering efforts.
• Lead collaborative onboarding for the Data Analytics team, allowing them to contribute to the data lake presentation layer codebase.
• Help groom and define tasks and stories for data platform engineering backlog.
• Interface with application data owners to build collaborative working relationships with a focus on improving data quality at the source.
• Mentor and coach Data Engineers providing technical growth opportunities and raise the skillset of the team.
• Identify, anticipate, and mitigate issues and challenges with the Enterprise Data Platform.
• Creates and communicates the Technical Vision for the Enterprise Data Platform.

SPECIALIZED KNOWLEDGE, SKILLS, and ABILITIES

• Bachelor of Science (B.S.) from a 4-year college or university in Computer Science, Information Management Systems, or equivalent experience
• 5 or more years of experience working with relational databases such as SQL, RDBMS (Relational Database Management Systems) (SQL Server), PostgreSQL, AWS RDS, or RedShift
• 5 or more years of experience with Big Data technologies such as (Redshift, Snowflake, BigData, Hadoop)
• 5 or more years of experience with programming languages such as (Python, Scala, Java, C#)
• 3 or more years of experience with streaming data processing tools such as Spark (PySpark or Scala), Kinesis, or cloud functions such as AWS Lambda
• 5 or more years of experience with batch data processing pipelines
• 5 or more years of experience with job administration and scheduling of complex data workloads using tools such as Apache Airflow
• 2 or more years of experience with AWS platform data tools such as: EMR, S3, Athena, DynamoDB, EC2, Glue, and ECS (Elastic Container Service) experience working in a modern DevOps-enabled environment with tools such as GitHub, Docker, and Terraform.