The Sr. Data Engineer brings DataOps, data engineering, DevOps and cloud computing expertise in expanding and optimizing our data lake processing pipeline and data analytics architecture. They bring technical expertise to design, architect, implement and support solutions to scale and optimize data flow into and out of cloud-based data lakes for cross functional teams to consume for Tableau reporting, Ad-hoc querying and Machine learning processing. #LI-IB1; #LI-Remote

In addition, expertise in cross-team (NetOps, DevOps, InfoSec, SecOPs, etc.) collaboration via use of professional skills are utilized to perform stretch roles as Sr. Data Engineer.

The Sr. Data Engineer will be a mentor and SME to other members of the team, as well as consult to management and leadership.

Job Responsibilities: 

• Design, build, and maintain a scalable and low-latency infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources and targets using AWS data lake related services and analytics-related technologies. 
• Support and collaborate with down-stream consumers, i.e. Tableau, Ad-hoc and Machine Learning to ensure their success by assembling and delivering required data sets, security, and acceptable query times
• Perform exporting/sync of data to third-party products, i.e. Salesforce, FTP Servers, etc. using AWS AppFlow, SFTP SSH, etc.
• Be SME from within the team to provide expertise to bring innovative ideas from concept to fruition via collaborative POCs, Spikes and /or discussions with team members.
• Identify, design, document and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc. 
• Understand and guide a comprehensive testing, continuous integration and development framework for schema, data, and functional processes/pipelines, etc. 
• Be SME for tools used by Data Engineering, i.e. GitHub, Jira, Confluence, etc.
• Manage AWS accounts dedicated to DataOps team with root level access that have DevOps, NetOps, InfoSec and DataOps stretch-role responsibilities. 

Skills Needed:  

• Strong experience with AWS cloud services: S3, DMS, RDS, Lambda, Redshift, Glue, AppFlow, EC2, in cross-region and cross-account implementations
• Strong experience with AWS security: IAM Roles/Policies/Users, Okta/SSO
• Strong experience with AWS networking: VPC, Subnets, security groups, route tables, gateways, etc.
• Strong experience with technical writing to document processes, best practices, create architecture diagrams, etc. using Atlassian Confluence, Lucid Charts, etc.
• Strong experience with Agile Scrum: Managing, creating and maintaining Jira board, issues, sprint shutdown, grooming, status updates with management, supporting documentation and sprint demos. Perform stretch-role of Scrum Master.
• Strong experience with Infrastructure as Code via Terraform, AWS CDK for Python, AWS CloudFormation, etc.
• Strong experience with NoSQL databases, i.e. DynamoDB, DocumentDB and/or MongoDB
• Strong experience with relational/DW SQL databases such as MySQL, PostgreSQL, and Redshift
• Strong experience with programming in Python via python.org and Anaconda python using AWS Boto3, PyCharm IDE, etc. and data engineering related packages: pandas, numpy, scipy, sqlalchemy, pyarrow, redshift connector
• Strong experience building and optimizing data pipelines, architectures, data lakes, data sets, etc. 
• Strong experience with reading and writing of multiple file formats, i.e. JSON, text/CSV, and parquet
• Strong experience with transferring and receiving of files via SFTP SSH using ssh CLI, Python, Filezilla, etc.
• Strong experience with Code repo management - Git and GitHub, managing and overseeing repos, branches, pull requests, etc. using standalone CLI and GUI tools, and integrated with PyCharm.
• Strong experience with Salesforce: reading and writing of data to and from the data lake, Salesforce Query Language, and Salesforce objects and structures.
• Strong experience API usage and connectivity via Python and Postman REST API client tool.
• Experience with AWS CloudWatch alerting and notification integration with third-party tools: PagerDuty, Slack, etc.
• Experience with managing and overseeing dedicated AWS accounts: billing, tuning, security and access management, network and connectivity management, root level login/access/tasks, etc. 
• Experience with containerization technologies, such as Docker, AWS ECR, AWS Lambda
• Experience with third-party Data and File management tools, i.e. DBeaver, FileZilla, etc.
• Experience with AWS and third-party ETL tools: AWS Glue, Hevo, Boomi, etc.
• Experience with master data, metadata, versioning, tagging, cataloging of data sources
• Familiarity with data visualization and analytics tools like Tableau

Educational Requirements & Work Experience: 

• 5+ years of experience in a Data Engineering role in a team setting 
• 5+ years of experience working with AWS cloud using data related services
• 5+ years of experience working with RDS, Redshift and NoSQL
• 2+ years of experience working with Data Lakes in AWS
• Undergraduate degree in Computer Science, Statistics, Informatics, Information Systems or another quantitative field. Graduate degree preferred.  
• AWS certifications a plus
• Experience leading data analysts or managing teams a plus

The pay range for this role takes into account the wide range of factors that are considered in making compensation decisions including but not limited to experience and training; skill sets; licensure and certifications; and other business and organizational needs. A reasonable estimate of the base salary compensation range is $122,445 to $165,000 per year.

We also offer a range of benefits and programs to meet employee needs based on eligibility. These benefits include comprehensive health care coverage, a health savings account, disability, and life insurance, a Critical Illness and accident plan, a flex spending account (medical and dependent care), a 401k plan with a company match, mental health support TaskHuman, EAP, financial coaching, Rocket Lawyer, and more. HeartFlow offers 12 paid holidays, 15 vacation days, and 80 hours of sick leave.