Big Data Lead Engineer

Big Data Lead Engineer

This job is no longer open

The world isn’t standing still, and neither is Allstate. We’re moving quickly, looking across our businesses and brands and taking bold steps to better serve customers’ evolving needs. That’s why now is an exciting time to join our team. You’ll have opportunities to take risks, challenge the status quo and shape the future for the greater good.

You’ll do all this in an environment of excellence and the highest ethical standards – a place where values such as integrity, inclusive diversity and accountability are paramount. We empower every employee to lead, drive change and give back where they work and live. Our people are our greatest strength, and we work as one team in service of our customers and communities.

Everything we do at Allstate is driven by a shared purpose: to protect people from life’s uncertainties so they can realize their hopes and dreams. For more than 89 years we’ve thrived by staying a step ahead of whatever’s coming next – to give customers peace of mind no matter what changes they face. We acted with conviction to advocate for seat belts, air bags and graduated driving laws. We help give survivors of domestic violence a voice through financial empowerment. We’ve been an industry leader in pricing sophistication, telematics, digital photo claims and, more recently, device and identity protection. We are the Good Hands. We don’t follow the trends. We set them.

Job Summary:

This role is responsible for leading efforts of medium to high complexity and critical tracks of delivery of Big Data platforms to make decisions. This includes the development and recommendation of new Big Data systems for analyzing data; the coding and development of advanced analytics solutions to make/optimize business decisions and processes; searching for and integrating new tools to improve descriptive, predictive, and prescriptive analytics; and discovery of new technical challenges that can be solved with existing and emerging Big Data hardware and software solutions. This role will also begin project management of complex projects. This role contributes to the structured and unstructured Big Data / Data Science tools of Allstate from traditional to emerging analytics technologies and methods. The role is responsible for assisting in the recruitment, selection, mentorship, and development of other team members.

Key Responsibilities:

  • Is an active technical leader within the department
  • Partners with Allstate Technology teams on Big Data efforts
  • Identifies new areas of data, research and Big Data technology that can solve business problems
  • Utilize effective project planning techniques to break down complex projects into tasks, manage scope of projects, and ensure deadlines are kept
  • Leverages, contributes, and uses Big Data best practices / lessons learned to develop technical solutions used for descriptive analytics, ETL, predictive modeling, and prescriptive real time decisions analytics.
  • Influences within the department on the effectiveness of Big Data systems to solve their business problems
  • Is recognized as an expert in Big Data and is influential within the department
  • Responsible for creating detailed technical specifications, developing application and system code, participating in code reviews and module testing
  • Designing detailed technical components for complex applications utilizing high-level architecture, design patterns and reusable code
  • Supporting detailed project estimates and project work plans
  • Programming code independently for intermediate to complex modules following development standards
  • Planning and conducting code reviews for changes and enhancements that ensure standards compliance and systems interoperability
  • Creating test plans for module-level integration and tracking/resolving defects
  • Identifying code support issues, action resolutions, and providing technical assistance for moderately complex problems
  • Preparing and packaging production-ready code modules for staging
  • Ability to problem solve and reach resolution with a broader perspective in mind
  • Able to effectively communicate with peers, business analysts, project managers, quality control, and across other technology team boundaries
  • Follows documentation and development standards

Functional Skills:

  • 8+ years of solid Hadoop Development experience
  • Expertise in different Big data eco system tools - Spark, MapReduce, Python, Scala, Unix shell scripting and Hadoop.
  • Proficient with Spark on Containers/docker/K8 platform.
  • Experience working with AWS S3 /Object store /Scality.
  • Knowledge on open-source Spark & analytical solutions
  • Experience with code repository, build and deployment tools - Artifactory, GitHub, Jenkins, CI/CD methodology
  • Experience with data analytics technologies on AWS is a strong plus.
  • Basic understanding of Relational Databases (RDBMS), SQL & No-SQL databases is a plus
  • Ability to communicate and present advanced technical topics to general audiences
  • Enjoy mentoring, coaching, and delivering compassionate, direct feedback to team members as well as receiving and acting on feedback from others.
  • Leading project teams of various skills levels

Education and Experience:

• Master's Degree or equivalent experience
• 5-8 years of related experience

Certificates, Licenses, Registrations:

• None

Supervisory Responsibilities:

This job does have supervisory duties.

Notes:

The preceding description is not designed to be a complete list of all duties and responsibilities. May be required to perform other related duties as assigned. Regular, predictable attendance is an essential function of this job.

Compensation offered for this role is $130,000-$165,000 Plus Annual Incentive Bonus Potential and is based on experience and qualifications.

The candidate(s) offered this position will be required to submit to a background investigation, which includes a drug screen.

Good Work. Good Life. Good Hands®.

As a Fortune 100 company and industry leader, we provide a competitive salary – but that’s just the beginning. Our Total Rewards package also offers benefits like tuition assistance, medical and dental insurance, as well as a robust pension and 401(k). Plus, you’ll have access to a wide variety of programs to help you balance your work and personal life -- including a generous paid time off policy. For a full description of Allstate’s benefits, visit allstate.jobs/benefits/

Learn more about life at Allstate. Connect with us on Twitter, Facebook, Instagram and LinkedIn or watch a video.

Allstate generally does not sponsor individuals for employment-based visas for this position.

Effective July 1, 2014, under Indiana House Enrolled Act (HEA) 1242, it is against public policy of the State of Indiana and a discriminatory practice for an employer to discriminate against a prospective employee on the basis of status as a veteran by refusing to employ an applicant on the basis that they are a veteran of the armed forces of the United States, a member of the Indiana National Guard or a member of a reserve component.

For jobs in San Francisco, please click “here” for information regarding the San Francisco Fair Chance Ordinance.
For jobs in Los Angeles, please click “here” for information regarding the Los Angeles Fair Chance Initiative for Hiring Ordinance.

To view the “EEO is the Law” poster click “here”. This poster provides information concerning the laws and procedures for filing complaints of violations of the laws with the Office of Federal Contract Compliance Programs

To view the FMLA poster, click “here”. This poster summarizing the major provisions of the Family and Medical Leave Act (FMLA) and telling employees how to file a complaint.

It is the Company’s policy to employ the best qualified individuals available for all jobs. Therefore, any discriminatory action taken on account of an employee’s ancestry, age, color, disability, genetic information, gender, gender identity, gender expression, sexual and reproductive health decision, marital status, medical condition, military or veteran status, national origin, race (include traits historically associated with race, including, but not limited to, hair texture and protective hairstyles), religion (including religious dress), sex, or sexual orientation that adversely affects an employee's terms or conditions of employment is prohibited. This policy applies to all aspects of the employment relationship, including, but not limited to, hiring, training, salary administration, promotion, job assignment, benefits, discipline, and separation of employment.

This job is no longer open
Logos/outerjoin logo full

Outer Join is the premier job board for remote jobs in data science, analytics, and engineering.