Spark ETL Tech Lead

Spark ETL Tech Lead

This job is no longer open

The world isn’t standing still, and neither is Allstate. We’re moving quickly, looking across our businesses and brands and taking bold steps to better serve customers’ evolving needs. That’s why now is an exciting time to join our team. You’ll have opportunities to take risks, challenge the status quo and shape the future for the greater good.

You’ll do all this in an environment of excellence and the highest ethical standards – a place where values such as integrity, inclusive diversity and accountability are paramount. We empower every employee to lead, drive change and give back where they work and live. Our people are our greatest strength, and we work as one team in service of our customers and communities.

Everything we do at Allstate is driven by a shared purpose: to protect people from life’s uncertainties so they can realize their hopes and dreams. For more than 89 years we’ve thrived by staying a step ahead of whatever’s coming next – to give customers peace of mind no matter what changes they face. We acted with conviction to advocate for seat belts, air bags and graduated driving laws. We help give survivors of domestic violence a voice through financial empowerment. We’ve been an industry leader in pricing sophistication, telematics, digital photo claims and, more recently, device and identity protection. We are the Good Hands. We don’t follow the trends. We set them.

Job Summary:

As a Spark ETL Tech Lead in the Allstate Benefits Technology Team, your ETL knowledge and deep statistical understanding of data will help drive the complete transformation of a legacy system.  You'll be hands on with the latest technology to develop new tools and capabilities and you'll be initiating, planning, conducting and coordinating data projects and analytical exercises.  You'll also add value by supporting ad-hoc data requests and will be involved in testing efforts to ensure data integrity.

Key Responsibilities:

  • Identifies and resolves simple to complex data issues
  • Performs and evaluates simple to complex trend analysis (e.g. competitor analysis, risk management, financial)
  • Advanced knowledge in the use of data sources and applications to conduct research
  • Plans, implements, manages data projects into a cohesive project plan to move the most important work forward and begin to build the information needed to best manage customer data and analysis
  • Assists in the development the Allstate ABI data quality and validation guidelines
  • Contributes to the common vision of elevating the organizations’ commitment to clean, reliable, and accurate customer and business data
  • Functions as a liaison in the relationship management of external data vendors/partners
  • Ensure a cross functional view of business data is available to ABI
  • Supports the partnership with finance and compliance to ensure externally reported data is accurate and balanced appropriately
  • Ensure protocol is adhered to for customer data quality including joining disparate information systems to create a holistic view of the customer and his/her interactions with Allstate 
  • Works with product owners to prioritize features for ongoing sprints and managing a list of technical requirements, known defects, and issues.
  • Help maintain the integrity and security of the company database.
  • Understanding of data warehousing and data modeling techniques.
  • Take responsibility for Big Data development and implementation.
  • Debugging troubleshooting issues with current systems.

Job Qualifications

Education and Experience

  • Bachelor’s Degree or equivalent experience

REQUIRED SKILLS:

  • 5+ years of software development experience with proven hands-on experience in Spark/NoSQL, Scality S3 Storage, Dremio & SQL Databases.
  • Ab Initio ETL is a skill that would be nice to have, but not required for role.
  • Strong analytic skills related to working with unstructured datasets.
  • Strong SQL skills and query optimization in relational database.
  • Must have hands-on experience in working on shell scripts in Unix.
  • Familiar with Safe Agile application development methods
  • Experience supporting and working with cross-functional teams in a dynamic environment.
  • Knowledge of best practices related to data security, and application disaster recovery
  • Must have previous experience in data migration/conversion from various database platforms
  • Excellent documentation, communication & team building skills
  • Proven ability to lead multiple high priorities initiative with aggressive timelines leveraging an agile framework

Functional Skills

  • Uses sound decision making capabilities including problem solving approaches, decision frameworks; ability to design and lead complex analysis
  • Thrives in a collaborative environment
  • Advanced quantitative and analytical skills
  • Ability to synthesize large amounts of complex data into meaningful conclusions and recommendations
  • Strong presentation skills; dynamically (persuasively) presenting ideas to management
  • Ability to inspire trust and influence management
  • Ability to persuasively advocate a position and build momentum for change and action
  • Advanced ability to understand Business Insurance consumer trends, industry/ competitive dynamics, and economics
  • Working knowledge of commercially available analytical tools (i.e. Excel, ACCESS, SAS) would be nice to have in this role
  • Proven track record managing multiple priorities given aggressive timelines; comfortable performing in a fast paced, dynamic and ambiguous business environment; manage both internal and external resources
     

Compensation offered for this role is $80,500.00-$134,800.00 per year and is based on experience and qualifications.

The candidate(s) offered this position will be required to submit to a background investigation, which includes a drug screen.

Good Work. Good Life. Good Hands®.

As a Fortune 100 company and industry leader, we provide a competitive salary – but that’s just the beginning. Our Total Rewards package also offers benefits like tuition assistance, medical and dental insurance, as well as a robust pension and 401(k). Plus, you’ll have access to a wide variety of programs to help you balance your work and personal life -- including a generous paid time off policy. For a full description of Allstate’s benefits, visit allstate.jobs/benefits/

Learn more about life at Allstate. Connect with us on Twitter, Facebook, Instagram and LinkedIn or watch a video.

Allstate generally does not sponsor individuals for employment-based visas for this position.

Effective July 1, 2014, under Indiana House Enrolled Act (HEA) 1242, it is against public policy of the State of Indiana and a discriminatory practice for an employer to discriminate against a prospective employee on the basis of status as a veteran by refusing to employ an applicant on the basis that they are a veteran of the armed forces of the United States, a member of the Indiana National Guard or a member of a reserve component.

For jobs in San Francisco, please click “here” for information regarding the San Francisco Fair Chance Ordinance.
For jobs in Los Angeles, please click “here” for information regarding the Los Angeles Fair Chance Initiative for Hiring Ordinance.

To view the “EEO is the Law” poster click “here”. This poster provides information concerning the laws and procedures for filing complaints of violations of the laws with the Office of Federal Contract Compliance Programs

To view the FMLA poster, click “here”. This poster summarizing the major provisions of the Family and Medical Leave Act (FMLA) and telling employees how to file a complaint.

It is the Company’s policy to employ the best qualified individuals available for all jobs. Therefore, any discriminatory action taken on account of an employee’s ancestry, age, color, disability, genetic information, gender, gender identity, gender expression, sexual and reproductive health decision, marital status, medical condition, military or veteran status, national origin, race (include traits historically associated with race, including, but not limited to, hair texture and protective hairstyles), religion (including religious dress), sex, or sexual orientation that adversely affects an employee's terms or conditions of employment is prohibited. This policy applies to all aspects of the employment relationship, including, but not limited to, hiring, training, salary administration, promotion, job assignment, benefits, discipline, and separation of employment.

This job is no longer open
Logos/outerjoin logo full

Outer Join is the premier job board for remote jobs in data science, analytics, and engineering.