Nielsen Media would not function without our Technology teams! We are catalysts for delivery quality, on-time, reliable measurements to clients, and we are cultivators, growing our employees through education, skill building and experiences. Around the globe, our Technology teams are relentless in our pursuit of superior analytics, technology, process and support.

Technology and Engineering - USA Chicago, Illinois 

The Global Outcomes with Nielsen team is looking for a Senior Data Engineer to help us bring data-intensive products to market, maintain existing products for our clients, and work closely with our data science teams in a cloud-native, Python and Spark-heavy big data stack.

A typical day at this role includes attending a standup with data engineers, data scientists, and our product owner, talking about various data sources we’re integrating into our machine learning pipelines with upstream Nielsen teams, guiding data scientists in how to access the data, and turning their analyses into production-ready Spark code that runs using Airflow.

Work with other data engineers, data scientists, architects, and product owners on an agile scrum team that delivers products to production.

Gather, analyze and convert business requirements into AWS (Amazon Web Services) cloud-based solutions.

Design and build systems that load and transform a large volume of structured and semi-structured data. When we say “big data” we don’t mean a few gigabytes – we work with multi-terabyte datasets on a daily basis.

Build and test cloud-based data pipelines and applications (primarily in Python and Apache Spark + SQL) for new and existing backend systems.

Write reusable, well-tested code and components (e.g. RESTful APIs, Python packages, etc.) that can be used by multiple project teams.

Assist in troubleshooting and debugging of ETL code and resolving data integrity issues alongside our data scientists and client-facing customer success teams.

Work in a serverless environment. We don’t maintain VMs nor do we manually deploy infrastructure. Automation and scalability is critical.

Write code with performance, maintainability, scalability, and reliability in mind.

Our tech stack: Python, Apache Spark, SQL, Apache Airflow, Hive, AWS Glue, AWS Athena, AWS EC2, AWS S3, AWS CodeBuild, AWS CloudFormation, YARN, Git, RESTful Microservices, Kubernetes (k8s).

Master’s degree in computer science, engineering, or a related field with an information technology focus (foreign equivalent degree acceptable) plus 3 years of experience in software design and development.

OR

Bachelor’s degree in computer science, engineering, or a related field with an information technology focus (foreign equivalent degree acceptable) plus 5 years of experience in software design and development

3 years of experience with:

delivering end-to-end applications and pipelines (including architecting open source-based ETL pipelines and designing, building and implementing big data solutions).

2 years of experience with:

AWS, Azure, or Google Cloud Platform, preferably in a serverless tech stack.

Designing and developing Apache Spark-based applications using Python (PySpark) or Scala and Spark SQL.

Comfort with the Linux command line, Git, Agile Scrum and at least one data orchestration tool e.g. Apache Airflow, Luigi, Azkaban, AWS Data Pipeline, Oozie, etc..

As the arbiter of truth, Nielsen Global Media fuels the media industry with unbiased, reliable data about what people watch and listen to. To discover what’s true, we measure across all channels and platforms⁠—from podcasts to streaming TV to social media. And when companies and advertisers are armed with the truth, they have a deeper understanding of their audiences and can accelerate growth. 

Do you want to move the industry forward with Nielsen? Our people are the driving force. Your thoughts, ideas and expertise can propel us forward. Whether you have fresh thinking around maximizing a new technology or you see a gap in the market, we are here to listen and take action. Our team is made strong by a diversity of thoughts, experiences, skills, and backgrounds. You’ll enjoy working with smart, fun, curious colleagues, who are passionate about their work. Come be part of a team that motivates you to do your best work!