Senior Data Engineer (01393)

Senior Data Engineer (01393)

This job is no longer open
At Nielsen, we believe that career growth is a partnership. You ultimately own, fuel and set the journey. By joining our team of nearly 14,000 associates, you will become part of a community that will help you to succeed. We champion you because when you succeed, we do too. Embark on a new initiative, explore a fresh approach, and take license to think big, so we can all continuously improve. We enable your best to power our future. 

Nielsen Global Media collects billions of data points on what consumers watch. We study consumers in more than 100 countries, and help our clients globally identify unmet needs, develop new products and deliver them in an engaging way.  We are constantly evolving, not only in terms of where we measure, or who we measure, but in how our insights can help drive profitable growth.
 
DESCRIPTION
 
Conduct one-on-one meetings with the manager and team meetings to gather the information needed for developing new software or code. Analyze methodology documents to understand client requirements. Design and develop software and related database infrastructure that will feed into the media data lake (ecosystem) through Nielsen’s data flow processes. Conduct code review with senior developers before pushing the code into production. Support analytics performed by the Data Science team by creating or modifying and then deploying and supporting software applications and data infrastructure to respond to custom analytics requests and client inquiries, and incorporate changing analytical and statistical methodologies, standards and best practices. Combine software engineering expertise with knowledge of Nielsen TV measurement tools and services. Create job flow, automate the jobs and run jobs. Implement data pipeline Directed Acyclic Graphs (DAGs) and maintenance DAGs. Configure and setup DAGs based on the data to run Spark commands in parallel and sequential. Create hooks, as well as Python, Bash, Spark and custom operators. Schedule a pipeline to run the DAG to fetch data for loading into a database. Collaborate with cross-functional Nielsen Media teams to develop and implement customized software solutions that will validate enhancements to the universe estimate methodologies. Perform unit testing using test cases and fix any bugs. Write and run test scripts for Media and Annual Estimate data. Run test scripts for data generated from team members.  Collaborate with Data Scientists by leveraging software engineering expertise to identify and address software development and data quality issues, and detect and resolve quality escapes. Perform detailed analysis of data quality issues, report to other teams on significant data changes, and rectify the problem. Respond to internal data analytics inquiries by determining how to execute the analyses with the platforms and applications available, modifying specifications as needed, providing an estimated delivery date, and accurately delivering data-driven analyses on time. Tools used: Python, SQL, Apache Spark, and Presto, AWS (S3 and EMR); Git, ORC, Parquet, AWS Lambda, Java, Databricks, Athena, Terraform, Bash, Rest API, Amazon ECR, Airflow, GitLab CICD, Gitlab Package Registry.
 
QUALIFICATIONS
 
Minimum Requirements: Bachelor’s degree in computer science, engineering, or a related field with a technology focus (foreign equivalent degree acceptable) plus 3 years of experience in software development. This must include: Two (2) years of experience in: data engineering; (big) data workloads; including handling data quality issues and big data scaling issues; Python, SQL, Apache Spark, and Presto; AWS (S3 and EMR); Git for source code version-control; data lakes and data warehousing; column-oriented data storage formats (ORC, Parquet); and Airflow. Must also include One (1) year of experience in: AWS System Manager Parameter Store, AWS Step Functions, AWS Lambda using Java and Python; Databrick; Athena; Terraform; Bash; Rest api development; Amazon ECR and Amazon EventBridge; Airflow DAG development; AWS Code pipeline; AWS CloudFormation, AWS CodeBuild, AWS Docker Services, AWS SNS, and AWS Glue; IAM Roles Configuration; and GitLab CICD, and Gitlab Package Registry. Salary $109,528- $155,000/year. M-F, 40 hours/week.
 
About Nielsen:
 
Enabling your best to power a better media future. Our comprehensive benefits package (including health & wellness plans for full-time employees, 401(k) retirement coupled with a Nielsen match, a generous paid time off policy, and if eligible, a discretionary incentive/bonus) is designed to be inclusive for all employees and families, and we take pride in ensuring that employees are rewarded holistically for the role they are doing and their performance.
Nielsen is committed to hiring and retaining a diverse workforce. We are proud to be an Equal Opportunity/Affirmative Action-Employer, making decisions without regard to race, color, religion, gender, gender identity or expression, sexual orientation, national origin, genetics, disability status, age, marital status, protected veteran status or any other protected class.
 
#LI-DNI
#IND-DNS
 
As the arbiter of truth, Nielsen Global Media fuels the media industry with unbiased, reliable data about what people watch and listen to. To discover what’s true, we measure across all channels and platforms⁠—from podcasts to streaming TV to social media. And when companies and advertisers are armed with the truth, they have a deeper understanding of their audiences and can accelerate growth. 

Do you want to move the industry forward with Nielsen? Our people are the driving force. Your thoughts, ideas and expertise can propel us forward. Whether you have fresh thinking around maximizing a new technology or you see a gap in the market, we are here to listen and take action. Our team is made strong by a diversity of thoughts, experiences, skills, and backgrounds. You’ll enjoy working with smart, fun, curious colleagues, who are passionate about their work. Come be part of a team that motivates you to do your best work!
This job is no longer open
Logos/outerjoin logo full

Outer Join is the premier job board for remote jobs in data science, analytics, and engineering.