Description

As Udemy looks toward the future of online education, skill assessments lie at the heart of promoting learner progression and growth. Udemy’s skill assessment group is looking for a data scientist with a strong background in psychometrics, statistical modeling, machine learning, and hands-on experience with empirical research methods in the social sciences.

As a data scientist at Udemy, you will be partnering with product managers, designers, and engineers to build algorithmic and model-driven solutions to power skill assessments. This role will require a combination of applied measurement experience, data-driven decision making, and the data science and coding chops to dive headfirst into development on production systems. A successful candidate will have knowledge of best industry practices for growing and maintaining a large set of assessment products, while being courageously experimental in finding new solutions to ongoing issues in test development and maintenance.

Here Are Some Questions You Could Be Working On:

• What is the appropriate methodology for equating/linking assessments on our platform?
• How do we best maintain and utilize operational item banks?
• What other user activities can be used as indicators of learning?
• How do assessment results relate to other user outcomes?
• How do users engage with other learning products after completing assessments?
• How can content recommendations be improved using assessment outcomes?

Here Is What You Will Be Doing:

• Collaborate with other data scientists, PMs, designers, and engineers to maintain and grow our skill assessment platform
• Effectively communicate and present to both technical and non-technical audiences
• Critically evaluate and implement the state-of-the-art research literature in educational assessment
• Develop production-grade data pipelines and continuously improve the quality and accessibility of data
• Develop psychometric models for skill assessment through all phases, including prototyping, productionization, and experimental evaluation
• Define metrics and build dashboards to track the adoption and health of skill assessments
• Collaborate with content developers and instructional designers to develop and refine item writing and review processes

We Are Excited About You Because You Have:

• 3+ years of hands-on industry experience in a psychometric or data science-oriented role
• Domain experience in educational assessment and psychometrics (e.g., Classical Test Theory (CTT), Item Response Theory (IRT), Computerized Adaptive Testing (CAT), equating and scaling) 
• Strong SQL ability and proficiency with 1+ programming languages (e.g., R or Python)
• End-to-end experience organizing and analyzing large data sets for operational purposes
• Ability to communicate effectively with non-technical stakeholders
• Strong knowledge of applied statistics and experimental design
• Ability to work autonomously, relentlessly prioritize, and collaborate with team members across time zones

We Are More Excited About You Because You Have:

• 5+ years of relevant industry experience
• Experience with machine learning and causal inference
• Familiarity with standard software engineering practices (e.g., Scrum)
• Experience using big data technologies (e.g., Hadoop, Spark)

Minimum Salary Expectations:  125,000

About Udemy 

At Udemy, we’re all about improving lives through the power of learning. We are a leading global learning company and one of the world’s largest education platforms with more than 54 million learners. Our goal is to provide flexible, effective skill development to empower organizations and individuals. Talented people are everywhere, but opportunities can be hard to come by. That’s why we’re focused on revolutionizing learning, using our skills and expertise to help others develop theirs and reach their full potential. Individually, we bring our unique perspective to reimagine the way we share knowledge. Together, we can improve lives by empowering our learners, our instructors, and businesses around the world.

We are proud to be recognized for our world-class employee experience. Learn more about our Great Place to Work certification here and find out what it's like to work at Udemy on our blog.

Udemy is headquartered in San Francisco with global offices in the US, Turkey, Ireland, Australia, India, and Brazil. Learn more on our company page.

Information regarding data privacy is available within the Udemy Careers Privacy Notice.

At Udemy, we value diversity and inclusion and consider qualified applicants without regard to race, color, religion, sex, national origin, ancestry, age, genetic information, sexual orientation, gender identity, marital or family status, veteran status, medical condition or disability. We will consider for employment qualified applicants with arrest and conviction records.

Udemy Benefits:

• Eligibility: Regular, full-time employees are eligible for Udemy’s benefit programs.
• Health Plans: Medical, dental, and vision coverage (100% coverage for employee-only).
• HSA/FSA/Commuter: Pre-tax savings/spending plans available; generous HSA employer contributions for those enrolled in the HDHP medical plan.
• Life/Disability: Employer-paid life insurance (supplemental available), in addition to short-term and long-term disability.
• Retirement: Access to 401(k) with annual employer contribution.
• Wellbeing: Corporate memberships for meditation and mindfulness, therapy and coaching, financial planning, primary care, tele-health, health advocacy, parent/newborn support, and employee discounts.
• Education: Free access to the entire course library on the Udemy and Udemy for Business platforms; annual stipend for external learning beginning at six months of employment.
• Charitable Matching Program: Employer match of monetary contributions to eligible nonprofits and charities that carry a 501(c)(3) tax status.
• Vacation: 15 days per year of Paid Time Off for hourly; flexible Discretionary Time Off for salaried.
• Parental Leave: 8 weeks of leave at 100% pay for parents who take time off from work following the date of birth, adoption, or foster placement beginning at six months of employment; this amount is in addition to pregnancy-disability benefits at 100% pay, if applicable.
• Holidays: 11 paid holidays throughout the year