Recorded Future collects data from many dark web and underground sources. These include forums, markets, shops, file download sites, and alternative social media platforms. As a data engineer, you will build capabilities to reliably collect text, image, and binary data from these sources, and you will build analytic capabilities that distill this raw data into high fidelity intelligence about the criminal underground.

What you’ll do as a Data Engineer: II  

• Expand our collection reach with new underground (UG) sources, and strengthen our methods by investigating collection issues and fixing the root causes.
• Solve hard underground data collection problems, such as evading anti-bot methods, coordinating the work of many collector agents, and safely collecting binary data from untrustworthy sources.
• Build high value analytics on raw data from UG sources. Examples: find networks of actors/accomplices, and highlight notable conversation threads.

What you will bring to the Data Engineer II role:

• 2+ years experience in software engineering using Python. You write clean, production-grade code that your teammates can easily work with. You have some familiarity with scraping frameworks and/or browser orchestration like Selenium. 
• Great problem solving capabilities and experience troubleshooting data issues. In UG data collection, you cannot ask the source’s webmaster for tech support!
• Proactive communication and effective collaboration with your teammates to get technical problems resolved. You are a self-starter. The ball is always in your court.

#LI-Remote