At Talkdesk, our Engineering team follows a micro-service architecture approach to build the next generation of Talkdesk, with vertical teams responsible for all the decisions under their services. Through our Agile Coaches, we promote agile and collaborative practices, we are huge fans of Scrum, pair programming and we won’t let a single line of code reach production without peer code reviews. We strongly believe that the only true authority stems from knowledge, not from position and we always treat others with respect, deference and patience.

Do you want an opportunity to deliver strategic and tactical influence regarding the application of Big Data technologies?

We are looking for an experienced Data Engineer who will steer development of a highly scalable and extensible real time and batch oriented data pipeline for collection, storage, processing, and analysis of large sets of data from multiple data sources, including our real-time communications platform

Responsibilities

• Selecting, integrating and building any Big Data tools and frameworks required to provide requested capabilities (e.g., Spark, Flink, Cassandra, Hadoop, etc.);

• Implementing ETL process, and building data pipelines from event based services to make data easily accessible for Analytics;

• Monitoring performance and advising any necessary infrastructure changes;

• Defining data retention policies and capacity planning;

• Work closely with data scientists, product managers, data analysts, user experience experts, and quality engineers to build new features to empower our business through data.

Required Skills:

• Proficient understanding of distributed computing principles;

• Proficiency with building stream-processing systems in scala, java, python, or another programming language

• Experience with integration of data from multiple data sources;

• Advanced knowledge of SQL and experience with SQL databases, such as PostgreSQL and Redshift;

• Experience with NoSQL databases, such as MongoDB, Cassandra, and HBase;

• Knowledge of various ETL techniques and frameworks, such as Flume;

• Experience with various messaging systems, such as Kafka and RabbitMQ;

• Experience with cloud environments such as AWS and Google Cloud;

• Strong written and verbal English communication skills.

Nice to Haves/Pluses:

• BS/MS Degree in Computer Science, Applied Math, or an engineering-related field;

• Experience in Agile development methodology/Scrum;

• Experience with Amazon Redshift/Snowflake;

• Good understanding of Lambda and Kappa Architectures, along with their advantages and drawbacks;

• Experience with Cloudera/MapR/Hortonworks;

• Management of Hadoop, Spark, Flink, Kafka clusters with all included services;

• Ability to solve any ongoing issues with operating storage and processing clusters;

• Experience with Big Data ML toolkits, such as Mahout, SparkML or H2O.