Mission

As a Backline Technical Solutions Engineer - Spark you will allow our customers to be successful with the Databricks platform by resolving important technical escalations that come from customers and the frontline support team. You will be the technical bridge between the frontline support team and the engineering team, thereby becoming the first line of defense for engineering. You will ensure that all issues are vetted by you before it reaches the engineering team. Along with this you will also ensure that the global support team stays updated through technical trainings, workshops and other enablement programs.

Outcomes

• Troubleshoot, resolve and if required suggest deep code-level analysis of Spark to address complex customer issues related to spark core internals, spark sql, structured streaming, Databricks Delta and other data engineering and product areas
• Provide best practices guidance and suggest appropriate programming usage of Spark core libraries and APIs for custom-built solutions developed by Databricks customers
• Build detailed troubleshooting guides and run books to enhance the efficiency of the support team.
• Contribute to automation and tooling projects to make daily troubleshooting, and efficient.
• Partner with the Spark Engineering Team within Databricks and ensure that the support team is aware of upcoming features and releases. 
• Proactively reach out to the front line team members and provide technical assistance for long running customer cases.
• Provide suggestions for improving Spark runtime performance in customer-specific environments
• Pinpoint specific areas of the Spark codebase that are being affected by known bugs and try to suggest possible workarounds.
• Strengthen your understanding of the entire Spark codebase and work towards becoming an Apache Spark contributor/committer.
• Demonstrate true sense of ownership and coordinate with engineering and escalation teams to achieve resolution of customer issues and requests
• Value collaboration and coach other team members
• Participate in weekend and weekday on call rotation

Competencies 

• Minimum 6 years experience developing, testing, and sustaining Python or Java or Scala-based applications. Expert level knowledge in python/scala is desired.
• High-level understanding of the Apache Spark codebase is desired. 
• Must be comfortable with compiling and building the Apache Spark source code.
• Identify and apply patches/bug fixes to the Apache Spark source code is desired.
• Experience in Big Data/Hadoop/Spark/Kafka/Elasticsearch data pipelines.
• Solid experience with SQL-based database systems is preferred.
• Linux/Unix administration skills.
• Skills in Linux JVM, GC, Thread dump-based troubleshooting is required.
• Experience with AWS or Azure related services is preferred.
• Candidates must possess excellent written and oral communication skills.
• Good technical skills in Programming, data structures and algorithms is preferred.
• Demonstrated analytical and problem-solving skills, particularly those that apply to a “Distributed Big Data Computing” environment.
• Bachelor’s degree in Computer Science or a related field is required

About Databricks

Databricks is the data and AI company. Founded by the original creators of Apache Spark™, Delta Lake and MLflow, Databricks simplifies data and AI so data teams can collaborate and innovate faster. More than five thousand organizations worldwide —including Shell, Conde Nast and Regeneron — rely on Databricks as a unified platform for massive-scale data engineering, collaborative data science, full-lifecycle machine learning and business analytics. Venture-backed and headquartered in San Francisco (with offices around the globe) Databricks is on a mission to help data teams solve the world’s toughest problems. To learn more, follow Databricks on Twitter, LinkedIn and Facebook.