Snowflake - Senior Data Engineer

Snowflake - Senior Data Engineer

phData works exclusively in the realm of data and machine learning. By partnering with the biggest cloud data platforms (Snowflake, Azure, AWS, Cloudera), we’re able to help the world’s largest companies solve their toughest challenges.

Our work is challenging and our standards are high, but we invest heavily in our employees, starting with a 2-4 week bootcamp to ensure you’ll be successful. Plus, you’ll get to work with the brightest minds in the industry and the latest data platforms on the market. And, because the data and ML industry is changing rapidly, you will always have the opportunity to learn - whether that’s a new technology, diving deeper into your preferred stack, or picking up an entirely new skill set.

Even though we're growing extremely fast, we maintain a remote-first, exciting work environment. We hire top performers and allow them the autonomy to deliver results. Our award-winning workplace fosters learning, creativity, teamwork. Most importantly, our team has the option to work from the conveniences of their home or work from our state of art Bangalore office located in the heart of the city.

  • 2022 Snowflake Partner of the Year
  • 2022 Best Places to Work
  • Best Places to Work (2017, 2018, 2019, 2020, 2021)
  • Inc. 5000 Fastest Growing US Companies (2019, 2020, 2021)
  • Minneapolis/St. Paul Business Journal’s Fast 50 (2021)
  • Snowflake Elite, Snowpark Accelerated & Snowflake Partner of the Year (2020 & 2021)

Core Competencies

A Senior Data Engineer, in phData is responsible for developing, validating, and maintaining the small/large scale complex data integration and data pipelines workloads. This includes working with technologies such as Apache Spark (PySpark or Scala or Spark SQL) or data services from cloud providers like AWS/Azure or Cloud Native platforms like Databricks (Datalake or Lake House) or Snowflake Cloud Data Platform.  They may also be involved and participate in the data integration, data modeling, data governance, and data security tasks, and may work closely with architects and other customer technical stakeholders to understand data needs and implement appropriate solutions. Ultimately, the goal of this role is to participate in the data development projects to enable our customer to extract value from their data and bring the data solution live.

Must Have Competencies & Technical Skills

  1. Good working experience developing end to end data pipeline for small and large scale data sets from a variety of sources based on specification.  
  2. Possesses a strong understanding of the technical aspects of the data engineering software development lifecycle, including development, validation, deployment, and performance monitoring.
  3. Solid working experience with snowflake native objects like SnowPipe, external/internal stages, external tables, integration objects, stored procedures (Snowflake Scripting or JavaScript), functions, stream & task, data masking, time travel, cloning, data sharing to build data pipelines and object promotion techniques from non-govern (Dev)  to govern environments (QA/Prod).
  4. Practical experience with data loading/unloading techniques (SnowSQL CLI, Copy Command put/get), structure & unstructured data (JSON or Parquet) and warehouse selection to onboard small and larget scale data in snowflake tables. 
  5. Basic awareness of role based access control and object ownership with snowflake objects including object hierarchy and how they are implemented.
  6. Familiar with basic concepts of cloud storage solutions such as AWS S3 or Azure ADLS Gen2 or GCP Buckets, and understands how they integrate with Snowflake as external stages. 
  7. Having basic awareness of the common data integration & data transformation patterns for small and large scale data sets.
  8. Extensive practical experience with various types of SQL, including standard, analytical, and snowflake specific extended SQLs. 
  9. Solid understanding of data validation process using some kind of utilities or manual process for small and large scale data sets.
  10. Hands-on experience troubleshooting, optimizing, and enhancing data pipelines and bringing improvements in the production environment using snowflake query history and query profile features.  
  11. Possesses a good  understanding of dimensional modeling techniques and their practical application. This includes knowledge of various dimensional modeling patterns.
  12. Well versed with version control, continuous integration and deployment procedure (Ex. GitHub or GitLab or Bitbucket or Code Build or Jenkins etc).

Nice To have Skills (In Past 2 Years)

  1. End to End Data migration experience from Legacy (Oracle or SQL Server or DB2 or Netezza etc) to Snowflake Cloud Data
  2. Data Transformation tool like dbt (dbt cli or dbt cloud)
  3. Data Integration Cloud Tool like Fivetran
  4. Low Code No Code ETL Tool like Matillion
  5. Data Change Management Tool like Flyway or Liquibase. 

Behavioral Requirement

  1. Must be curious and hungry to learn.
  2. Be ready to learn and adapt to new technologies or new types of tools as and when required. 
  3. Demonstrated ability to work independently as well as with team.
  4. Good communication skills (verbal and written) - one of the most important skills when working with phData as a consulting and service organization.
  5. Good and collaborative team player with in-house team as well as hybrid team with members from client, vendor, and in-house resources. 
  6. Given the project execution dimension and deliverables, a strong sense of time management is required.

Qualifications Requirements

  1. BE/BTech in computer science or MCA or equivalent degree
  2. Should have 5-6 years experience as SQL Developer or Data engineer with 1+ years experience building solution in Snowflake Cloud Data Platform.
  3. Good Programming or Scripting Language Experience (Python or Java or Scala or  JavaScript or Snowflake Scripting). 

Perks and Benefits:

  1. Medical Insurance for Self & Family
  2. Medical Insurance for Parents
  3. Term Life & Personal Accident
  4. Wellness Allowance
  5. Broadband Reimbursement
  6. Professional Development Allowance
  7. Reimbursement of Skill Upgrade Certifications
  8. Certification Bonus
Logos/outerjoin logo full

Outer Join is the premier job board for remote jobs in data science, analytics, and engineering.