Snowpark : Data engineering with Snowflake.

Learn fundamentals of Snowflake-snowpark API.

What is Snowpark ?

With Snowpark, Snowflake allows developers to write code in their preferred language.

Along with Snowflake’s original SQL interface now snowflake allows you to write code in,

         1. Python

         2. Scala

         3. Java

Some of the key features of Snowpark are,

1. Your code will be pushed to snowflake leveraging the compute power of snowflake warehouses.

2. You will not end up exporting data to different environment but rather your code is shipped to the data.

3. You can build complex data pipelines or data products using Snowpark

   
   

Snowpark also address below overheads in conventional data pipelines,

1. Long startup time of node clusters: Systems like Hadoop and Spark requires cluster of nodes to process data. Most of the time it takes 5-10 minutes to just start the cluster. In case of Snowpark we will be using snowflake warehouse to process our data.

2. Problem of small files , Problem of using right joins to shuffle data across nodes, Problem of garbage collection. uncertainty when the compute nodes goes down.

   
   

   All the above problems are well handled with Snowpark.

What you will learn ?

You will learn the basics of SNOWPARK API.

    > Basic read and write operations.

    > Read data from s3 and load that to snowflake table.

    > We will do deep analysis of how SNOWPARK API works.

    > Do's and Don'ts of SNOWPARK.

    > Build data components to process data.

    > Build data pipeline to process data.