Snowflake core Technology Interpretation series one architecture design

2022-05-10 0 By

Background: On September 16, 2020, Snowflake raised $3.4 billion at a valuation of $70.4 billion on its first day of trading.Snowflake became the largest software IPO to date, with a market cap topping $120 billion.Snowflake offers a cloud-based data storage and analytics service, commonly referred to as “data warehouse as a service,” that allows enterprise users to store and analyze data using cloud-based hardware and software.Snowflake has been running on Amazon S3 since 2014, Microsoft Azure since 2018, and Google Cloud platform since 2019. Its Snowflake Data Exchange allows customers to discover, Exchange, and securely share Data.Snowflake is a huge commercial success. How does technology support its $100 billion market cap?What’s so good about it?OLAP kernel technology enthusiast Zhechuan explains Snowflake’s core technology principles.This article is the first in this series.Snowflake is an enterprise-class SaaS data warehouse product built entirely in the cloud that takes full advantage of cloud features. It is flexible (pay-as-you go), high security, extremely scalable and resilient, and supports multi-tenancy, transactions, standard SQL syntax, and semi-structured and unstructured data.Snowflake launched in 2015 and is now a petabyte cloud-native data warehouse that runs millions of queries a day.Figure 1 Shows the overall architecture of Snowflake.Snowflake’s core components are designed to be highly reliable and scalable based on the ultimate goal of ease of use, operability, and reliability.Components are independent of each other and communicate with each other through RESTful interfaces.The Snowflake kernel component can be divided into three layers from the bottom up: 1) Data storage.Snowflake’s data store is built on top of Amazon S3 storage service and is primarily used to store table data and query results.2) Virtual warehouse.A virtual warehouse is built on an elastic cluster of Amazon EC2 VMS and is responsible for executing user query requests.3) Cloud service components.The cloud service component is the brain of Snowflake. It includes concurrent access control, infrastructure management, optimizer, transaction management, security management, and metadata management. Metadata includes schema information, table information, authorization information, secret keys, statistics, and more.As an appetizer, this article begins with an overview of Snowflake architecture design.This will be followed by a series of articles covering Snowflake’s key cloud native technology and other innovations, cloud service components, and cloud-oriented features.The original link: this is ali cloud original content, shall not be reproduced without permission.