Why Consumption-Based Apps Thrive on Snowflake

Data-intensive applications typically process, mine, and analyze huge volumes of highly varied structured, semi-structured, and unstructured data. To do this effectively while meeting customer requirements for speed, responsiveness, and accuracy, these applications require near-unlimited storage and highly available, flexible, and scalable resources on demand.

At the same time, SaaS providers are exploring consumption-based pricing models as a way to expand business opportunities and grow revenue—as well as satisfy customer demands for flexible, usage-based pricing. However, consumption-based models require modern infrastructure that can support critical operations such as accurate usage tracking, granular visibility into the costs of providing services, centralized data access, and demonstration of business value to consumers.

SaaS providers need to make a key decision: What data platform solves all these challenges and provides an excellent foundation to build and grow? But before we answer that question, let’s take a quick look at the basics of consumption-based pricing.

What is consumption-based pricing, and why does it matter?

With consumption-based pricing (also known as usage-based pricing), customers are only charged for resources consumed. For customers, key benefits of this pricing model include:

• Pay as you go: Only pay for what you use, when you’ve used it
• Scale as you grow: Adjust your usage according to demand
• No up-front costs

Why should SaaS providers care about usage-based pricing? This blog post gives you several reasons, but first and foremost, usage-based pricing offers you a chance to easily communicate your value to your customers and show that you’re a partner, not just a vendor (after all, you only succeed if they use your solution). You also have an opportunity to grow your revenue as customers try your solution, see its value, and find new uses for it. And you can build momentum in your industry as you develop a better understanding of your customers’ needs and usage patterns.

Consumption-based pricing lets you charge customers in alignment with the way they use your product or solution. Here are a few examples:

• Charge based on resources used (compute, storage, and serverless features)
• Charge a flat fee + variable pricing (tiered usage)
• Charge based on data retrieved (price per GB queried) or charge per query

Why Snowflake is the right platform for consumption-based apps

Building on Snowflake allows you to leverage Snowflake’s own consumption-based pricing model, along with several other elements critical for consumption-based apps:

Scalability: Enables the continuous ingestion of massive amounts of data from all applications and systems, as well as near real-time access and usage of all data.

Performance and speed: Handles almost any volume of data and allows concurrent data usage to power fast analysis and machine learning models—without sacrificing speed or performance.

Data transformation: Transforms data to deliver a single source of data that can be joined or linked together, as well as ingested and pushed back to external applications. This automation enables re-forecasting on a daily basis without burden or complexity.

Data types: Brings together data types produced by different applications and systems, including structured, semi-structured, and unstructured data.

Centralized source of truth: Enables you to access relevant data instantly from all data sources to develop a 360-degree view of your business.

Secure Data Sharing: Empowers users to share live data from its original location, without copying or transmitting data.

Snowflake Data Marketplace: Allows you to easily share data via Snowflake Data Marketplace and leverage Snowflake’s built-in pricing models and billing infrastructure to create new product lines with consumption-based pricing.

Security and governance: Delivers a best-in-class security model to restrict access and provide governance at every data level, thus enabling fine-tuned control to ensure the right data is shared with the right people.

Let’s take a closer look at how various Snowflake features address some of the challenges faced by data applications and SaaS providers using consumption-based models.

Scalability

The Snowflake platform delivers near-limitless scale and concurrency so you can confidently offer the scale-on-demand capabilities that are key to usage-based models. Snowflake is also the only platform that can integrate data across multiple clouds and regions to give users a seamless experience.

Figure 1: Snowflake separates compute and storage to enable highly flexible and scalable operations.

Customers naturally want to provision and pay for required resources only. Trying to provision resources ahead of time can be error-prone and wasteful. Snowflake’s unique architecture (see Figure 1) enables application providers to easily provision highly available and scalable dedicated or shared resources on demand. The multi-cluster shared data architecture with separate compute clusters (virtual warehouses) enables seamless scaling of both storage and compute resources.

Virtual warehouses can either be standard or multi-cluster warehouses. Standard warehouses consist of a fixed number of compute nodes based on warehouse size. Multi-cluster warehouses can dynamically provision additional clusters as needed. Snowflake automatically suspends resources when they are not utilized. Data is persisted on cloud storage (S3, Azure Blob, or GCS), which can be scaled to virtually any capacity.

Account management

Applications need a robust set of account management features to handle all the tasks involved in onboarding new customers, accurately reporting consumption (whether hourly, daily, monthly, or another metric), and monitoring the services offered.

Traditionally, provisioning new accounts required manual intervention, which led to delays because each account had its own set of administration pages and views. The Organization feature in Snowflake (currently in public preview), however, enables you to manage multiple accounts from a single interface, quickly create new accounts in any cloud or region, and view usage for multiple accounts in a unified view (see Figure 2).

Figure 2: Snowflake Organizations feature allows you to manage multiple accounts from a single view.

Typical account model options include one of the following patterns:

Separate account for each customer: Dedicated compute and database objects enable easy and exact billing per customer.

Single account with dedicated resources 

• Dedicated compute (virtual warehouses) and dedicated database objects: Dedicated compute nodes for each customer allows data providers to accurately measure usage and scale horizontally as needed. Similarly, with dedicated database objects, it is easy to measure storage usage and understand consumption trends.

• Dedicated compute (virtual warehouses) and shared database objects: In this model, storage usage will need to be allocated to customers based on customer-specific data volumes. Snowflake’s object tagging and storage metrics at table level allows customers to allocate the storage costs appropriately. Although maintenance of database objects is easier with this approach, dedicated storage has the benefit of reducing overall complexity.

Single account with shared resources: Allocating usage for shared resources to different customers using traditional methods can be difficult. With Snowflake object tagging and query tagging features, you can easily allocate usage with a simple ratio_to_report window function.

For more details on these options, including application setup and pros and cons, check out our white paper: Design Patterns for Building Multi-Tenant Applications on Snowflake.

Serverless capabilities

Snowflake’s serverless capabilities enable data providers to run tasks on behalf of their customers in a serverless way. Tag the tasks that include customer information, and you can easily share usage information. Because the workload is fully dedicated, tasks will perform much better and be more efficient—a big plus for providers offering usage-based pricing to their customers.

Cost transparency for billing and Usage

Snowflake provides summary and detailed usage metrics for all services used. Therefore, it is easy to understand the spend and monitor and manage the billing. The Snowflake database consists of Organization_Usage, Account_Usage, Reader_Account_Usage, and Data_Sharing_Usage.

Organization_Usage views provide summary usage data for all the accounts in an organization.

Account_Usage views provide information specific to a particular account, including:

• Credits consumed by virtual warehouses
• Storage costs
• Credits consumed by other Snowflake services such as Snowpipe, Automatic Clustering, materialized views, search optimization, and serverless tasks
• History of all the queries with key performance metrics (such as execution time, bytes scanned, bytes spilled to remote storage, and query load percent)
• Access history and login history

Once query tagging and object tagging is in place, you can build an accurate charge-back model easily by allocating credits consumed by shared resources.

Applications and providers can expose the charge-back data to each of their customers in a separate database using private, secure sharing similar to Account_Usage. Thus, the data providers can easily pass the cost along to customers. Usage data can be shared with the consumers without moving the data as shown in Figure 3.

Figure 3: Data providers and consumers both get up-to-date, transparent access to usage data through Snowflake—without moving the data itself.

Cost management and monitoring

Resource monitors and alerts provide an easy way to monitor resources, and alert and suspend when certain thresholds are exceeded. Thresholds can be adjusted on demand by actively monitoring the usage trends on a daily basis. Snowflake’s Snowsight admin dashboard is a very useful tool for monitoring and understanding usage (see Figure 4).

Figure 4: Snowsight dashboards give you at-a-glance information about organizations, consumption, and storage so you can monitor usage easily and quickly take proactive action.

Data governance and security

SaaS providers need to support industry best practices around data governance and security, but building it from the ground up can be complex and time-consuming. Snowflake provides robust security features that allow developers to incorporate governance every step of the way, without extra effort.

For example, Snowflake provides a rich set of role-based access controls and Dynamic Data Masking features, such as Row Access Policies and column-level masking, to ensure that only authorized users can access sensitive data. Powerful data classification features and tagging make it easy to track and categorize sensitive data. These features allow SaaS providers to create charge-back views and share them with customers using appropriate row-level security, so customers see only their data—an elegant solution that helps grow customers’ trust.

Conclusion

Snowflake continually enhances, optimizes, and improves its offerings to ensure customers continue to get value from Snowflake products. That’s another reason why consumption-based apps thrive on Snowflake. With a huge network of providers and consumers sharing data seamlessly, it’s easy to see why Snowflake is an excellent platform for building apps with consumption-based pricing models. Learn more about building on Snowflake at ​​https://developers.snowflake.com.

The post Why Consumption-Based Apps Thrive on Snowflake appeared first on Snowflake.