New Snowflake Features Released in May–July 2023

At Snowflake Summit, we announced a wave of product innovations: Snowpark ML Modeling API, Snowflake Native App Framework, Dynamic Tables and more. If you missed out, not to worry! Read our Summit recap blog for highlights across industries or watch Summit sessions now on-demand. Let’s dive into all the new releases in May, June and July. 

Applications 

Snowflake Native App Framework now available in AWS – public preview

Snowflake Native Apps are an entirely new way to put data to work. Consumers can discover and purchase Snowflake Native Apps on Snowflake Marketplace, then install and run them within their own Snowflake account. Developers can now start building and testing Snowflake Native Apps in their accounts in AWS. Distribution and monetization capabilities will be available in public preview on AWS later this year. The Snowflake Native App Framework remains available in private preview on Google Cloud Platform and Azure. Learn more here.

Build your own pricing strategy for your Snowflake Native Apps – public preview

Snowflake offers a range of monetization options, from subscription-based models to usage-based models priced per month or per query. Custom Event Billing capabilities enable you to build your own pricing strategy. You can charge customers based on their usage and specify billing events based on your preferences, such as consumed rows, ingested rows and more. Learn more here

Streaming and Continuous Pipelines & Data Onboarding 

Snowpipe Streaming  – general availability

Streaming pipelines are often challenging to build. Data comes in a continuous manner, and often a separate architecture is required to handle streaming data. What remains challenging is how streaming data is brought together with batch data. That’s why we built Snowpipe Streaming, now generally available to handle row-set data ingestion. Snowpipe Streaming enables low-latency streaming data pipelines to support writing data rows directly into Snowflake from business applications, IoT devices or event sources such as Apache Kafka, including topics coming from managed services such as Confluent Cloud or Amazon MSK. Learn more here.

Better replication and schematization for streaming ingestion – public preview

With this release, we are adding the support of Snowpipe Streaming with Snowflake replication. Snowflake supports the replication and failover of Snowflake tables populated by Snowpipe Streaming, and its associated channel offsets from a source account to a target account in different regions and across cloud platforms with replication. Snowpipe streaming supports both database replication and group-based replication.

The new Kafka connector, built with Snowpipe Streaming, now supports schema detection and evolution. The structure of tables in Snowflake can be defined and evolved automatically to support the structure of new Snowpipe streaming data loaded by the Kafka connector. 

Build incremental pipelines declaratively – public preview

Once the data is landed, building and maintaining streaming and continuous pipelines is a whole different story. This is where Dynamic Tables (currently in public preview) come in. Dynamic Tables is a new table type that drastically simplifies continuous data pipelines for transforming both batch and streaming data, declaratively.

Instead of defining data transformation steps as a series of tasks and having to monitor dependencies and scheduling, you can simply by defining the end state of the transformation with dynamic tables, and leave the complex pipeline management to Snowflake. Using just a few lines of SQL (or Snowpark Python), you can build continuous pipelines with just a few lines of code. Learn more here.

Onboard data easily with schema detection and evolution – pubic preview

We are pleased to announce a public preview of the schema detection feature for JSON and CSV, and table schema evolution, making data onboarding easier. Schema can be automatically detected for Apache Parquet, Apache Avro and ORC files, further expanding our support to include JSON and CSV files. Not only for detection, we are also making improvements for table schema evolution. The structure of tables in Snowflake can now evolve automatically to support the structure of new data received from the data sources.

Snowpark 

Faster and more intuitive end-to-end ML development in Snowflake – public preview

Now in public preview, the Snowpark ML Modeling API scales out feature engineering and simplifies ML training execution natively in Snowflake. Learn more here.

New Python versions in Snowpark – public preview

Snowpark now supports Python versions 3.9 and 3.10. Visit Snowflake Documentation for more details. 

Advanced Analytics 

Enhance speed and decision quality with ML-Powered Functions – public preview

Enhance speed and quality of decisions with ML-Powered Functions, now in public preview. These familiar SQL functions abstract the complexity of ML frameworks and algorithms for time-series forecasting, anomaly detection and more. Now you can scale from one to millions of ML-powered insights with the elasticity and near-zero operations of Snowflake’s engine, and share insights in analytics or BI tools integrated with Snowflake’s consistent data governance across model inputs and outputs. Learn more about ML-Powered Functions in our blog or in Snowflake documentation.

Advance geospatial analysis with a new data type: GEOMETRY – general availability

GEOMETRY, a data type designed for those using local spatial reference systems (SRS), is now generally available in Snowflake. GEOMETRY uses planar geometry and supports thousands of projections, which is especially useful in representing local areas while minimizing distortion errors. Learn more about how to use high-precision geospatial data by watching this virtual hands-on lab.

Perform orientation shape transformations for geospatial – general availability 

Streamline how you perform orientation transformations with our new set of functions, all in GA: ST_SIMPLIFY, ST_BUFFER, ST_AZIMUTH and ST_MAKEPOLYGONORIENTED. Learn more about the functions in this blog post.

Handle invalid geospatial shapes – general availability

Now, you can load and store invalid shapes in our GEOGRAPHY and GEOMETRY columns, and either fix them after the import or keep using them as they are. Additionally, we added a new function, ST_ISVALID, to verify whether a shape is valid. Learn more in Snowflake documentation.

Collaboration and Marketplace 

Accelerate purchases on Snowflake Marketplace by paying with Snowflake Capacity – general availability

Eligible Snowflake customers can now purchase third-party data and Snowflake Native Apps seamlessly on Snowflake Marketplace from across clouds with their Snowflake Capacity commitment, accelerating access to third-party content for data teams and streamlining procurement. Check out this blog post to see how you can enroll in the program.

Automated fulfillment of data across regions and clouds – general availability

Listing providers in Snowflake can now ensure their consumers always have fresh, up-to-date data irrespective of their region or cloud. Providers set a replication frequency and select targeted cloud regions, in which their public listings are automatically replicated and kept in sync as soon as there is demand for their listing in a new region. For listings targeted at specific accounts, providers don’t need to specify anything but the replication frequency—Snowflake ensures the data is automatically fulfilled in the respective target accounts’ cloud regions. See Snowflake documentation on how to set up Cross-Cloud Auto-Fulfillment on your listings.

Manage costs of automatically fulfilling data across regions and clouds – public preview

Two new Views have been added to the data sharing usage schema to help Listing Providers manage the cost of Cross-Cloud Auto-Fulfillment; that is, the automated replication of data across regions and clouds. The new Views provide visibility into storage costs associated with replicated data in remote Snowflake regions for the purpose of fulfilling consumer demand for a listing’s data product in a region, as well as visibility into compute costs associated with refreshing the data in remote Snowflake regions. See more details on understanding and managing related costs here as well as schema definitions in Snowflake documentation.  

Observability

Application observability gets better with logging and tracing via event tables – public preview

We are improving application observability for developers and data engineers. Prior to Snowflake logging and tracing via event tables, there was no easy way to capture application logs and traces, and no centralized location to consume them. Now with logging and tracing via event tables, developers can better monitor and observe in order to build applications more effectively. 

With this new feature, developers and data engineers can easily instrument their code to capture logs and traces across UDFs, UDTFs and stored procedures for all languages: Java, Scala, JavaScript, Python and Snowflake Scripting. Learn more here.

Data Lake

Connect Snowflake to s3-compatible storage devices on-premises or in the cloud – general availability

Customers can use Snowflake to access data in s3-compatible storage devices while getting the ease of use, elasticity, unified governance, resilience and connectivity of Snowflake’s platform. Use cases could include performing analytics on data lakes with External Tables, simplified ingestion of on-premises files to tables in the cloud, or even using Snowpark Python, Java or Scala to process files stored externally. For more information, including a list of supported storage providers and our public test suite, please read our product documentation.

Snowsight

More reasons to upgrade to Snowsight 

Snowsight, an updated web interface for Snowflake, has become the default destination for many new and longtime Snowflake users. Most recently, all on-demand customers (those that pay month-to-month based on their usage with a credit card) now see Snowsight by default when they log in. With big new functionality such as new Python Worksheets, the ability to upload a CSV to smaller usability enhancements like Worksheet tabs, and the ability to create named stages, Snowsight gets better every day. Learn more in our Snowsight documentation.

Snowflake Marketplace  

Snowflake customers can tap into Snowflake Marketplace for access to more than 1,800 live data sets, packages of data sets, or data services (applications are currently in private preview) from over 430 third-party data providers and data service providers (as of April 30, 2023), as well as market their own products across the Snowflake Data Cloud. Visit Snowflake Marketplace.  Here are all the providers who posted new listings in May, June and July:

Transportation 

AAA Inc. 

Keyrus 

Government

Acezd 

California State Water Resources Control Board 

Security

aero 

Anvilogic 

Criminal IP 

VGS 

Financial 

Arcesium 

Bloomberg Data Management Services Limited 

Equileap

ExtractAlpha 

Foodtruck.ai 

Grata 

GoldenSource LLC 

Helios Life Enterprises, Inc. 

IFIS Japan

ISS ESG 

LSEG 

Mastercard 

Maxa 

Resonate 

Richmond Global Sciences 

Taiwan Economic Journal (TEJ) 

TipRanks 

Marketing

Aterio.io

Bond Brand Loyalty 

Experian Data Quality 

FlashIntel 

NowVertical Group 

OneSignal 

Persado Inc 

S&P Global Mobility 

Geospatial

EarthDefine LLC 

HtAG Analytics

Michael Bauer Research GmbH 

PTV Planung Transport Verkehr GmbH 

The PropTech Cloud 

WeDoTech

Zeal Co., Ltd. 

Commerce

Datos Inc. 

FullStory 

LaunchDarkly 

Momentum Commerce LLC 

Planhat 

SELECT 

tgndata 

Demographics

Denominator 

Digiseg 

Good Boy Studios 

Healthcare and Life Sciences

Everfortune 

IntegraConnect 

NTT DATA 

Payerset LLC 

Precision Data 

Rearc 

Connectors

Flurry Insights 

Matillion

Energy

Housitive

Lookup Tables

Logitix

Environment

Neural Alpha 

Media

Relo Metrics 

RSG Media 

Identity

Roqad 

Sports

Sports Innovation Lab 

—–

​​Forward-Looking Statements

This post contains express and implied forward-looking statements, including statements regarding (i) Snowflake’s business strategy, (ii) Snowflake’s products, services, and technology offerings, including those that are under development or not generally available, (iii) market growth, trends, and competitive considerations, and (iv) the integration, interoperability, and availability of Snowflake’s products with and on third-party platforms. These forward-looking statements are subject to a number of risks, uncertainties, and assumptions, including those described under the heading “Risk Factors” and elsewhere in the Quarterly Reports on Form 10-Q and Annual Reports of Form 10-K that Snowflake files with the Securities and Exchange Commission. In light of these risks, uncertainties, and assumptions, actual results could differ materially and adversely from those anticipated or implied in the forward-looking statements. As a result, you should not rely on any forward-looking statements as predictions of future events. 

© 2023 Snowflake Inc. All rights reserved. Snowflake, the Snowflake logo, and all other Snowflake product, feature, and service names mentioned herein are registered trademarks or trademarks of Snowflake Inc. in the United States and other countries. All other brand names or logos mentioned or used herein are for identification purposes only and may be the trademarks of their respective holder(s). Snowflake may not be associated with, or be sponsored or endorsed by, any such holder(s).

The post New Snowflake Features Released in May–July 2023 appeared first on Snowflake.

LATEST ARTICLE

See Our Latest

Blog Posts

admin July 15th, 2024

In an increasingly interconnected world, where technology drives every facet of life, even the weather predictions we rely on may […]

admin July 10th, 2024

There is a scene in Mission: Impossible – Rogue Nation where Tom Cruise is hanging onto the outside of a […]

admin July 10th, 2024

Regulated and sovereign markets across the world have stringent requirements stipulating certain important data be kept within geographical borders or […]