Starburst updates enterprise offering, extends support for data mesh architecture

Starburst has launched the latest version of Starburst Enterprise, providing customers with net new capabilities alongside more advanced connectivity, improved performance, and more security features.

Among the new capabilities, Starburst Cached Views expands the concept of traditional materialised views to be applied to the data mesh, the company states.

Starburst Cached Views allows domain experts to transform and move data according to their needs.

In addition, the release includes a new proprietary Parquet reader, to help improve read performance on Parquet files by an average of 20% over Trino.

In addition to Starburst Cached Views and Parquet reader, this release of Starburst Enterprise includes these new offerings to help serve data mesh architectures.

Standardised data lineage and governance support allows data owners to control access while giving data consumers insight into the lineage of their data.

Starbursts Atlas integration allows sensitive data to be tracked as it is moved to ensure future access maintains the same security controls.

Starburst can authenticate with a number of OIDC compliant identity providers (Okta, Azure and on-premises ADFS) and pass-through the identity to authenticate to many underlying data sources that support OIDC. This allows organisations to centralise authentication control for improved security and reduced operational overhead.

As more data products are created in the mesh, it's important to continue to add connectivity, preventing data silos, the company states.

In addition, in this release Starburst has added connectivity for DynamoDB.

Previously, customers needed to extract data from DynamoDB and load into another database for SQL analytics. Now Starburst customers can use the DynamoDB connector and query the data directly on DynamoDB with Starburst without data movement.

As organisations create, manage and consume more and more data, lakehouses allow organisations to cheaply store large amounts of data while giving users the fine-grained control they need to keep their data up to date, and compliant with data regulations.

Finally, Starburst has announced improvements to support lakehouse architectures.

Enhanced support for the Delta Lake format includes a native statistics collection that is used by Starburst's cost-based optimiser to improve query performance by a factor of 2.

Starburst also integrates with the Delta Lake OPTIMISE command which allows for more efficient data organisation thus further increasing the performance of Delta Lake queries.

Numerous performance enhancements include dynamic filtering accelerates federated JOINS, aggregation pushdown moves query processing closer to the source, and improvements to the Parquet reader significantly improve read throughput on data lakes.

Starburst Data VP product and co-founder, Matt Fuller, says, "Achieving a successful data mesh architecture requires the ability to access data in disparate systems and sources.

"Starburst Cached Views enables users to query data from other systems, and transparently cache that data in their own domain for increased performance.

"This can provide significant performance advantages by pre-calculating complex queries and joins and caching closer to consumers. It can also significantly reduce data egress costs and enable domain experts to create a performant semantic layer.