Several data cloud innovations have been unveiled at the Google Cloud summit, allowing customers to work with unlimited data across all workloads and extend access to everyone.
The announcements include BigLake and Spanner change streams to help unify customer data while ensuring its delivered in real-time. Vertex AI, Workbench and Model Registry to close the data to AI value gap. And a unified business intelligence (BI) experience includes a new Workspace integration and new programs that further enable the company's data cloud partner ecosystem.
Removing all data limits
"Today, we are announcing the preview of BigLake, a data lake storage engine, to remove data limits by unifying data lakes and warehouses," says Google VP and general manager for Database, Data Analytics - Looker, Gerrit Kazmaier.
"Managing data across disparate lakes and warehouses create silos and increases risk and cost, especially when data needs to be moved. BigLake allows companies to unify their data warehouses and lakes to analyse data without worrying about the underlying storage format or system, eliminating the need to duplicate or move data from a source and reducing cost and inefficiencies."
He says with BigLake, customers gain fine-grained access controls, with an API interface spanning Google Cloud and open file formats like Parquet and open source processing engines like Apache Spark. These capabilities extend a decade's worth of innovations with BigQuery to data lakes on Google Cloud Storage to enable a flexible and cost-effective open lake house architecture.
"Another major innovation we've announced is Spanner change streams," says Kazmaier. "Coming soon, this new product will further remove data limits for our customers, allowing them to track changes within their Spanner database in real-time to unlock new value."
Spanner change streams track Spanner inserts, updates, and deletes to stream the changes in real-time across a customer's entire Spanner database. This ensures customers always have access to the freshest data. They can easily replicate changes from Spanner to BigQuery for real-time analytics, trigger downstream application behaviour using Pub/Sub or store changes in Google Cloud Storage for compliance.
Remove the limits of data workloads
"Our AI portfolio is powered by Vertex AI, a managed platform with every machine learning tool needed to build, deploy and scale models, and is optimised to work seamlessly with data workloads in BigQuery and beyond," says Kazmaier.
"We're announcing new Vertex AI innovations that will provide customers with an even more streamlined experience to get AI models into production faster and make maintenance even easier."
Vertex AI Workbench, which is now generally available, brings data and machine learning systems into a single interface. Teams have a common toolset across data analytics, data science, and machine learning. With native integrations across BigQuery, Serverless Spark, and Dataproc, Vertex AI Workbench enables teams to build, train and deploy machine learning models 5X faster than traditional notebooks.
Extending the reach of data
"We have launched Connected Sheets for Looker, and the ability to access Looker data models within Data Studio," says Kazmaier.
"Customers now can interact with data however they choose, whether through Looker Explore, from Google Sheets, or using the drag-and-drop Data Studio interface. This will make it easier for everyone to access and unlock data insights to drive innovation and make data-driven decisions with this new unified Google Cloud business intelligence platform."
He says the unified BI experience makes it easy to tap into governed, trusted enterprise data, incorporate new data sets and calculations, and collaborate with peers.
The largest online commerce and payments ecosystem in Latin America, Mercado Libre has been an early adopter of Connected Sheets for Looker. Using this integration, they have provided broader access to data through a spreadsheet interface that their employees are already familiar with. By lowering the barrier to entry, they have built a data-driven culture in which everyone can inform their decisions with data.
Doubling down on the data cloud partner ecosystem
Kazmaier says closing the data-to-value gap with these data innovations would not be possible without Google's partner ecosystem. More than 700 software partners power their applications using Google's data cloud. Many partners like Bloomreach, Equifax, Exabeam, Quantum Metric, and ZoomInfo have started using Google's data cloud capabilities with the Built with BigQuery initiative, which provides access to dedicated engineering teams, co-marketing, and go-to-market support.
"Our customers want partner solutions that are tightly integrated and optimised with products like BigQuery," says Kazmaier. "So we're announcing Google Cloud Ready - BigQuery, a new validation that recognises partner solutions that meet a core set of functional and interoperability requirements.
"We already recognise more than 25 partners in this new program. It reduces costs for customers associated with evaluating new tools while also adding support for new customer use cases.
"We're also announcing a new Database Migration Program to help our customers accelerate the move from on-premise and other clouds to Google's managed database services. This includes tooling, resources, expertise and incentives to offset the cost of migrating databases," he adds.