Cloudera’s AI play is all about data, leveraging a decade of experience

This year has marked a significant uptake in artificial intelligence offerings, both standalone and AI-enhanced solutions, as organisations look to maximise the potential of ever-advancing technologies.

At Cloudera's EVOLVE24 events, which has the hybrid data company visiting several prominent cities globally, the company's executives have been sharing with partners and customers how the company's AI play is all about data. Cloudera aims to support initiatives with infrastructure and analytics, and allowing customers to bring their AI models to the data and not the other way around.

At Evolve Singapore, targeted towards the company's APAC market, TechDay sat down with top executives from the company to discuss how Cloudera is approaching AI, and why data is so crucial.

A history in data and hybrid infrastructure

Founded in 2008, Cloudera has been focused on making a name for itself as a big data and analytics leader for more than a decade. Fast forward to this year and the organisation has 25+ exabytes of data under management, which is on par with the hyperscalers of the industry. Those leading the organisation across product and strategy, attest to the fact that Cloudera is "the only true hybrid data platform for data, analytics and AI".

Frank O'Dowd, Cloudera Chief Revenue Officer, comments that Cloudera has 100 times more data under management than other cloud-only vendors, and empowers global enterprises to transform data of all types, on any public or private cloud, into valuable, trusted insights. Cloudera's open data lakehouse delivers scalable and secure data management with portable cloud-native analytics, enabling customers to bring GenAI models to their data while maintaining privacy and ensuring responsible, reliable AI deployments.

In addition, Cloudera supports the entire data lifecycle, managing data from end-to-end and running business intelligence, AI, machine learning and streaming analytics on the same data, without data silos. This becomes very attractive to organisations that have to be strict about data compliance and fair use, as well as those wanting to experiment with various AI and analytics advancements.

The two core parts of Cloudera's AI play

Priyank Patel, Cloudera's VP of Product Management breaks down Cloudera's fundamental AI innovation. He says, "When we think of AI in Cloudera, we have it in two pillars. There is AI with the Cloud Data Platform, which is where we want to make the best platform that is used for building and running AI applications for our customers. The second is AI Assistant, which is AI infused across our platform with co-pilots."

In June 2024 Cloudera completed the acquisition of Verta's Operational AI Platform, with the intention of strengthening and accelerating the company's operational AI capabilities overall. As the executives emphasised, this acquisition and other initiatives are all aimed at supporting businesses to create AI applications and drive GenAI initiatives on a foundation of trusted enterprise data.

The AI assistants in-built into Cloudera's platform are the SQL AI Assistant, AI Chatbot in Cloudera Data Visualization, and Cloudera Copilot for Cloudera Machine Learning. Each is designed to empower customers to accelerate the development of data, analytics and AI business applications and gain more valuable data analysis. The assistants can offer helpful insights, additional options and flag any potential issues so businesses can maximise their capabilities of the Cloudera platform.

Patel alludes to ongoing innovation saying, "Internally within Cloudera, there are a few pilot projects that we started out last year and have now moved forward, particularly on Cloudera support and the internal operations of Cloudera."

He says, "We have a support team that is working with our enterprise customers. Traders of Cloudera can obviously benefit from AI models, helping synthesise the large amount of the knowledge bases and support tickets that are there to quickly arrive at answers or resolutions for our customers. And so we have seen success with internal projects that were there as well."

Announcing new AMPs

This month, Cloudera has announced new Accelerators for ML Projects (AMPs), designed to reduce time-to-value for enterprise AI use cases. As reported by the company, the new additions focus on providing enterprises with AI techniques and examples within Cloudera that can assist AI integration and drive more impactful results.

The latest AMPs and updates include:

• Fine-tuning studio: Provides users with an all-encompassing application and "ecosystem" for managing, fine tuning, and evaluating LLMs.
• RAG with knowledge graph: A demonstration of how to power a RAG (retrieval augmented generation) application with a knowledge graph to capture relationships and context not easily accessible by vector stores alone.
• PromptBrew: Offers AI-powered assistance to create high-performing and reliable prompts via a simple user interface.
• Chat with your documents: Building upon the previous LLM Chatbot Augmented with Enterprise Data AMP, this accelerator enhances the responses of the LLM using context from an internal knowledge base created from the documents uploaded by the user.

Dipto Chakravarty, Chief Product Officer at Cloudera, comments, "In today's environment, enterprises are constrained with time and resources to get AI projects off the ground. Our AMPs are catalysts to fast-track AI projects from concept to reality with pre-built solutions and working examples, ensuring that use cases are dependable and cost effective, while reducing development time. This enables enterprises to swiftly experience the productivity gains and efficiencies that come from AI initiatives."

Understanding the massive opportunity of AI

Cloudera is emphasising its AI play through the undeniable need for robust data infrastructure, not to mention the growing role of AI assistants, and there is a significant opportunity in this area.

Cloudera's The State of Enterprise AI and Modern Data Architecture report highlighted that although a high majority of enterprises are adopting AI in some capacity (88%), many are still lacking the necessary data infrastructure and employee skills to truly benefit from it.

A key finding of the survey is that all AI efforts are ultimately tied back to trustworthy data. While 94% of respondents said that they trust their data, 55% also said they would rather get a root canal than try to access all of their company's data. This frustration is driven by challenges including contradictory datasets (49%), an inability to govern data across platforms (36%), and too much data (35%).

From automating and streamlining IT processes, to building chatbots capable of supporting front-line customer needs quickly and effectively, to leveraging analytics to foster better decision-making, the survey revealed the top use cases for AI included improving customer experiences (60%), increasing operational efficiency (57%), and expediting analytics (51%).

Commenting on these findings, Cloudera Chief Strategy Officer, Abhas Ricky, says, "For the majority of companies, the quality of their data is not great, it's distributed across various infrastructures and not documented in an efficient manner, and we're seeing the fallout from that presented in the challenges identified by the survey."

Ricky continues, "Managing data where it resides is the most important thing when it comes to adopting AI - being able to run models in a cost efficient manner where that data already lives. Instead of bringing the data to the models, enterprises are starting to realise the advantages of bringing AI models to their data."