Databricks enhances Mosaic AI to advance generative AI systems

Databricks has announced a series of advancements to its Mosaic AI platform aimed at aiding customers in the development of production-quality Generative AI applications. These improvements focus on supporting the construction of compound AI systems, enhancing model quality, and introducing new AI governance tools.

Many organisations face significant challenges in scaling Generative AI projects from pilot stages to full-scale production due to concerns over privacy, quality, and cost. Despite advancements in foundational models, issues such as inaccuracies, unsafe responses, and the potential exposure of confidential data persist. Recognising these hurdles, Databricks is encouraging the use of compound AI systems, which incorporate multiple elements—including various models, retrievers, and vector databases—to achieve higher production standards.

"We believe that compound AI systems will be the best way to maximise the quality, reliability, and measurement of AI applications going forward, and may be one of the most important trends in AI in 2024," said Matei Zaharia, Co-founder and CTO at Databricks. "Databricks is uniquely positioned to capitalise on these trends with the investments we're making to improve quality, augmenting the model with real-time data and agents and tools to give it new capabilities it has little knowledge of."

Databricks is launching several new components as part of Mosaic AI, including the Mosaic AI Agent Framework, Mosaic AI Agent Evaluation, Mosaic AI Tools Catalog, Mosaic AI Model Training, and the Mosaic AI Gateway.

Mosaic AI Agent Framework aims to simplify the creation of robust RAG (Retrieval-Augmented Generation) applications by enabling developers to use foundational models alongside enterprise data quickly and safely. Mosaic AI Agent Evaluation, an AI-assisted tool, evaluates the quality of RAG applications, allowing for rapid iteration and redeployment. Collectively, these tools are designed to assist organisations in achieving production-quality Generative AI solutions.

The Mosaic AI Tools Catalog allows companies to manage, share, and register tools using Databricks Unity Catalog, thereby ensuring secure and governed model usage. Mosaic AI Model Training enables the fine-tuning of open-source foundation models with private data, making these models more accurate for specific use cases and reducing costs due to lower computational resources required.

Mosaic AI Gateway provides a unified interface for querying, managing, and deploying a variety of models. It supports usage tracking, rate limits, and safety filters, helping organisations maintain oversight and quality across their AI applications.

Several companies have already begun leveraging these new capabilities. Denis Kamotsky, Principal Software Engineer at Corning, noted the benefits: "We built an AI research assistant using Databricks Mosaic AI Agent Framework to index hundreds of thousands of documents including US patent office data. Having our LLM-powered assistant respond to questions with high accuracy was extremely important to us."

Tom Thomas, VP of Analytics at FordDirect, also highlighted the advantages: "We needed to create a unified chatbot to help our dealers assess their performance, inventory, trends, and customer engagement metrics. Databricks Mosaic AI Agent Framework allowed us to integrate our proprietary data and documentation into our Generative AI solution that uses RAG."

Kenan Colson, VP Data & AI at Lippert, remarked, "Mosaic AI Agent Framework has been a game-changer for us, because it allowed us to evaluate the results of our GenAI applications and demonstrate the accuracy of our outputs while maintaining complete control over our data sources."