Matei Zaharia

Reranking in Mosaic AI Vector Search for Faster, Smarter Retrieval in RAG Agents Boost RAG agent quality with reranking—deliver more relevant answers in less time with a single parameter in Mosaic AI Vector Search.

We built a thing! The Databricks Reranker is now in Public Preview. It's as easy as changing the arguments to your vector search call, and doesn't require any additional setup.

Read more: www.databricks.com/blog/reranki...

This is a joint effort across our engineering and research teams, based on new tuning methods we developed like TAO and ALHF. I think this type of declarative development is the future of AI -- help users build evals and auto-optimize based on those. Try it today!

Moreover, to steer your agents in Agent Bricks, you can use natural language feedback; the system optimizes all components of the agent (e.g. retrievers, guardrails, etc) based on it -- something we call Agent Learning from Human Feedback (ALHF). More feedback = better agent.

Agent Bricks automatically searches and combines the latest AI development techniques to give you a high-quality. It gets really great results quickly compared to DIY agents, e.g. state of the art performance on information extraction and question answering out of the box.

Introducing Agent Bricks: Auto-Optimized Agents Using Your Data Discover Agent Bricks by Databricks — a new way to build production-ready AI agents using your data. Automatically evaluate, optimize, and scale agents with higher accuracy and lower cost.

Excited to launch Agent Bricks, a new way to build auto-optimized agents on your tasks. Agent Bricks uniquely takes a *declarative* approach to agent development: you tell us what you want, and we auto-generate evals and optimize the agent.

www.databricks.com/blog/introdu...

Congrats Justine!

Introducing Apache Spark 4.0 Explore Apache Spark 4.0's key updates: advanced SQL features, improved Python support, enhanced streaming, and productivity boosts for big data analytics.

Apache Spark 4.0 is out with some huge improvements across the board. SQL’s much more powerful, Spark Connect makes it easier to run apps, new languages and more. It’s amazing to see the community still growing fast and releasing over 5000 patches in 4.0. www.databricks.com/blog/introdu...

#MLSys 2025 is next week! You can still register at mlsys.org.

The Power of Fine-Tuning on Your Data: Quick Fixing Bugs with LLMs via Never Ending Learning (NEL) Discover how fine-tuning small open-source LLMs on interaction data enables faster, cheaper, and more accurate code fixes with Databricks Quick Fix.

Nice results on never-ending learning for code editing. We believe that a lot of AI applications will be customizable this way (to every company's codebase, users, etc). The combined AI serving, data and MLOps environment on Databricks makes these easy to build.
www.databricks.com/blog/power-f...

MLflow Tracing | Introduction & Tutorial YouTube video by MLflow

🎥 New Video: Get Hands-On with MLflow Tracing!

In this video, @danliden.com walks through how #MLflow Tracing boosts observability in #GenAI apps—great for debugging, experimentation & organizing data workflows.

Watch now ➡️ www.youtube.com/watch?v=iRbB...

#opensource #oss

Key to TAO is a search and scoring process that leverages test-time compute only during training, and new RL methods and models from our team. More details, in our blog: www.databricks.com/blog/tao-usi...

TAO's trained model quality also scales with compute spent during training, not with human labeling effort, and the resulting models are always low inference cost.

Our new method, Test-time Adaptive Optimization (TAO), only needs input examples of a task and can outperform supervised fine-tuning on thousands of human-labeled examples. It brings efficient OSS models like Llama to the quality of expensive larger models.

TAO: Using test-time compute to train efficient LLMs without labeled data LIFT fine-tunes LLMs without labels using reinforcement learning, boosting performance on enterprise tasks.

Really cool result from the Databricks research team: You can tune LLMs for a task *without data labels*, using test-time compute and RL, and outperform supervised fine-tuning! Our new TAO method scales with compute to produce fast, high-quality models. www.databricks.com/blog/tao-usi...

The #MLSys2025 program is up and registration is open! Check out accepted papers at mlsys.org/virtual/2025... and sign up to attend at mlsys.org/Register.

Exciting news—MLflow 2.21.0 is live! 🎉 This release includes significant features, enhancements, and bug fixes to improve documentation, #GenAI prompt management, tracing & more.

🔗 Explore all the new features & improvements: mlflow.org/releases/2.2...

#opensource #oss #mlflow

🧵Introducing LangProBe: the first benchmark testing where and how composing LLMs into language programs affects cost-quality tradeoffs!

We find that, on avg across diverse tasks, smaller models within optimized programs beat calls to larger models at a fraction of the cost.

Improving Retrieval and RAG with Embedding Model Finetuning Fine-tune embedding models on Databricks to enhance retrieval and RAG accuracy with synthetic data—no manual labeling required.

We're probably a little too obsessed with zero-shot retrieval. If you have documents (you do), then you can generate synthetic data, and finetune your embedding. Blog post lead by @jacobianneuro.bsky.social shows how well this works in practice.

www.databricks.com/blog/improvi...

We're bringing in a new era of enterprise data management and agentic AI with SAP Business Data Cloud with Databricks.

✅ Unifies your SAP and non-SAP data

✅ Natively embeds Databricks technology

✅ AI agents streamline workflows

Learn more: sap.to/sapbdc

Sponsor registration is open for #MLSys 2025. We have the most submissions ever to MLSys so it promises to be a great conference! mlsys.org/Sponsors/spo...

Researchers open source Sky-T1, a ‘reasoning’ AI model that can be trained for less than $450 So-called reasoning AI models are becoming easier — and cheaper — to develop. On Friday, NovaSky, a team of researchers based out of UC Berkeley’s Sky Computing Lab, released Sky-T1-32B-Preview, a reasoning model that’s competitive with an earlier version…

Researchers open source Sky-T1, a ‘reasoning’ AI model that can be trained for less than $450

"Sky-T1-32B-Preview, our reasoning model that performs on par with o1-preview on popular reasoning and coding benchmarks."
That was quick! Is this already the Alpaca moment for reasoning models?
Source: novasky-ai.github.io/posts/sky-t1/

meta-llama/Llama-3.3-70B-Instruct · Hugging Face We’re on a journey to advance and democratize artificial intelligence through open source and open science.

Congrats to Meta on releasing Llama 3.3, a 70B model that matches the performance of Llama-405B! Open weight models are advancing so rapidly and the cost to get this performance is quickly going down. We're thrilled to let users serve & customize this on Databricks. huggingface.co/meta-llama/L...

Compound AI Systems, Inference-time Compute Meetup @ NeurIPS 2024 · Luma Meetup for practitioners and researchers working on and interested in compound AI systems, inference-time strategies and scaling laws, networks of networks,…

Compound AI Systems, Inference-time Compute Meetup @ NeurIPS 2024, with many AI luminaries as panelists. Poster submissions are open: lu.ma/q5r8b67t