Heiko Hotz

Oh, hello! May I meet you?

I really like tiny (you could even say "nano") bananas. They are so full of flavour 😋

Advanced version of Gemini with Deep Think officially achieves gold-medal standard at the International Mathematical Olympiad Our advanced model officially achieved a gold-medal level performance on problems from the International Mathematical Olympiad (IMO), the world’s most prestigious competition for young...

Read more about it here: deepmind.google/discover/blo...

This year, the advanced Gemini model operated end-to-end in natural language, producing rigorous mathematical proofs directly from the official problem descriptions – all within the 4.5-hour competition time limit.

At IMO 2024, AlphaGeometry and AlphaProof required experts to first translate problems from natural language into domain-specific languages, such as Lean, and vice-versa for the proofs.

This achievement is a significant advance over last year’s result.

An advanced version was able to solve 5 out of 6 problems.

Official results are in - Gemini achieved gold-medal level in the International Mathematical Olympiad! 🏆

Let AI Tune Your Voice Assistant | Towards Data Science A practical guide to automating prompt engineering for voice assistants

The result? Saves tons of time, money, and builds super reliable voice assistants that have undergone a rigorous evaluation process. No more guesswork! 📈
Full details + code here: towardsdatascience.com/let-ai-tune-...

GOOD NEWS: I built an #AutomatedPromptEngineering (APE) pipeline specifically for voice AI! 🤖✨ My new @towardsdatascience blog post dives deep.
What it does:
✅ Creates diverse audio tests
✅ Automates performance eval
✅ LLM optimizes your prompts! 👇

Evaluating voice-driven agents got you pulling your hair out? 😩 Evaluating voice agents is WILD. Accents, noise, weird speech... how do you even test?! Manual prompt engineering for that? A total nightmare. 👇

Thanks for sharing @towardsdatascience.com 🤗

WWDC INTERVIEW: Craig & Joz on Why Siri's Not Ready, AI Vision and iPadOS Shocker! YouTube video by Tom’s Guide

Tom's Guide: www.youtube.com/watch?v=Pt3q...

Apple Execs Defend Siri Delays, AI Plan and Apple Intelligence | WSJ YouTube video by The Wall Street Journal

WSJ interview: www.youtube.com/watch?v=NTLk...

Building a v1 GenAI app on an existing platform while overhauling the foundation for a better 'V2' is a common strategy. But explaining this to everyday consumers is challenging. These kinds of interviews really help communicate that effectively. What did you think?

To give an example, right out of the gate she asks, "𝙇𝙖𝙨𝙩 𝙮𝙚𝙖𝙧 𝙮𝙤𝙪 𝙖𝙣𝙣𝙤𝙪𝙣𝙘𝙚𝙙 𝙖 𝙨𝙢𝙖𝙧𝙩𝙚𝙧 𝘼𝙄-𝙙𝙧𝙞𝙫𝙚𝙣 𝙎𝙞𝙧𝙞. 𝙒𝙝𝙚𝙧𝙚 𝙞𝙨 𝙨𝙝𝙚?"
From a developer's point of view, Apple's answers made a lot of sense: a 'V1' worked, but didn't meet their high quality/reliability standards when users went 'off the beaten path'.

This year, however, Craig Federighi and Greg Joswiak were interviewed by other outlets, including Tom's Guide, TechRadar, and The Wall Street Journal. I particularly liked Joanna Stern's interview and her style: direct, concise, and challenging.

While it's not fair to characterise Gruber as an "Apple fanboy," I consistently found his questions too long-winded and too softball. By the end, it often felt (to me, at least) like just a few folk were a bit too cosy on stage.

WWDC interviews with Apple executives just got a facelift - and it is refreshing!
For years, high-level Apple execs would come to John Gruber's (from Daring Fireball) Talk Show at WWDC. I often found these interviews less than insightful, and sometimes even annoying.

i definitely hear you on that one 😅 out of curiosity - what are the benefits you are looking to gain from an agent framework (in general)?

Introducing Gemini-Powered Slide Creation by Voice!

In this quick demo, I’ve integrated a “Slide Creation Agent” into my personal project, Project Pastra. Watch how it effortlessly generates slides based on voice instructions.

GitHub - heiko-hotz/gemini-multimodal-live-dev-guide: A developer guide for Gemini's Multimodal Live API A developer guide for Gemini's Multimodal Live API - heiko-hotz/gemini-multimodal-live-dev-guide

Not perfect by any means, but much better already than "traditional" voice assistants, and we are only at the beginning of this journey.

You can try it yourself with the Developer Guide for Gemini's Multimodal Live API 🤗

github.com/heiko-hotz/g...

I believe that multimodal AI models have the potential to change that. They allow me to speak much more freely about what I want them to do and oftentimes they understand and execute in the way I expected them to.

But soon I realised that these voice assistants still require a rigid syntax: I would have to phrase commands in a very specific way for the voice assistant to understand what I meant.

To me it was a magical moment when I got my first Amazon Echo in 2015 and could just shout words into the air and got a response.

Gemini's Multimodal Live API with Calendar Tool YouTube video by Heiko Hotz

Multimodal AI models have the potential to finally deliver on the dream of language being the ultimate human-computer interface 🎙️

youtu.be/0OEDHAjY6LM

Fade Out YouTube video by Secret Level

Fade Out. Directed by Jason Zada. Created with Google’s Veo 2.

youtu.be/9yQXkdA3u8k?...

GitHub - heiko-hotz/gemini-multimodal-live-dev-guide: A developer guide for Gemini's Multimodal Live API A developer guide for Gemini's Multimodal Live API - heiko-hotz/gemini-multimodal-live-dev-guide

Check it out and start building your own voice assistant 🤗

github.com/heiko-hotz/g...

It is a full-featured web application for real-time conversations with audio and video input, memory, and tool use! And it works great on mobile phones, too.

But Gemini's Multimodal Live API actually lets you build a comparable experience today! I'm proud to share a step-by-step developer guide that will help you build Project Pastra.