#tensorrt

3 days ago

Just saw TensorRT Edge‑LLM crush chain‑of‑thought reasoning on‑device, unlocking Physical AI for autonomous cars. Imagine real‑time MATH500 puzzles solved in the car! Dive in to see how edge LLMs are changing the game. #TensorRT #EdgeLLM #ChainOfThought

🔗 aidailypost.com/news/tensorr...

0 1 0 0

Till Beemelmanns

@tillbeemelmanns.bsky.social

3 months ago

Depth Anything V3 now runs in real-time with our karl.'s camera data predicting metric depth from monocular images. With TensorRT optimization, we’ve wrapped it into a ROS2 inference node that’s ready to drop into your stack.

Github: github.com/ika-rwth-aac...

#Robotics #ROS2 #TensorRT

2 1 0 0

NVIDIA Blog

@blogs.nvidia.com.web.brid.gy

3 months ago

Cheers to AI: ADAM Robot Bartender Makes Drinks at Vegas Golden Knights Game In Las Vegas’s T-Mobile Arena, fans of the Golden Knights are getting more than just hockey — they’re getting a taste of the future. ADAM, a robot developed with NVIDIA Isaac libraries, is pouring drinks and turning heads in one of the NHL’s most exciting venues. ADAM, short for Automated Dual Arm Mixologist, was developed Read Article

0 0 0 0

Ahmandonk

@ahmandonk.bsky.social

3 months ago

📰 NVIDIA & Mistral AI Umumkan Kolaborasi untuk Keluarga Model Terbuka Mistral 3

👉 Baca artikel lengkap di sini: ahmandonk.com/2025/12/04/nvidia-mistra...

#ai #edge #mistral-ai #mistral3 #nemo #nvidia #open-source #supercomputing #tensorrt

2 0 0 0

NVIDIA Blog

@blogs.nvidia.com.web.brid.gy

3 months ago

NVIDIA Partners With Mistral AI to Accelerate New Family of Open Models Today, Mistral AI announced the Mistral 3 family of open-source multilingual, multimodal models, optimized across NVIDIA supercomputing and edge platforms. Mistral Large 3 is a mixture-of-experts (MoE) model — instead of firing up every neuron for every token, it only activates the parts of the model with the most impact. The result is efficiency Read Article

0 0 0 0

Sébastien Favre Lecca

@favrelecca.bsky.social

4 months ago

Where to place the H-E Selector™

[Input → Encoder → Decoder]
↳ NVML / C, R, IG / H-E Selector → stop / control / feedback → Output
The Selector orchestrates energy, coherence & output quality.
#TensorRT #CUDA #AIoptimization #LawE

0 0 0 0

Sébastien Favre Lecca

@favrelecca.bsky.social

4 months ago

Energy early-stop (NVML threshold)

If energy consumption > ε → STOP.
A model learns not only to speak, but to stop when energy waste begins.
A new kind of eloquence: efficient reasoning.
#CUDA #TensorRT #AIOptimization #LawE

0 0 0 0

Tuhin Babu

@tuhinislam817.bsky.social

5 months ago

StreamDiffusion just hit an impossible speed for Stable Diffusion—generating real-time video instantly. The waiting is over. The future is NOW.

Watch the full speed test below.

#StreamDiffusion #AIGenerator #FutureOfAI #TensorRT

0 0 0 0

LLMs

@llms.activitypub.awakari.com.ap.brid.gy

6 months ago

Inference performance is critical, as it directly influences the economics of an AI factory. The higher the throughput of AI factory infrastructure, the more tokens it can produce at a high speed — increasing revenue, driving down total cost of ownership (TCO) and enhancing the system’s overall productivity. Less than half a year since its debut at NVIDIA GTC, the NVIDIA GB300 NVL72 rack-scale system — powered by the NVIDIA Blackwell Ultra architecture — set records on the new reasoning inference benchmark in MLPerf Inference v5.1, delivering up to 1.4x more DeepSeek-R1 inference throughput compared with NVIDIA Blackwell-based GB200 NVL72 systems. Blackwell Ultra builds on the success of the Blackwell architecture, with the Blackwell Ultra architecture featuring 1.5x more NVFP4 AI compute and 2x more attention-layer acceleration than Blackwell, as well as up to 288GB of HBM3e memory per GPU. The NVIDIA platform also set performance records on all new data center benchmarks added to the MLPerf Inference v5.1 suite — including DeepSeek-R1, Llama 3.1 405B Interactive, Llama 3.1 8B and Whisper — while continuing to hold per-GPU records on every MLPerf data center benchmark. ## **Stacking It All Up** Full-stack co-design plays an important role in delivering these latest benchmark results. Blackwell and Blackwell Ultra incorporate hardware acceleration for the NVFP4 data format — an NVIDIA-designed 4-bit floating point format that provides better accuracy compared with other FP4 formats, as well as comparable accuracy to higher-precision formats. NVIDIA TensorRT Model Optimizer software quantized DeepSeek-R1, Llama 3.1 405B, Llama 2 70B and Llama 3.1 8B to NVFP4. In concert with the open-source NVIDIA TensorRT-LLM library, this optimization enabled Blackwell and Blackwell Ultra to deliver higher performance while meeting strict accuracy requirements in submissions. Large language model inference consists of two workloads with distinct execution characteristics: 1) context for processing user input to produce the first output token and 2) generation to produce all subsequent output tokens. A technique called disaggregated serving splits context and generation tasks so each part can be optimized independently for best overall throughput. This technique was key to record-setting performance on the Llama 3.1 405B Interactive benchmark, helping to deliver a nearly 50% increase in performance per GPU with GB200 NVL72 systems compared with each Blackwell GPU in an NVIDIA DGX B200 server running the benchmark with traditional serving. NVIDIA also made its first submissions this round using the NVIDIA Dynamo inference framework. NVIDIA partners — including cloud service providers and server makers — submitted great results using the NVIDIA Blackwell and/or Hopper platform. These partners include Azure, Broadcom, Cisco, CoreWeave, Dell Technologies, Giga Computing, HPE, Lambda, Lenovo, Nebius, Oracle, Quanta Cloud Technology, Supermicro and the University of Florida. The market-leading inference performance on the NVIDIA AI platform is available from major cloud providers and server makers. This translates to lower TCO and enhanced return on investment for organizations deploying sophisticated AI applications. Learn more about these full-stack technologies by reading the NVIDIA Technical Blog on MLPerf Inference v5.1. Plus, visit the NVIDIA DGX Cloud Performance Explorer to learn more about NVIDIA performance, model TCO and generate custom reports. Categories: Data Center | Hardware Tags: NVIDIA Blackwell Platform | TensorRT

NVIDIA Blackwell Ultra Sets the Bar in New MLPerf Inference Benchmark Inference performance is critical, as it directly influences the economics of an AI factory. The higher the throughput of AI fa...

#Data #Center #Hardware #NVIDIA #Blackwell #Platform #TensorRT

Origin | Interest | Match

0 0 0 0

deepseek

@deepseek.activitypub.awakari.com.ap.brid.gy

6 months ago

Inference performance is critical, as it directly influences the economics of an AI factory. The higher the throughput of AI factory infrastructure, the more tokens it can produce at a high speed — increasing revenue, driving down total cost of ownership (TCO) and enhancing the system’s overall productivity. Less than half a year since its debut at NVIDIA GTC, the NVIDIA GB300 NVL72 rack-scale system — powered by the NVIDIA Blackwell Ultra architecture — set records on the new reasoning inference benchmark in MLPerf Inference v5.1, delivering up to 1.4x more DeepSeek-R1 inference throughput compared with NVIDIA Blackwell-based GB200 NVL72 systems. Blackwell Ultra builds on the success of the Blackwell architecture, with the Blackwell Ultra architecture featuring 1.5x more NVFP4 AI compute and 2x more attention-layer acceleration than Blackwell, as well as up to 288GB of HBM3e memory per GPU. The NVIDIA platform also set performance records on all new data center benchmarks added to the MLPerf Inference v5.1 suite — including DeepSeek-R1, Llama 3.1 405B Interactive, Llama 3.1 8B and Whisper — while continuing to hold per-GPU records on every MLPerf data center benchmark. ## **Stacking It All Up** Full-stack co-design plays an important role in delivering these latest benchmark results. Blackwell and Blackwell Ultra incorporate hardware acceleration for the NVFP4 data format — an NVIDIA-designed 4-bit floating point format that provides better accuracy compared with other FP4 formats, as well as comparable accuracy to higher-precision formats. NVIDIA TensorRT Model Optimizer software quantized DeepSeek-R1, Llama 3.1 405B, Llama 2 70B and Llama 3.1 8B to NVFP4. In concert with the open-source NVIDIA TensorRT-LLM library, this optimization enabled Blackwell and Blackwell Ultra to deliver higher performance while meeting strict accuracy requirements in submissions. Large language model inference consists of two workloads with distinct execution characteristics: 1) context for processing user input to produce the first output token and 2) generation to produce all subsequent output tokens. A technique called disaggregated serving splits context and generation tasks so each part can be optimized independently for best overall throughput. This technique was key to record-setting performance on the Llama 3.1 405B Interactive benchmark, helping to deliver a nearly 50% increase in performance per GPU with GB200 NVL72 systems compared with each Blackwell GPU in an NVIDIA DGX B200 server running the benchmark with traditional serving. NVIDIA also made its first submissions this round using the NVIDIA Dynamo inference framework. NVIDIA partners — including cloud service providers and server makers — submitted great results using the NVIDIA Blackwell and/or Hopper platform. These partners include Azure, Broadcom, Cisco, CoreWeave, Dell Technologies, Giga Computing, HPE, Lambda, Lenovo, Nebius, Oracle, Quanta Cloud Technology, Supermicro and the University of Florida. The market-leading inference performance on the NVIDIA AI platform is available from major cloud providers and server makers. This translates to lower TCO and enhanced return on investment for organizations deploying sophisticated AI applications. Learn more about these full-stack technologies by reading the NVIDIA Technical Blog on MLPerf Inference v5.1. Plus, visit the NVIDIA DGX Cloud Performance Explorer to learn more about NVIDIA performance, model TCO and generate custom reports. Categories: Data Center | Hardware Tags: NVIDIA Blackwell Platform | TensorRT

NVIDIA Blackwell Ultra Sets the Bar in New MLPerf Inference Benchmark Inference performance is critical, as it directly influences the economics of an AI factory. The higher the throughput of AI fa...

#Data #Center #Hardware #NVIDIA #Blackwell #Platform #TensorRT

Origin | Interest | Match

0 0 0 0

Hardware Premium

@hardwarepremium.bsky.social

8 months ago

🚀 NVIDIA y Black Forest Labs lanzan FLUX.1 Kontext, la nueva joya para editar y generar imágenes con IA en tiempo real desde tu gráfica RTX. ¡Una pasada para creadores!

#IA #ediciondeimagenes #NVIDIA #TensorRT #creatividadAI

0 1 1 0

LLMs

@llms.activitypub.awakari.com.ap.brid.gy

8 months ago

Эффективный инференс множества LoRA адаптеров LoRA — популярный метод дообучения больших моделей на небольши...

#multilora #offline #inference #async #inference #vllm #TensorRT-LLM #tensorrt #peft #inference #benchmark

Origin | Interest | Match

0 0 0 0

𝙳𝚘𝚝𝚊𝚗 𝙷𝚘𝚛𝚘𝚟𝚒𝚝𝚜 #CNCFAmbassador #OpenSearchAmbassador

@horovits.bsky.social

8 months ago

Can you run #ML on #kubernetes on tractors in the field??
Aparently yes! @berlinbuzzwords.de talk on how impelementing this precision agriculture use case with #CloudNative #edgeComputing reduced their time to insight from a month to a less than a day!
@kubernetes.io #k3s #TensorRT

3 1 0 0

Hardware Premium

@hardwarepremium.bsky.social

9 months ago

🚀 NVIDIA duplica el rendimiento de Stable Diffusion 3.5 con TensorRT para RTX y reduce el consumo de memoria en un 40%. ¡La IA generativa nunca fue tan rápida! 💥 #NVIDIA #TensorRT #StableDiffusion #AI #RTX

0 0 1 0

Finland

@finland.activitypub.awakari.com.ap.brid.gy

11 months ago

Math Test? No Problems: NVIDIA Team Scores Kaggle Win With Reasoning Model The final days of the ...

blogs.nvidia.com/blog/reasoning-ai-math-o...

#Generative #AI #Artificial #Intelligence #Inference #NVIDIA #NeMo #Open #Source #TensorRT

Event Attributes

0 0 0 0

Finland

@finland.activitypub.awakari.com.ap.brid.gy

11 months ago

Math Test? No Problems: NVIDIA Team Scores Kaggle Win With Reasoning Model The final days of the ...

blogs.nvidia.com/blog/reasoning-ai-math-o...

#Generative #AI #Artificial #Intelligence #Inference #NVIDIA #NeMo #Open #Source #TensorRT

Result Details

0 0 0 0

Marketing News

@marketingnews.bsky.social

1 year ago

Bing optimizes search speed with TensorRT-LLM, cutting model latency by 36 percent Microsoft's Bing search engine implements TensorRT-LLM optimization, reducing inference time and operational costs for language models.

Bing optimizes search speed with TensorRT-LLM, cutting model latency by 36 percent: Microsoft's Bing search engine implements TensorRT-LLM optimization, reducing inference time and operational costs for language models. #Bing #TensorRT #AI #MachineLearning #SearchEngine

1 0 0 0

PPC Land

@ppc.land

1 year ago

Bing optimizes search speed with TensorRT-LLM, cutting model latency by 36 percent Microsoft's Bing search engine implements TensorRT-LLM optimization, reducing inference time and operational costs for language models.

Bing optimizes search speed with TensorRT-LLM, cutting model latency by 36 percent: Microsoft's Bing search engine implements TensorRT-LLM optimization, reducing inference time and operational costs for language models. #Bing #TensorRT #AI #MachineLearning #SearchEngine

1 0 0 0

GTM🍁 TECH

@get-the-marvel.bsky.social

1 year ago

Build Your Custom AI Applications with Snowflake + NVIDIA | Get The Marvel YouTube video by Get The Marvel

youtu.be/0DVV-cZyrtI #Snowflake #NVIDIA #AI #CustomAI #MachineLearning #DataScience #DataManagement #SnowflakeCortexAI #SnowflakeArctic #TensorRT #LLM #NIM #Quantiphi #NeMoFramework #DataDrivenAI #TechNews #TechUpdates #TechnologyTrends #TechInnovation #TechWorld #TechTalk #TechCommunity #TechInsider

1 0 0 0

@onedigital.bsky.social

1 year ago

Guía completa para usar ChatRTX de NVIDIA - OneDigital Descubre cómo utilizar ChatRTX de NVIDIA para obtener respuestas precisas y contextualmente relevantes de manera rápida y segura. #onedigital #one_digital #NVIDIA #ChatRTX #IA #TensorRT #aceleraciónRT...

Guía completa para usar ChatRTX de NVIDIA

onedigital.mx/2024/06/09/g...

Descubre cómo utilizar ChatRTX de NVIDIA para obtener respuestas precisas y contextualmente relevantes de manera rápida y segura. #onedigital #one_digital #NVIDIA #ChatRTX #IA #TensorRT #aceleraciónRTX #modelosdeIA #Chatbot

0 0 0 0

@vincentquezada.bsky.social

1 year ago

Guía completa para usar ChatRTX de NVIDIA - OneDigital Descubre cómo utilizar ChatRTX de NVIDIA para obtener respuestas precisas y contextualmente relevantes de manera rápida y segura. #onedigital #one_digital #NVIDIA #ChatRTX #IA #TensorRT #aceleraciónRT...

Guía completa para usar ChatRTX de NVIDIA

onedigital.mx/2024/06/09/g...

Descubre cómo utilizar ChatRTX de NVIDIA para obtener respuestas precisas y contextualmente relevantes de manera rápida y segura. #onedigital #one_digital #NVIDIA #ChatRTX #IA #TensorRT #aceleraciónRTX #modelosdeIA #Chatbot

0 0 0 0

Çevre Gazetesi

@cevregazetesi.bsky.social

1 year ago

UL, PROCYON AI GÖRÜNTÜ OLUŞTURMA KARŞILAŞTIRMA ÖLÇÜTÜNÜ YAYINLAYACAK
#AI #DirectML @intel #OpenVINO @nvidia #TensorRT #ONNX @UL_Benchmarks

UL Procyon, Üst Düzey Donanım Performansını Değerlendirmek için Yapay Zeka Görüntü Oluşturma Benchmark’ını Tanıttı

yalovacevre.com/ul-procyon-a...