Real-Time Response to Anomalies with Foundation Modeling - DRIVE Labs Ep. 37
10 449
10.6
NVIDIA1.7 млн
Следующее
17 дней – 8 9553:01Популярные
24 дня – 12 8011:41
108 дней – 3 8342:47Опубликовано 24 октября 2024, 17:15
This collaborative work between NVIDIA Research and Stanford University was awarded the Outstanding Paper Award at the 2024 Robotics: Science and Systems conference.
__________________________________
Foundation models, e.g., large language models (LLMs), trained on internet-scale data possess zero-shot generalization capabilities that make them a promising technology towards detecting and mitigating out-of-distribution failure modes of robotic systems.
In this work, we present a two-stage reasoning framework:
1: A fast binary anomaly classifier analyzes observations in an LLM embedding space, which may then trigger a slower fallback selection stage.
2: The fall back selection stage utilizes the reasoning capabilities of generative LLMs.
We show that our fast anomaly classifier outperforms autoregressive reasoning with state-of-the-art GPT models, even when instantiated with relatively small language models. This enables our runtime monitor to improve the trustworthiness of dynamic robotic systems, such as quadrotors or autonomous vehicles, under resource and time constraints.
Timestamps
00:00:28 - Real-time detection/response to anomalies is crucial for safety.
00:00:34 - NVIDIA & Stanford collaboration received Outstanding Paper Award at RSS 2024.
00:01:34 - Traditional systems often struggle with edge cases.
00:02:21 - NVIDIA’s platforms enable LLM deployment for real-world use cases.
00:02:59 - Autonomous robots tap into the reasoning power of LLMs.
00:03:14 - Leveraging generative AI within NVIDIA Omniverse.
Resources:
Project page: sites.google.com/view/aesop-ll...
RSS presentation: youtube.com/watch?v=2_XzYYLG2i...
Paper: arxiv.org/abs/2407.08735v1
Watch the full DRIVE Lab series: nvda.ws/3LsSgnH
Learn more about DRIVE Labs: nvda.ws/36r5c6t
Follow us on social:
Twitter: nvda.ws/3LRdkSs
LinkedIn: nvda.ws/3wI4kue
#NVIDIADRIVE #DRIVELabs #AutonomousVehicles #SelfDrivingCars #GenerativeAI #Simulation #LLM
__________________________________
Foundation models, e.g., large language models (LLMs), trained on internet-scale data possess zero-shot generalization capabilities that make them a promising technology towards detecting and mitigating out-of-distribution failure modes of robotic systems.
In this work, we present a two-stage reasoning framework:
1: A fast binary anomaly classifier analyzes observations in an LLM embedding space, which may then trigger a slower fallback selection stage.
2: The fall back selection stage utilizes the reasoning capabilities of generative LLMs.
We show that our fast anomaly classifier outperforms autoregressive reasoning with state-of-the-art GPT models, even when instantiated with relatively small language models. This enables our runtime monitor to improve the trustworthiness of dynamic robotic systems, such as quadrotors or autonomous vehicles, under resource and time constraints.
Timestamps
00:00:28 - Real-time detection/response to anomalies is crucial for safety.
00:00:34 - NVIDIA & Stanford collaboration received Outstanding Paper Award at RSS 2024.
00:01:34 - Traditional systems often struggle with edge cases.
00:02:21 - NVIDIA’s platforms enable LLM deployment for real-world use cases.
00:02:59 - Autonomous robots tap into the reasoning power of LLMs.
00:03:14 - Leveraging generative AI within NVIDIA Omniverse.
Resources:
Project page: sites.google.com/view/aesop-ll...
RSS presentation: youtube.com/watch?v=2_XzYYLG2i...
Paper: arxiv.org/abs/2407.08735v1
Watch the full DRIVE Lab series: nvda.ws/3LsSgnH
Learn more about DRIVE Labs: nvda.ws/36r5c6t
Follow us on social:
Twitter: nvda.ws/3LRdkSs
LinkedIn: nvda.ws/3wI4kue
#NVIDIADRIVE #DRIVELabs #AutonomousVehicles #SelfDrivingCars #GenerativeAI #Simulation #LLM
Свежие видео
5 дней – 701:01
8 дней – 133 6654:23
14 дней – 4374:04
20 дней – 70 98715:17
22 дня – 8135:33
23 дня – 2 1083:27Случайные видео
20 дней – 110 6505:29
144 дня – 64 9549:08
210 дней – 7657:43
364 дня – 4 6281:45
24.11.22 – 2 11830:48
20.03.07 – 59 4050:48
2 дня – 1 0060:26
4 дня – 645:26
5 дней – 5 7530:45
6 дней – 3 5620:23
8 дней – 6 0000:22
12 дней – 7 1450:08
17 дней – 66611:42
1 день – 5 0226:14