Here are some selected quotes:
LLMs enhanced with reasoning and memory - Incredible Universal Personal Assistants
…And in the case of chatbots and those kinds of systems, ultimately, they will become these incredible universal personal assistants that you use multiple times during the day for really useful and helpful things across your daily lives.
From what books to read to recommendations on maybe live events and things like that to booking your travel to planning trips for you to assisting you in your everyday work. And I think we’re still far away from that with the current chatbots, and I think we know what’s missing: things like planning and reasoning and memory, and we are working really hard on those things. And I think what you’ll see in maybe a couple of years’ time is today’s chatbots will look trivial by comparison to I think what’s coming in the next few years.
LLMs with Tools
Actually, there’s a whole branch of research going into what’s called tool use. This is the idea that these large language models or large multimodal models, they’re expert at language, of course, and maybe a few other capabilities, like math and possibly coding. But when you ask them to do something specialized, like fold a protein or play a game of chess or something like this, then actually what they end up doing is calling a tool, which could be another AI system, that then provides the solution or the answer to that particular problem. And then that’s transmitted back to the user via language or pictorially through the central large language model system. So it may be actually invisible to the user because, to the user, it just looks like one big AI system that has many capabilities, but under the hood, it could be that actually the AI system is broken down into smaller ones that have specializations.
LLMs critiquing themselves
So these systems need to have a better understanding of the media they’re dealing with. And maybe also give these systems the ability to reason and plan because then they could potentially turn that on their own outputs and critique themselves. And again, this is something we have a lot of experience in in games programs. They don’t just output the first move that you think of in chess or Go. You actually plan and do some search around that and then back up. And sometimes they change their minds and switch to a better move. And you could imagine some process like that with words and language as well.
See also:
