The Robot Report #1 â Reveries
đ a linked post to
randsinrepose.com »
—
originally shared here on
Whenever I talk about a knowledge win via robots on the socials or with humans, someone snarks, âWell, how do you know itâs true? How do you know the robot isnât hallucinating?â Before I explain my process, I want to point out that I donât believe humans are snarking because they want to know the actual answer; I think they are scared. They are worried about AI taking over the world or folks losing their job, and while these are valid worries, itâs not the robotâs responsibility to tell the truth; itâs your job to understand what is and isnât true.
Youâre being changed by the things you see and read for your entire life, and hopefully, youâve developed a filter through which this information passes. Sometimes, it passes through without incident, but other times, itâs stopped, and you wonder, âIs this true?â
Knowing when to question truth is fundamental to being a human. Unfortunately, weâve spent the last forty years building networks of information that have made it pretty easy to generate and broadcast lies at scale. When you combine the internet with the fact that many humans just want their hopes and fears amplified, you can understand why the real problem isnât robots doing it better; itâs the humans getting worse.
Iâm working on an extended side quest and in the past few hours of pairing with ChatGPT, Iâve found myself constantly second guessing a large portion of the decisions and code that the AI produced.
This article pairs well with this one I read today about a possible social exploit that relies on frequently hallucinated package names.
Bar Lanyado noticed that LLMs frequently hallucinate the names of packages that donât exist in their answers to coding questions, which can be exploited as a supply chain attack.
He gathered 2,500 questions across Python, Node.js, Go, .NET and Ruby and ran them through a number of different LLMs, taking notes of any hallucinated packages and if any of those hallucinations were repeated.
One repeat example was âpip install huggingface-cliâ (the correct package is âhuggingface[cli]â). Bar then published a harmless package under that name in January, and observebd 30,000 downloads of that package in the three months that followed.
Iâll be honest: during my side quest here, Iâve 100% blindly run npm install
on packages without double checking official documentation.
These large language models truly are mirrors to our minds, showing all sides of our personalities from our most fit to our most lazy.