Warning: Some posts on this platform may contain adult material intended for mature audiences only. Viewer discretion is advised. By clicking ‘Continue’, you confirm that you are 18 years or older and consent to viewing explicit content.
You see the irony right? I genuinely can’t fathom your intent when telling this story, but it is an absolutely stellar example.
Yes, I did mean for it to be an example.
And yes, I do think that correctly framing a question is crucial whether you’re dealing with a person or an LLM. But I was elaborating on whether a person’s process of answering a question is fundamentally similar to an LLM’s process. And this is one way that it’s noticeably different. A person will size up who is asking, what they’re asking, and how they’re asking it… and consider whether they should actually answer the exact question that was asked or suggest a better question instead.
You can certainly work around it, as the asker, but it does require deliberate disambiguation. I think programmers are used to doing that, so it may feel like not that big of a deal, but if you start paying attention to how often people are tossing around half-formed questions or statements and just expecting the recipient to fill in the gaps… It’s basically 100% of the time.
We’re fundamentally social creatures first, and intelligent creatures second. (Or third, or not at all, depending.) We think better as groups. If you give 10 individuals a set of difficult questions, they’ll bomb almost all of them. If you give the questions to a group of 10, they’ll get almost all of them right. (There’s several You Are Not So Smart episodes on this, but the main one is 111.)
Asking a question to an LLM is just completely different from asking a person. We’re not optimized for correctly filling out scantron sheets as individuals, we’re optimized for brainstorming ideas and pruning them as a group.
Yes, I did mean for it to be an example.
And yes, I do think that correctly framing a question is crucial whether you’re dealing with a person or an LLM. But I was elaborating on whether a person’s process of answering a question is fundamentally similar to an LLM’s process. And this is one way that it’s noticeably different. A person will size up who is asking, what they’re asking, and how they’re asking it… and consider whether they should actually answer the exact question that was asked or suggest a better question instead.
You can certainly work around it, as the asker, but it does require deliberate disambiguation. I think programmers are used to doing that, so it may feel like not that big of a deal, but if you start paying attention to how often people are tossing around half-formed questions or statements and just expecting the recipient to fill in the gaps… It’s basically 100% of the time.
We’re fundamentally social creatures first, and intelligent creatures second. (Or third, or not at all, depending.) We think better as groups. If you give 10 individuals a set of difficult questions, they’ll bomb almost all of them. If you give the questions to a group of 10, they’ll get almost all of them right. (There’s several You Are Not So Smart episodes on this, but the main one is 111.)
Asking a question to an LLM is just completely different from asking a person. We’re not optimized for correctly filling out scantron sheets as individuals, we’re optimized for brainstorming ideas and pruning them as a group.
If you fed that information into one I bet you would get different answers.
That is information that isn’t available to it generally.