AI chatbot, ChatGPT, a creation of OpenAI, is under scrutiny due to its frequent inability to distinguish fact from fiction, leaving users often led astray by the information it provides.
The Warning Sign Often Ignored
OpenAI has highlighted on its homepage one of the many limitations of ChatGPT – it may sometimes provide incorrect information.
Although this warning holds true for several information sources, it brings to light a concerning trend. Users often disregard this caveat, assuming the data provided by ChatGPT to be factual.
Unreliable Legal Aid, The Case of Steven A. Schwartz
The misleading nature of ChatGPT came into stark focus when US lawyer Steven A.
Schwartz turned to the chatbot for case references in a lawsuit against Colombian airline Avianca. In a turn of events, all the cases the AI suggested turned out to be non-existent.
Despite Schwartz’s concerns about the veracity of the information, the AI reassured him of its authenticity.
Such instances raise questions about the chatbot’s reliability.
A Misunderstood Reliable Source?
The frequency with which users treat ChatGPT as a credible source of information calls for a wider recognition of its limitations.
Over the past few months, there have been several reports of people being misled by its fallacies, which have been largely inconsequential but nonetheless worrying.
One concerning instance involved a Texas A&M professor who used ChatGPT to verify if students’ essays were AI-generated.
ChatGPT confirmed, incorrectly, that they were, leading to the threat of failing the entire class. This incident underscores the risk of the misinformation that ChatGPT can spread, potentially leading to more serious consequences.
Cases like these do not entirely discredit the potential of ChatGPT and other AI chatbots. In fact, these tools, under the right conditions and with adequate safeguards, could be exceptionally useful.
However, it’s crucial to realize that at present, their capabilities are not entirely reliable.
The Role of the Media and OpenAI
The media and OpenAI bear some responsibility for this issue.
Media often portrays these systems as emotionally intelligent entities, failing to emphasize their unreliability. Similarly, OpenAI could do more to warn users of the potential misinformation that ChatGPT can provide.
Recognizing ChatGPT as a Search Engine
The tendency of users to utilize ChatGPT as a search engine should be acknowledged by OpenAI, leading them to provide clear and upfront warnings.
Chatbots present information in a regenerated text format and a friendly, all-knowing tone, making it easy for users to assume the information is accurate.
This pattern reinforces the need for stronger disclaimers and cautionary measures from OpenAI.
The Path Forward
OpenAI needs to implement changes to reduce the likelihood of users being misled.
This could include programming ChatGPT to caution users to verify its sources when asked for factual citations, or making it clear when it is incapable of making a judgment.
OpenAI has indeed made improvements, making ChatGPT more transparent about its limitations.
However, inconsistencies persist and call for more action to ensure that users are fully aware of the potential for error and misinformation.
Without such measures, a simple disclaimer like “May occasionally generate incorrect information” seems significantly inadequate.
