On a regular basis, I use AI tools in the course of my work. AI models take care of tedious tasks and provide a way for me to troubleshoot various issues as they arise.
After more than a year of posing questions to various conversational AI chatbots, I’ve come to the conclusion that the most important skill you need for using AI chatbots is: critical thinking.
Don’t be fooled by self-confident AI Chatbots
Certainly, mainstream chatbots will warn you that they can “make mistakes.” However, this disclosure is in tiny print and is repeatedly countered by the confidence with which these chatbots present their own answers to your questions.
To the best of my ability, I follow best practices when crafting my prompts: I provide context, ask specific questions, specify my expected output, upload examples, and then adjust the queries as needed based on initial responses. After learning the hard way, I stopped accepting the first answers I get from AI, and I always follow up with refinement questions.
Despite the confident way that chatbots frame their responses, my advice is to ignore that aspect completely. Some people you may encounter in life who will confidently answer any question on any topic because they’ve been socialized to believe they have to be an expert on everything. They’ll give you answers to questions they have no place answering, and lack humility or the ability to harbor any doubts about their “expertise”. This unfortunate human flaw is reflected in modern chatbots, only multiplied.
If you challenge any answers provided by a chatbot, they typically “take the blame” and then with 100% assurance turn over another solution – that may also be 100% wrong. Therefore, my general rule with AI chatbots is to question everything. We must analyze the confident answers we get from chatbots as objectively as possible and then form our own conclusions.
Believe the fine print that says you should “check important information” when asking AI for an answer to your question. When a chatbot says it has definitely, absolutely solved the difficult coding bug you’ve been troubleshooting for hours, chances are this “solution” was sourced from a GitHub post from 2017 that is definitely, absolutely incorrect.
Programmed to Lie
I’ve asked my chatbot of choice to stop using the em dash when I request a simple audit of copy for a website page. It assured me it would never do it again, followed by results that are riddled with em dashes. You simply cannot trust chatbots, and you must employ your critical thinking skills every step of the way.
In many cases, I’ve challenged my chatbot’s solution by asking it to provide the sources it used to support or derive its conclusions. Typically, this results in the chatbot contradicting itself by finding sources that undermine its original conclusion. That scenario is very concerning – where did the original conclusion come from? A deeply nested comment in a Facebook post? A blog from a random amateur website that hasn’t been updated since Obama was in office? Yikes.
Chatbot language models are not to be trusted, which is true of many human beings as well (unfortunately). Sometimes they get it right, but you need to make your own judgment call with every chatbot interaction. I’m surprised how illogical chatbots can be at times. Some answers I’ve received were so riddled with internal contradictions I’ve been compelled to ask my chatbot, “Are you okay?”
Perhaps, someday, chatbots will evolve beyond their Dunning-Krueger cognitive biases. I look forward to a time when I can ask an AI chatbot a question and the answer that comes back is, “I don’t know.” I guarantee you that day is not today. In the meantime, keep your critical thinking skills sharp, and push back on these chatbot language models and their fake confidence.