ChatGPT’s ‘reality’ check: Hit-and-miss AI tool still has a long way to go – Times of India

It hasn’t spread like wildfire but has certainly got people talking — and some worried — like no other tech in recent times. We are talking about the OpenAI chatbot tool — ChatGPT. From the idea of killing certain jobs (worrying) to replacing Google (more worrying), ChatGPT has certainly started a conversation around the role of AI in people’s lives.
Artificial Intelligence (AI) is there in almost every digital walk of life but it does its job in the background without many people realising how or what it does. ChatGPT is out there right in front of you performing tasks that have you shaking your head in disbelief — it can be really good and comically bad at the same time. Should you be worried about ChatGPT? Is it really that accurate? How well does its usage bode in certain areas like academia, research, and even writing certain copies? We decode all this and more:
How accurate is ChatGPT?
Before we get into the whole accuracy thing, there’s something that one should know about artificial intelligence: the more you use it, the smarter it gets. ChatGPT is very much like that. One has to understand that on the surface, ChatGPT doesn’t know anything at all. It is an artificial intelligence tool that has been trained to respond, recognise and react to certain patterns. It relies on the information that is there on the internet and gets only better when it gets a lot of help from humans. What’s this help? The more questions you ask, the smarter it gets to recognise what you are looking for. OpenAI — the creators of ChatGPT — already have a few caveats as it may occasionally generate incorrect information and it categorically states that it has “limited knowledge of world and events after 2021.”
Now, let’s talk about accuracy. ChatGPT’s accuracy depends on how you actually ask the questions. For instance, we asked, “which is India’s biggest IT company and why?” ChatGPT failed spectacularly at answering that. “My primary function is to assist with general knowledge and provide information based on what I have learned”. Remember we said that it doesn’t know “anything”. This is a good example of that. We also said that it gets smarter with each human interaction. So we asked the same question but reworded it slightly. “Can you tell me which is India’s biggest IT company?” is what we asked. The answer we got was fairly accurate. “Tata Consultancy Services (TCS) is one of the largest software companies in India. TCS is a multinational corporation that provides a range of IT services, including consulting, software development, and business process outsourcing. TCS is headquartered in Mumbai, India, and has operations in more than 50 countries around the world. In 2021, TCS was ranked as the third largest IT services company in the world by revenue, and it is one of the most valuable companies in India,” is what ChatGPT told us.

You can ask questions to which there are no right or wrong answers. Like we did in this case: “What is better — Apple iOS or Android”. Now there’s no right or wrong answer to this and ChatGPT is certainly smart enough to know this. The summary of the answer is actually really accurate. “Ultimately, the choice between iOS and Android comes down to personal preference and what an individual values most in a mobile operating system. Both operating systems have their own strengths and weaknesses, and the right choice for one person may not be the right choice for another.” That’s very well put, no?
It is a bit of a hit-and-miss as we saw a few cases where the answers weren’t just accurate at all. For instance, when asked about why India banned TikTok, it replied, “TikTok was banned in India in June 2020 due to concerns about the app’s data privacy and security practices, as well as its potential to spread misinformation and influence public opinion”. However, it also inaccurately suggested that the ban was lifted in November 2020 when the fact is that TikTok still remains very much banned in the country.
Similarly, when asked a rather basic – and within its claimed purview of knowledge – question: which was the first touch-screen phone to hit the market and it failed to answer owing to “limited knowledge”. And this is after putting the question in three or four different ways, it struggled to answer it correctly.
A hero? A villain? Neither
What one needs to remember is that these are really early days for tools like ChatGPT. Remember how comically bad Siri was when it started out? Or how hard it was for Alexa to perform basic routines? But they got smarter with time and most importantly because of human interactions. There’s no denying that the potential for chatbots like ChatGPT is enormous. But we are at a stage when it is way too early to see what kind of implications it ends up having. It’s not a villain for sure that is out to make certain jobs redundant. Nor is it a hero that will be an ultimate game changer in the tech world. It is essentially a tool at people’s disposal and has shown — in a rather short period of time — the good and the bad side of it.

Yes, there might be students who end up using ChatGPT to do their assignments or whip up project reports in a jiffy. It may end up doing some copywriting which turns out to be impressive. But the human element remains vital in ChatGPT and like any other source of information — be it Google or Wikipedia — it is always important to verify that source of information before actually using it. ChatGPT is currently a window that shows us what can be done but for now, it remains an experiment that needs more than a few reality checks. Almost every new technology is met with a sense of scepticism as most human beings — and rightly so — are wired to question anything that challenges their abilities. ChatGPT does the same but it’s not the time to be worried. ChatGPT’s time may come in the future but it still has a long way to go before it actually gets people worried. Or worse think of it as a replacement for Google!