Dutch IT Channel – ChatGPT is showing strange behavior

ChatGPT was launched to the world just over a year ago. Chatbot software has become very popular in a short time. In less than two months, ChatGPT reached one hundred million users, which is an unprecedented achievement in the world of technology. The tool owes its popularity to the many tasks it can perform. Whether it's writing text, generating code, or filling out a spreadsheet: ChatGPT has become an indispensable tool in the workplace for many people.

But in recent weeks, the chatbot has begun to exhibit somewhat strange behavior. Users of X, formerly Twitter, have increasingly reported that ChatGPT does not answer their questions or only partially answers them. For example, instead of generating code, the tool shows users how to write it themselves. Do you want to fill out a CSV file (specification for table files)? Do it yourself.

OpenAI itself does not know

The problem quickly reached such proportions that OpenAI itself was forced to respond. “We haven't updated the model since November 11, and there's certainly no intention to. Model behavior can be unpredictable and we're looking to fix that,” the company behind the chatbot said. In other words, OpenAI itself has no idea. about what's happening.

We've heard all your comments about GPT4 being too lazy! We haven't updated the model since November 11th, and that's certainly not intentional. The form's behavior can be unpredictable, and we're looking into fixing it 🫡

— ChatGPT (@ChatGPTapp) December 8, 2023

Then it is up to the users to find out what is wrong. Some have come up with the surprising idea that this has to do with the time of year. More specifically, ChatGPT is said to suffer from a type of seasonal depression. The logic behind this is that the model would have learned from the datasets it was trained on that people slow down in December, and it could mimic that behavior.

Michel Vandendreich, co-founder of Leuven AI startup Raccoons, explains this further. “There are people who have investigated this issue and are talking about a statistically significant result.” For example, if you enter an assignment in May, you will receive more complex answers than in December, according to these researchers. “But OpenAI itself didn't find any connection,” says Vandendreich.

He himself finds it unlikely that winter will have an impact on ChatGPT's behavior. “It seems very strange to me that this is actually the reason,” he said. Another possible explanation, according to experts at Raccoons Vandendriessche, is that energy is simply more expensive during the winter months, which could make OpenAI consciously decide to reduce production. On the other hand, OpenAI contradicts itself by saying it doesn't know the reason itself, Vandendreich admits.

Another possible explanation is that the load on the servers has simply become too much due to the enormous popularity of ChatGPT. In mid-November, for example, OpenAI was forced to temporarily stop accepting new signups. This happened shortly after the first OpenAI DevDay, where some new tools were announced, including the ability to create your own custom GPT chatbot. In mid-December, CEO Sam Altman posted the following on X: “We've enabled ChatGPT+ subscriptions again!” Thanks for your patience while we find more GPUs. However, since then there are still users complaining about the low efficiency of ChatGPT.

'black box'

While it's still not entirely clear what's going on and whether ChatGPT is suffering from the winter blues, one thing is certain. Technology is increasingly beginning to behave like a “black box.” This means that we only know the inputs and outputs, but in reality we have no idea what is happening inside the system.

“Even senior researchers at OpenAI cannot predict in any way, even by looking at the underlying code, the exact outcome,” Vandendreich says. After all, LLMs, or large language models (algorithms that can generate content using very large data sets; in the case of ChatGPT, almost the entire Internet) work by performing complex calculations to arrive at an answer. “These calculations are simply beyond people’s understanding,” the Raccoons co-founder said.

“As systems become more complex, it will increasingly happen that we simply don't know what's going on,” says Vandendreich. “However, this does not have to be a problem. As long as we can verify that the results are not excessive, we should ask ourselves how bad it is that we do not always have insight into how the AI ​​arrived at a particular answer.”

take a deep breath

But even if we don't always know why a chatbot is giving a certain answer, we can find ways to generate the desired output in the meantime. Here too, ChatGPT sometimes seems to take over human behavior.

For example, used on the larger the tip, the longer the answer becomes. Others say ChatGPT needs to give them a little encouragement to keep going, like a coach giving an athlete an extra push. Even when you ask the chatbot to take a deep breath before answering, it seems to work better. Even writing everything in capital letters seems to have an effect.

So, next time ChatGPT gives you a disappointing answer, just be patient. Maybe you just need to be more empathetic and ask very nicely if the chatbot wants to do it again. If that doesn't work, you can always try bribing the tool.