I was reading Harry Potter, the chambers of secret, the book for the first time and I came across this quote.

“Never trust anything that can think for itself if you can’t see where it keeps its brain”

-- Mr. Arthur Weasley, Harry Potter and the Chambers of secret (Chapter 18)

The quote just rang the bells in me against LLM. Doesn't it hit? Why do I feel guilty like Ginny who had been writing to the dark lord.

Spoilery stuff ahead who hasn't read Harry Potter (The chambers of Secret)

For context, the sentence is said as a caution from a father of a girl named Ginny, who has been writing in the magical diary of Tom Malvaro Riddle. The diary is magical. You write to it, the writing fades and it writes back the answer to you, the answer is coming from the dark lord. She didn't knew it was the dark lord, so she kept sharing secrets and trusting it like her friend. The dark lord kept on asking her more and kept feeding on her fears and doubts and thoughts. She became so immersed in his words that she almost followed her orders like in a trance. The trance led the school of Hogwarts to go awry and let the dark lord cause havoc and chaos.

Harry, saves the day as usual, she rescues her from the chambers. He destroys the diary in the end, with the fangs of the salazar slytherin's snake.

The father of Ginny Weasely, the sister of Ron Weasly is trying to scold her daughter softly to never trust somethings that can think on its own and you don't know its intention or logic behind.

End of spoiler stuff.

There is a girl who this quote is addressed to as she got carried away with the diary's ability to answer her questions. The act of her writing to the diary and talking to someone unfamiliar or even having ill purposes speaks so much. The person writing in the diary is unaware what the intention or the goal of the person are who is writing back to her questions and thoughts.

Likewise, we are writing our thoughts and questions to LLM, for that we don't know its purpose.

In hindsight, we know its predicting the next word in the sequence. But that too is quite shaky and very unpredictable, each model or version of those are trained on different set of data, might have biases and assumptions. The worst is we don't know what all of those are, in that regard, we even don't know if the answer it gives back is grounded or made up.

This is the point which the word hallucinate really comes into play. I can say, "the dark lord (or whoever writing back in the diary) hallucinated the person writing into the diary".

You can say hallucination in the context of LLM is it trying to say wrong things, or made up facts, which is true, but hallucination is half that, and the action that completes it is us accepting them. And the father of the kid, rightly is scolding the kid on not trusting any material which can think on its own and we don't know the mechanism, place or intention behind the thinking or suggestions it gives.

Do I have to write that long explanation for this? No, but I will. I need to speak how important this topic is, and how reading is making me realise this.

Because asking LLM about something is cool and good, but it truly depends on how much trust you put in it, you can't blindly start following the results it gives. The consequences could be just simple as you doing something wrong and worse could be you making others wrong, none of which are even close to good. There were some cases of hallucination which became catastrophic as seen here and here.

Right now the state of LLMs are not that scaled, I mean its not so much into the systems yet, but one it starts, it might get in the grey territory. It can use tools for now, but people are quite sceptical and hopefully they should be for serious stuff.

What am I saying is that LLMs are not dark lords per se, but we need to be careful here. We can interact with the diary, but not feed in our hopes in it, you can't let it feed on you, you have to develop the skill of loneliness without relying on an LLM or whoever for that matter, you have to embrace solitude and come out of it strong. I know its quite intriguing to ask very sensitive questions to LLMs, as nobody can really see them, but who are you validating against? A piece of code that produces text based on the likeliness of what people have already written on the internet? How true is that, what do we know if its from reddit conversations or some forum where the topics have taken controversial takes, what were those outcomes. Don't try to equate a response of LLM with a human, that's the least I would say.

Suggestions are good, blindly accepting is bad.

There is hope in the dark and there will always be, till someone remembers to turn on the lights of your mind.