My thoughts on Computerphile: Why are tokens expensive: Computerphile: Why are tokens expensive
Commentary
- Computerphile: Why are tokens expensive
- This was a eye opening and a great explanation of how you are billed for each message. Highly recommend people to check and understand to open a new chat if you have been cramping on the same chat for ages.
- Not only that becomes less expensive but makes the model a bit more clear on the things, otherwise its like finding a needle in a sand. Context bloat is a thing and so is the model’s ability to understand so many things at once.
- Lesson from this, start a new chat if llm is starting to deteriorate and hallucinating. Maybe even before that. I know its tempting to keep the conversation going, the model knows. a lot of things, you feel a resistance to open a new chat and provide all the details again. but that might help you solve the problem more effectively with a fresh perspective otherwise the model will create a mess of the code.