The most interesting thing I learned about LLMs was how GenAI can not only produce bias information but also that overall A.I. bias can be more extreme than real-world bias. This was covered in Jon Cheung’s article for The London Interdisciplinary school. I think most people don’t really think about what a generative A.I. model is really doing when it’s asked to make an image. What makes it worse it the fact that the biased images the generative A.I. produces that are uploaded to the internet will then by fed back to it. For example, if you ask an image generator for a series of pictures of CEOs there is high chance that it will mostly produce a mid 40s white make in suit standing and smiling in a boardroom. If you upload that to the internet that those images can then be used to train the same or another A.I. image generator.

Another interesting thing was learning how exactly the token system worked. I didn’t know that A token is not always a full word. It definitely can be a whole word but it can also be a fragment of a word, a phrase, or individual letters. Then the text generation happens using the probability distribution across every token in its vocabulary for what should come next. Real good semi-complicated stuff.

In terms of my uses for A.I. I only plan to use it for organizing/ summarizing given information. I don’t use it for anything outside of that.

Citations/Sources

Cheung, J. (2023, August 21). How AI image generators make bias worse. London Interdisciplinary School. https://www.lis.ac.uk/stories/how-ai-image-generators-make-bias-worse

3Blue1Brown. (2024). Transformers, the tech behind LLMs | Deep Learning Chapter 5, YouTube. https://www.youtube.com/watch?v=wjZofJX0v4M

Declaration of AI Use: No generative A.I. was used in the writing of the blog post, nor for grammar.