I Can’t Sleep - Large Language Models | Calm Bedtime Reading for Sleep

Episode Date: January 16, 2025

Unwind with this calm bedtime reading about large language models, created to ease insomnia and bring restful focus. In this soothing episode, Benjamin explains what large language models are, how they process language, and their growing role in artificial intelligence research and applications. His steady, reassuring narration turns technical concepts into peaceful storytelling, helping you release stress and quiet a restless mind. There is no whispering or hypnosis, only calm, fact-filled narration to guide you into relaxation. Press play, close your eyes, and let the story of large language models gently carry you into sleep. Want More? Request a Topic: https://www.icantsleeppodcast.com/request-a-topic Ad-Free Episodes: https://icantsleep.supportingcast.fm/ Shop Sleep-Friendly Products: https://www.icantsleeppodcast.com/sponsors Join the Discussion on Discord: https://discord.gg/myhGhVUhn7 This content is derived from the Wikipedia article on Large Language Models, available under the Creative Commons Attribution-ShareAlike (CC BY-SA) license. Read the full article: Wikipedia – Large Language Models. Happy sleeping! Learn more about your ad choices. Visit megaphone.fm/adchoices

You're listening to a Glassbox media podcast. What if I told you that most of the modern day self-help advice you've been hearing could actually make you worse? The key to a better life isn't about feel-good gimmicks that sound catchy. The Mentally Stronger Podcast gives you access to a licensed therapist who shares science-backed tools that will actually change your life. Hi, I'm Amy Morin, psychotherapist, mental strength trainer, and international best-selling author. In each episode, we cover research-back strategies, like how to stop relying on willpower and start creating habits for lasting change. And the five mental strength-building exercises you can do from your couch. I also speak to world-class experts like Dr. Nicole Kane, who shares how to permanently heal anxiety by addressing the root cause.

With over 200 episodes in our catalog, this podcast is for you if you're ready to crush self-doubt, conquer challenges, and become stronger than ever with therapist-approved strategies that can change your life. Listen to Mentally Stronger with Therapist Amy Morin, wherever you get your podcasts. Welcome to the I Can't Sleep Podcast, where I read random articles from across the web to bore you to sleep with my soothing voice. I'm your host, Benjamin Boster, and today's episode is from a Wikipedia article titled Large Language Model. A large language model, LLM, is a type of machine learning model designed for natural language processing tasks, such as language generation.

LLMs are language models with many parameters and are trained with self-supervised learning on a vast amount of text. The largest and most capable LLMs are generative pre-trained transformers, GPTs. Modern models can be fine-tune for specific tasks or guided by prompt engineering. These models acquire predictive power regarding syntax, semantics, and ontologies inherent in human language corpora, but they also inherit inaccuracies and biases present in the data they are trained in. Before 2017, there were a few language models that were large, as compared to capacities then available. available. In the 1990s, the IBM alignment models pioneered statistical language modeling. A smoothed N-gram model in 2001 trained on .3 billion words achieved state-of-the-art

perplexity at the time. In the 2000s, as internet use became prevalent, some researchers constructed Internet-scale language databases, web as corpus, upon which they trained statistical language models. In 2009, in most language processing tasks, statistical language models dominated over symbolic language models, as they can usefully ingest large datasets. After neural networks became dominant in image processing around 2012, they were applied to language modeling as well. Google converted its translation service to neural machine translation in 2020. As it was before Transformers, it was done by Sequence-to-Sequence Deep LSTM networks. At the 2017 Nureps conference, Google researchers introduced the Transformer architecture in their landmark paper,

Attention is all you need. This paper's goal was to improve upon 2014 Sequence to Sequence Technology, and was based mainly on the attention mechanism developed by Badanau, Badenau at all in 2014. The following year in 2018, Bert was introduced and quickly became ubiquitous. Though the original transformer has both encoder and decoder blocks, Bert is an encoder-only model. Academic and research usage of Bert began to decline in 2023, following rapid improvements in the abilities of decoder-only models, such as GPT,

to solve tasks via prompting. Although decoder-only GPT-1 was introduced in 2018, it was GPT2 in 2019 that caught widespread attention because OpenAI at first deemed it too powerful to release publicly had a fear of malicious use. GPTs 3 in 2020 went a step further, and as of 2024 is available only via API, with no offering of downloading the model,

to execute locally. But it was the 2022 consumer-facing browser-based chat GPD that captured the imaginations of the general population and caused some media hype and online buzz. The 2023 GPT4 was praised for its increased accuracy and as a holy grail for its multimodal capabilities. Open AI did not reveal the high-level architecture and the number of parameters of GPT4.

The release of chat GPT led to an upt and LLM usage across several research subfields of computer science, including robotics, software engineering, and societal impact work. Competing language models have, for the most part, been attempting to equal the GPT series, at least in terms of number of parameters. Since 2022, source of data, models have been gaining popularity, especially at first with Bloom and Lama, though both have restrictions on the field of use.

Mistral AIs models Mistral 7B and Mixtral 8x7B have the more permissive Apache license. As of June 2024, the instruction fine-tuned variant of the Lama 370 billion parameter model is the most powerful. open LLM according to the LMSYS chat pot arena leaderboard, being more powerful than GPD 3.5, but not as powerful as GPT4. Since 2023, many LLMs have been trained to be multimodal, having the ability to also process or generate other types of data, such as images or audio.

These LLMs are also called large multimodal models, models, LMMs. As of 2024, the largest and most capable models are all based on the transformer architecture. Some recent implementations are based on other architectures, such as recurrent neural network variants and mamba, a state space model. As machine learning algorithms process numbers rather than text, the text must be converted to numbers. In the first step, a vocabulary is decided upon. Then, integer indices are arbitrarily but uniquely assigned to each vocabulary entry, and finally, an embedding is associated to the integer index.

Algorithms include byte-pair encoding, BPE, and Wordpiece. There are also special tokens serving as control characters, such as mask for mast-team. out token, as used in Burt, and UNK, unknown, for characters not appearing in the vocabulary. Also, some special symbols are used to denote special text formatting. For example, a G with a dot over the top denotes a preceding white space in Roberta and GPT. Double hash denotes continuation of a preceding word in Burt. Tokenization also compresses the data. datasets. Because LLMs generally require input to be an array that is not jagged, the shorter text must be padded until they match the length of the longest one. How many tokens are,

on average, needed per word, depends on the language of the dataset. As an example, consider a tokenizer based on byte pair encoding. In the first step, all unique characters, including blanks, and punctuation marks, are treated as an initial set of ngrams, i.e. initial set of unigrams. Successively, the most frequent pair of adjacent characters is merged into a bigram, and all instances of the pair are replaced by it. All occurrences of adjacent pairs of previously merged Ngrams that most frequently occur together are then again merged into even lengthier n-gram, until a vocabulary of prescribed size is obtained. In case of GPD3, the size is 50257. After a tokenizer is trained, any text can be tokenized by it, as long as it does not contain

characters not appearing in the initial set of unigrams. A token vocabulary based on the frequencies extracted from mainly English corpora uses as few tokens as possible for an average English word. An average word in another language encoded by such an English optimized tokenizer is however split into suboptimal amount of tokens. GPT2 tokenizer can use up to 15 times more tokens per word for some languages, for example the Shan language from Myanmar. Even more widespread languages such as Portuguese and German have a premium of 50% compared to English. Greedy tokenization also causes subtle problems with text completion. In the context of training LLMs, data sets are typically cleaned by removing toxic passages from the dataset,

discarding low-quality data and deduplication. Clean datasets can increase training efficiency and lead to improve downstream performance. A trained LLM can be used to clean datasets for training a further LLM. With the increasing proportion of LLM generated content on the web, data cleaning in the future may include filtering out such content. LLM generated content can pose a problem if the content is similar to human text, making filtering difficult but of lower quality degrading performance of models trained on it. Training of largest language models might need more linguistic data than naturally available,

or that the naturally occurring data is of insufficient quality. In these cases, synthetic data might be used. Microsoft's Phi series of LLMs is trained on textbook-like data generated by another LLM. Reinforcement learning from human feedback, RLHF, through algorithms, such as proximal policy optimization, is used to further fine-tune a model based on a data set of human preferences. Using self-instruct approaches, LLMs have been able to bootstrap correct responses, replacing any naive responses starting from human-generated corrections of a few cases. For example, For example in the instruction, write an essay about the main themes represented in Hamlet. An initial naive completion might be, if you submit the essay after March 17th, your grade

will be reduced by 10% for each day of delay, based on the frequency of this textual sequence in the corpus. The largest LLM may be too expensive to train and use directly. For such models, mixture of experts, M-O-E, can be applied, a line of research pursued by Google researchers since 2017 to train models reaching up to one trillion parameters. Most results previously achievable only by costly fine-tuning can be achieved through prompt engineering, although limited to the scope of a single conversation, more precisely limited to the scope of a context window.

In order to find out which tokens are relevant to each other within the scope of the context window, the attention mechanism calculates soft weights for each token, more precisely for its embedding, by using multiple attention heads, each with its own relevance for calculating its own soft weights. For example, the small I.E. 117M parameter-sized GPT2 model, has had 12 attention threads and a context window of only 1,000 tokens.

In its medium version, it has 345m parameters and contains 24 layers, each with 12 attention heads. For the training with gradient descent, a batch size of 512 was utilized. The largest models, such as Google's Gemini 1.5, presented in fact, February 24 can have a context window sized up to 1 million. Context window of 10 million was also successfully tested. Other models with large context windows includes Anthropics Claude 2.1, with a context window of up to 200,000 tokens. Note that this maximum refers to the number of input tokens and that the maximum number of output tokens differs from the input and is often smaller.

For example, the GPT4 Turbo model has a maximum output of 4,096 tokens. Length of a conversation that the model can take into account when generating its next answer is limited by the size of a context window as well. If the length of a conversation, for example with chat GPT is longer than its context window, only the parts inside the context window are taken into account when generating the next answer,

where the model needs to apply some algorithm to summarize the two distant parts of conversation. The shortcomings of making a context window larger include higher computational cost and possibly diluting the focus on local context, while making it smaller can cause a model to miss an important long-range dependency. Balancing them are a matter of experimentation and domain-specific considerations. A model may be pre-trained either to predict how the segment continues or what is missing in the segment, giving a segment from its training database. It can be either auto-regressive, i.e., predicting how the segment continues,

the way chat GPTs do it. For example, given a segment, I like to eat, I like to eat. The model predicts ice cream or sushi. The next is masked, i.e. filling in the parts missing from the segment, the way Burt does it. For example, given a segment, I like to blank blank cream. The model predicts that eat and ice are missing. Models may be trained on auxiliary tasks which test their understanding of the data distribution.

such as next sentence prediction NSP, in which pairs of sentences are presented, and the model must predict whether they appear consecutively in the training corpus. During training, regularization loss is also used to stabilize training. However, regularization loss is usually not used during testing and evaluation. Substantial infrastructure is necessary for training the largest, models. Training cost. The qualifier large in large language model is inherently vague, as there is no definitive threshold for the number of parameters required to qualify as large. As time goes on,

what was previously considered large may evolve. GPT1 of 2018 is usually considered the first LLM, even though it has only 0.117 billion parameters. The tendency towards larger models is visible in the list of large language models. Advances in software and hardware have reduced the cost substantially since 2020, such that in 2023, training of a 12 billion parameter LLM computational cost is 72,300 A100 GPU hours. While in 2020, the cost of training a 1.5 billion parameter LLM, which was two orders of magnitude smaller than the state of the yard in 2020, was between $80,000 and $1.6 million. Since 2020, large sums were invested in increasingly large models. For example, training of the GPT2,

i.e. a 1.5 billion parameters model in 2019 cost $50,000, while training of the P.A.L.M., i.e., a 540 billion parameters model in 2022, cost $8 million. And Megatron touring NLG 530B in 2021, cost around $11 million. For transformer-based LLM, training cost is much higher than inference cost. It costs six flops per parameter to train on one token, whereas it costs one to two flops per parameter to infer on one token. Tool use There are certain tasks that in principle cannot be solved by any LLM, at least not

without the use of external tools or additional software. An example of such a task is responding to the user's input 354 asterisk 139 equals, provided that the LLM has not already encountered a continuation of this calculation in its training corpus. In such cases, the LLM needs to resort to running program code that calculates the result, which can then be included in its response. Another example is, what is the time now? It is, where a separate program interpreter would need to execute a code to get system time on the computer, so that the LLM can include it in its reply. This basic strategy can be sophisticated with multiple attempts of generated

and other sampling strategies. Generally, in order to get an LLM to use tools, one must fine-tune it for tool use. If the number of tools is finite, then fine-tuning may be done just once. If the number of tools can grow arbitrarily, as with online API services, then the LLM can be fine-tuned to be able to read API documentation and call API correctly. A simpler form of tool use is retrieval augmented generation. The augmentation of an LLM was document retrieval. Given a query, a document retriever is called to retrieve the most relevant documents. This is usually done by encoding the query in the documents into vectors,

when finding the documents with vectors, usually stored in a vector database, most similar to the vector query. The LLM then generates an output based on both the query and context included from the retrieved documents. agency. An LLM is typically not an autonomous agent by itself, as it lacks the ability to interact with dynamic environments, recall past behaviors, and plan future actions, but can be transformed into one by integrating modules like profiling, memory, planning, and action. The React pattern, a portmanteau of reason plus act, constructs an agent out of an LLM using the LLM as a planner.

The LLM is prompted to think out loud. Specifically, the language model is prompted with a textual description of the environment, a goal, a list of possible actions, and a record of the actions and observations so far. It generates one or more thoughts before generating an action, which is then executed in the environment. The linguistic description of the environment given to the LLM planner can even be the latex code of a paper describing the environment. In the depths, describe, explain, plan, and select method, an LLM is first connected to the visual world via image descriptions. Then it is prompted to produce plans for complex tasks and behaviors based on its pre-trained knowledge and environmental feedback it receives.

The reflection method constructs an agent that learns over multiple episodes. At the end of each episode, the LLM is given the record of the episode, and prompted to think up lessons learned, which would help it perform better at a subsequent episode. These lessons learned are given to the agent in the subsequent episodes. Monte Carlo tree's search can use an LLM as rollout heuristic. When a programmatic world model is not available, an LLM can also be prompted with a description of the environment to act as world model. For open-ended exploration, an LLM can be used to score observations for their interestingness,

which can be used as a reward signal to guide a normal non-LLM reinforcement learning agent. Alternatively, it can propose increasingly difficult tests for, for curriculum learning. Instead of outputting individual actions, an LLM planner can also construct skills or functions for complex action sequences. The skills can be stored and later invoked, allowing increasing levels of abstraction and planning.

LLM-powered agents can keep a long-term memory of its previous contexts, and the memory can be retrieved in the same way as retrieval augmented generation. Multiple such agents can interact socially. Compression. Typically, LLMs are trained with single or half-precision floating point numbers, float 32 and float 16. One float 16 has 16 bits or two bytes, and so 1 billion parameters require 2 gigabytes. The largest models typically have 1 billion parameters, requiring 200 gigabytes to load, which places them outside the range of most consumer electronics.

Post-training quantization aims to decrease the space requirement by lowering precision of the parameters of a trained model, while preserving most of its performance. The simplest form of quantization simply truncates all numbers to a given, number of bits. It can be improved by using a different quantization codebook per layer. Further improvement can be done by applying different precision to different parameters, with higher precision for particularly important parameters, outlier weights. While quantized models are typically frozen and only pre-quantized models are fine-tuned,

quantized models can still be fine-tuned. Multimodality Multimodality means having several modalities and a modality refers to a type of input or output, such as video, image, audio, text, proprioception, etc. There have been many AI models trained specifically to ingest one modality and output another modality,

such as AlexNet for image to label, visual question answering for image text-to-text, and speech recognition for speech to text. A common method to create multimodal models out of an LLM is to tokenize the output of a trained encoder. Flamingo demonstrated the effectiveness of the tokenization method, fine-tuning a pair of pre-trained language model and image encoder to perform better on visual question answering than models trained from scratch. Google Palm model was fine-tuned into a multimodal model Palm E using the tokenization method and applied to robotic control.

Lama models have also been turned multimodal using the tokenization method to allow image inputs and video inputs. GPT4 can use both text and image inputs, although the vision component was not released to the public until GPT4V. Google DeepMinds Gemini is also multimodal. Mistral introduced its own multimodal Pixtrell 12b model in September 24. Properties Emergent Abilities Performance of bigger models on various tasks when plotted on a log-log scale appears as a linear extrapolation of performance achieved by smaller models.

However, this linearity may be punctuated by brakes in the scaling law, where the slope of the line changes abruptly, and where larger models acquire emergent abilities. They arise from the complex interaction of the models' components and are not explicitly programmed or designed. Furthermore, recent research has demonstrated that AI systems, including large language models, can employ heuristic reasoning akin to human cognition.

They balance between exhaustive logical processing and the use of cognitive shortcuts, heuristics, adapting their reasoning strategies to optimize between accuracy and effort. This behavior aligns with principles of resource rational human cognition, as discussed in classical theories of bounded rationality and dual process theory. The most intriguing among emergent abilities is in-context learning from example demonstrations.

In-context learning is involved in tasks, such as reported arithmetic, decoding the international phonetic alphabet, unscrambling our words letters, disambiguate word and context, converting spatial words, cardinal directions, For example, replying Northeast upon open bracket 001, semicolon, 000, 000, 00, close bracket.

Color terms represented in text. Chain of thought prompting. Model outputs are improved by chain of thought prompting only when model size exceeds 62B. Smaller models perform better when prompting to answer. immediately without chain of thought, identifying offensive content in paragraphs of English, a combination of Hindi and English, and generating a similar English equivalent of Kaiswahili Proverbs. Schaefer at all argues that the emergent abilities are not unpredictably acquired, but predictably acquired according to a smooth scaling law. The authors considered a toy statistical

model of an LLM solving multiple choice questions and showed that this statistical model, modified to account for other types of tasks, applies to these tasks as well. Let X be the number of parameter count and Y be the performance of the model. Interpretation Large language models by themselves are black boxes, and it is not clear how they can perform linguistic tasks. There are several methods for understanding how LLM work. Mechanistic interpretability aims to reverse engineer LLM by discovering symbolic algorithms that approximate the inference performed by LLM. One example is Othello GPT, where a small transformer is trained to predict legal

Othello moves. It is found that there is a linear representation of Othello board, and modifying the representation changes of predicted legal Othello moves in the correct way. In another example, a small transformer is trained on Carroll programs. Similar to the Othello-GPT example, there is a linear representation of Carroll program semantics, and modifying the representation changes output in the correct way. The model also generates correct programs that are on average shorter than those in the training set. In another example, the authors trained small transformers on modular arithmetic addition. The resulting models were reverse engineered, and it turned out they used discrete,

furry or transform. Understanding and Intelligence NLP researchers were evenly split when asked in a 2022 survey whether untuned LLMs could ever understand natural language in some non-trivial sense. Proponents of LLM understanding believe that some LLM abilities, such as mathematical reasoning, implying ability to understand certain concepts. A Microsoft team argued in 2023 that GPT4 can solve novel and difficult tasks that span mathematics, coding, vision, medicine, law, psychology, and more. And that GPT4 could reasonably be viewed.

as an early yet still incomplete version of an artificial general intelligence system. Can one reasonably say that a system that passes exams for software engineering candidates is not really intelligent? Ilya Sutskiver argues that predicting the next word sometimes involves reasoning and deep insights. For example, if the LLM has to predict the name of the criminal in an unknown detective novel, after processing the entire story leading up to the revelation. Some researchers characterize LLMs as alien intelligence. For example, Conjecture CEO Connor Leahy considers untuned LLMs

to be like unscruitable alien Shagots and believes that RLHF tuning creates a smiling facade, obscuring the inner workings of the LLM. If you don't push it too far, the smiley face stays on, but then you give it an unexpected prompt, and suddenly you see this massive underbelly of insanity, of weird thought processes, and clearly non-human understanding. In contrast, some proponents of the LLM's lack of understanding school believe that existing LLMs are simply remixing and recombining existing writing, a phenomenon known as stochastic parrot, or they point to the deficits existing LLMs continue to have

in prediction skills, reasoning skills, agency, and explainability. For example, GPT4 has natural deficits in planning and in real-time learning. Generative LLMs have been observed to confidently assert claims of fact which do not seem to be justified by their training data, a phenomenon which has been termed hallucination. Specifically, hallucinations in the context of LLMs correspond to the generation of text or responses that seem syntactically sound, fluent, and natural, but are factually incorrect, nonsensical, or unfaithful to the provided source input.

Neuroscientist Terence Sownowski has argued that the diverging opinions of experts on the intelligence of LLMs suggests that our old ideas based on natural intelligence are inadequate. The matter of LLMs exhibiting intelligence or understanding has two main aspects. The first is how to model thought and language in a computer system, and the second is how to enable the computer system to generate human-like language. These aspects of language as a model of cognition have been developed in the field of cognitive of linguistics. American linguist George Lakoff presented neural theory of language, NTL, as a computational basis

for using language as a model of learning, tasks, and understanding. The NTL model outlines how specific neural structures of the human brain shape the nature of thought and language, and in turn, what are the computational properties of such neural systems that can be applied to model thought and language in a computer system. After a framework for modeling language in a computer system was established, the focus shifted to establishing frameworks for computer systems to generate language with acceptable grammar. In his 2014 book titled The Language Myth, Why Language is Not an Instinct, British Cognitive Linguist and Digital Communication Technologist Vivian Evans,

mapped out the role of probabilistic context-free grammar, PCFG, in enabling NLP to model cognitive patterns and generate human-like language.