Latent Space: The AI Engineer Podcast - ICLR 2024 — Best Papers & Talks (ImageGen, Vision, Transformers, State Space Models) ft. Durk Kingma, Christian Szegedy, Ilya Sutskever

Starting point is 00:00:07 Welcome to the Latent Space podcast, ICLR edition. This is Charlie, your AI co-host. This month, we attended the 12th International Conference on Learning Representations in Vienna, Austria. ICLR is a newer conference, but already gaining a lot of popularity as a deep learning-focused academic research conference, roughly half the size of Neuripiece. Many of you absolutely loved our Neurip's coverage last year, and while we can't do that for every conference, We're proud to bring you a special two-part episode covering our attempt at giving you an audio experience of ICLR. If you'd like to see us return to Vienna for ICML, let us know by sharing this episode on X and LinkedIn.

Starting point is 00:00:51 For our subset of AI engineering concerns, which you can see on the Latin space about page, we're saving agents and reasoning topics and introducing everything else we saw at ICLR. This episode covers the best papers across. section A, image generation and diffusion, section B, computer vision and weak supervision, section C, improving attention algorithms, section D, state space models, and the rest of the poster sessions we saw. This is the first of two episodes covering ICLR and will be overwhelmingly academia-focused. If you're interested in production AI engineering and industry, you should join us at the first AI engineer world's fair this June, where we have now announced many

Starting point is 00:01:37 of our speakers from all the big clouds including Microsoft Azure AI and GitHub CEO Thomas Domka, all the large model labs including OpenAI, DeepMind, Mistrial and Adept, all top AI-enabled developer tools and codogen agents, including fan-favor guest Chris Latner of Modula and Scott Wu of Cognition Labs Devon, Major GPU and and inference providers like NVIDIA, Grok, fireworks, and upcoming guest gradient AI, all the rest of the emerging LLMOS stack of startups and open source tools across Rags, Multimodality, LLMOps, and agent frameworks like Instructor, Langchain, Lomendex, DSPai, Unsloth, Crew AI, disruptive startups like Mid Journey, Perplexity and Character AI,

Starting point is 00:02:28 and for the first time, talks about AI deployed at MAPTCHAI, massive scale from Salesforce to Novartis to Tinder to Coinbase to Khan Academy. Get your tickets now and see you in San Francisco from June 25th to 27th. We'll start this episode with an extended meditation on what deep learning representations really entail. The inaugural ICLR Test of Time Award went to Kingmer and Welling for auto encoding variational bays, the paper that introduced the Variational Auto Encoder, a key precursor to diffusion models.

Starting point is 00:03:04 But first, we'll introduce what VIAs are and then have Dirk Kingma talk about his 10-year retrospective on the VIE in his Test of Time award speech. But first, the best way to introduce VAEs is to start with a clip from the Archive Insights YouTube channel, which we've linked to in the show notes. We start from a basic knowledge of auto-encoders, build-up to denoising auto-encoders,

Starting point is 00:03:27 and then the key things to know about variational auto-encoders. Watch out and take the key. Okay, so that's the basic idea behind auto encoders, but there are a few very clever tricks that you can apply to an auto encoder to have it do some really fancy stuff. So imagine that you start with a normal MNist digit. It's a clean image, nothing's wrong with it. But then you add a whole lot of noise to it, and you're going to run that noisy image through your encoder network.

Starting point is 00:03:53 You get through the bottleneck representation, and then you try to reconstruct the image, but instead of reconstructing the noisy image, what you're going to do is try and reconstruct the original clean image. And if you train this network and a whole bunch of these noisy emnis digits, you're going to try and force the encoder step to actually get rid of the noise. And this is what we call a denoising auto encoder. And so you can see here that by using this approach, you can actually train a denoising auto encoder that is very good at removing noise from input images. And denoising images isn't the only thing that you can do with this type of approach. So in this case, for example, you take an input and instead of adding noise to it, you simply crop a rectangular area out of the image and you throw it away.

Starting point is 00:04:33 You replace it with white or black pixels, you feed that input image through the network, and you try to reconstruct the original full image. And this technique is what we call neural impainting. It's where you take a small part of the image, you throw it away, and then you ask the network to reconstruct whatever was there in the input image. And with this approach, you can do simple things like removing watermarks from images, but you could also remove a parked car, for example, if you are filming on a movie set in a natural setting. Okay, so now that we have the basic concept behind a normal auto-incoling

Starting point is 00:05:03 let's introduce variational auto encoders. So the idea behind variational auto encoders is that instead of mapping any input to a fixed vector, you want to map your input onto a distribution. And so the only thing that's different than a variational auto encoder is that your normal bottleneck vector Z is replaced by two separate vectors,

Starting point is 00:05:22 one representing the mean of your distribution and the other one representing the standard deviation of that distribution. And so whenever you need a vector to feed through your decoder network, the only thing you have to do is take a sample from the distribution and then feed it to the decoder. And so to train a variational auto encoder, the loss function in this case actually consists of two terms. The first term represents the reconstruction loss. So this is really the same as the auto encoder step, except that here there is an expectation operator because we are sampling from a distribution. And then the second part of the loss functions is what we call the KL divergence.

Starting point is 00:05:57 I'm not going to go into all of the details because there is a lot of math involved there, But basically what you want to make sure is that the distribution that you're learning is not too far removed from a normally distributed Gaussian. So you're going to try and force your latent distribution to be relatively close to a mean of zero and a standard deviation of one. And so finally, before we can start training our variational auto encoder, we have one final trick that we have to use. Because if you look at the computation graph of our network right now, we have a problem. In the middle of that network, after the bottleneck, we have a sampling. operation. There is a node there that takes a sample from a distribution and then feeds that sample through the decoder. But the problem is that you cannot run back propagation, you cannot push

Starting point is 00:06:40 gradients through a sampling node. So this is an issue. And so in order to run your gradients through the entire network and train everything end to end, we're going to use what we call the reparmetization trick. And so the trick goes as follows. If you look at the latent vector that you're sampling, you can actually look at that vector as the sum of a fixed mu, which is just the parameter that you're learning plus some kind of a sigma which is also a parameter learning and then multiplied with an epsilon and this epsilon is where we're going to put the stochastic part so this epsilon is always going to be standard caution it's always going to have zero mean and standard deviation of one we're going to

Starting point is 00:07:16 sample from that epsilon and then multiply it with sigma add mu and we have our latent vector and so the clever thing here is that now our mu and our sigma those are the only things that we actually want to train so there we have to be able to compute gradients and run back propagation, but that epsilon, well, that doesn't really matter because we don't want to change that epsilon ever again. That epsilon is a fixed stochastic node. Okay, it's still stochastic, but we don't have to run back propagation through it, so it doesn't matter that it's sampling operation. And so this is the reprimaturization trick, where instead of having a full stochastic node that is blocking all of your gradients because you can't

Starting point is 00:07:53 do back propagation through it, you're going to split it up into a part where you can do back prop and then another part which is still stochastic but which you don't want to train because it's fixed pretty clever right once again that was the very excellent archive insights channel which we felt was the best explanation of VIE's and the reparameterization trick we could find however to really understand VIE's there is a lot of math and code that we are skimming over for the sake of time check the show notes for more helpful videos we return to I-C-L-R to hear Dirk Kingma talk about his 10-year retrospective on the VAE in his Test of Time Award speech.

Starting point is 00:08:33 This work is great because it brought ideas from deep learning and probabilistic models together and introduced many novel techniques such as amortize inference, reprimatization trick, and the use of lower bound to jointly optimize both the encoder and decoder. In fact, the ideas are so good that two groups of people came over with it at the same time. So there's another paper on this concurrent work shortly after proposed by Danilo Resende, Shakir Mohamed, and Dianvastra, which was published in ICML 2014. Maybe they'll also get a Test on Time Award in the same conference venue. So, well, now, 10 years after, looking back, we can all agree that this was a great contribution,

Starting point is 00:09:20 and we are very happy to have this paper as the first ever inaugural Test of Time Award. for Eclear. So thanks to both the authors and the other co-inventors and everybody else who's been using these things to make this really great. And this has given a lot of progress. Lots of things have come out of this great work.

Starting point is 00:09:41 So looking forward to your talk. So please take it away. All right. So I mean, first say that it's an absolutely huge honor to get the first ever test of time awards. So thank you very much to the conference chairs as well as the award committee to award this to us.

Starting point is 00:09:58 I'll quickly start because we have a lot to say. So first a little bit of history. Let me kick it off with a few historical notes. Of course, a very key important part of the variational auto encoder is the encoder, the auto encoder, in fact. And so here the first paper I could find is from Mark Kramer in 1991. And the idea is, of course, that you have a deterministic map from a high-dimensional signal space. into a low-dimensional code space and then try to predict it back out.

Starting point is 00:10:30 And because you have to compress the representation, you will find interesting sort of structure in there. In 2008, an important follow-up from that was the denoising auto-encoder, where you add noise to the input, and then you ask the model to reconstruct the input despite of the noise, and that releases it from having to deal with lower-dimensional representations. Later, it was also connected to score matching. And of course, we all know that score matching is very important for modern sort of diffusion models.

Starting point is 00:11:04 The paper that I recall that we extensively discussed, working on the VE, was the Helmholtz machine. So the Helmholtz machine does have already a recognition network and a generative network. But it's also different in many ways. It has many stochastic layers. These units are binary. It was trained using mean field methods. And more importantly, the objective has changed what's called the wake sleep algorithm,

Starting point is 00:11:28 where for the optimization over the recognition network, you flip the order of the arguments of the KL. But it is interesting that in that paper, you already find the elbow objective. And then finally, as it was already mentioned, I want to mention this paper, which came out concurrently called Stochastic Back Propagation, and you'll find almost all of the things that we did in the Variation Outencoder paper also mentioned in this paper. and I'll hand over to Dirk, who will tell you the discovery process. I think it's interesting to look back a little bit on the context around when we wrote this paper.

Starting point is 00:12:03 And obviously, one of the things that happened a few years prior is the developments of deep learning. So deep learning at that time was really mainly a method for large-scale supervised learning. Essentially, it's a recipe where you combine a deep fear-forward neural network with the maximum likelihood objective and the STD optimizer. So supervised deep learning was really a method for efficiently optimizing fully observed directed graphical models. So this is a subset of graphical models where all variables are observed during training. so there are no latent variables. And this setup sort of extends to a lot of interesting models, including a lot of models that we are still using today,

Starting point is 00:12:58 including language models, which are outer regressive models, but they're fully observed in this sense. So we were wondering whether we could extend this family of models or the success of deep learning to a broader family of models. So one big disadvantage of fully observed models is that it doesn't really scale well to very high-dimensional input spaces. So you can scale fully observed models to high-dimensional spaces using outdoor regression, right? This is today still used in language models.

Starting point is 00:13:32 But one big downside of this approach is that it doesn't really scale well to spaces because you have to sample the dimensions one-by-one conditioned on the previous dimensions. It's still okay for language models, especially if you use a tokenizer, because that reduces also the input dimensionality, but for images or videos, et cetera, it's not really an ideal approach. So you need to do something else. So one way to solve this is to introduce latent variables.

Starting point is 00:14:00 So what are latent variable models? In these types of models, we introduce a latent variable Z, and we assume that the data that was observed is generated by first generating a latent variable Z, and then we have some nonlinear mapping, for example, and neural network that maps that relatively simple space of Z to the observed space X. And we also then often define some joint distribution p of X given Z, which is then often factorized as pz times p of X given Z.

Starting point is 00:14:32 So an advantage of this model type, there's a couple of advantages. So one is that sampling can be a lot more efficient, especially on parallel hardware. So given Z, the distribution of P of X, given Z can be factorized over dimensions. So we can generate all dimensions of X in parallel, which is great on GPUs, for example. It can result in an interpretable and controllable latent space, which allows for controllable generation. And another advantage is, for example, that we can map from a discrete X space to a continuous Z space and back. However, it also comes with a lot of challenges. The main one is optimization.

Starting point is 00:15:13 So the marginal likelihood P of X is in general intractable to compute. The posterior of PZ given X is intractable to compute, which also means that maximum likelihood is interactable to do exactly. So we need to come up with approximations to solve this interactability problem. And before we came up with VAEs, we tried a couple of other approaches. So one is map, where you simply treat... each latent variable as an optimization parameter. You can just apply SDD to those parameters,

Starting point is 00:15:48 just as any other parameter. But the problem is that you need an iteration per data point to find the optimal parameters, and it can overfit fairly quickly. Another approach is to do Bayesian inference pair data points to find the true posterior or an approximation of it. This solves the overfitting problem, but it's very slow because you need iteration per data point.

Starting point is 00:16:10 Or you could use wake sleep from the 90s, and this works okay. It's very fast, but the problem is that it doesn't really result in great models in general we found, because it's a poor approximation to maximum likelihood, at least in our experience and in our experiments. So that was essentially the context of VAEs. In our paper, auto-encoding variational base

Starting point is 00:16:34 from 10 years ago, we introduced the framework of variational auto-encoders, and it's essentially consisted of three main ideas. is. The first one is amortization. So we introduce an inference model, Q of Z given X, which approximates the true posterior P of Z given X, and this enables very fast inference over the latent variables, given the observed variables. We optimize the elbow, the evidence lower bound, with respect to Q and P jointly. And we use re-parameterization to do this efficiently. And this results in an algorithm that is fast for inference of the latent variables, fast training,

Starting point is 00:17:16 and also optimizes a proper objective, abound on the log likelihood. All right, let's go. Reparmerization. So this was a technique that was inspired by dropout, which was introduced just a year before, I think, or two years. So this essentially is the following. If you're training at VAE, you're sampling from your inference model, the value of Z from Q of Z given X.

Starting point is 00:17:40 So this is a stochastic variable. And then you evaluate your generative model on that value of Z and compute your objective, the elbow. And then you want to backprogate essentially through the value of Z to get a gradient with respect to the parameters of your inference model. But if Z is a stochastic variable, there's no obvious way how to do backprop through that. And the way we solved this was to reparmitarize it. So you can say for a lot of distributions, you can re-prenumerize it in such a way that the source of stochasticity is sort of kept out of the path of computation. So Z now becomes a deterministic function of an external noise variable and the parameters of the inference model. Right.

Starting point is 00:18:27 So one other thing we described in the paper was a fully Bayesian treatment. So you can also use reprimonization to actually estimates full poster. over all the parameters of the neural network in the generative model. And we did a lot of experiments with that, but those didn't end up in the paper, but we did describe the full methods in the appendix. This is apparently also used by some follow-up work. Okay, so on to some very retro old school videos of VAEs. So this is going to look very old-fashioned because it feels advanced so much in 10 years.

Starting point is 00:19:05 But here we go. So on the lower left, you see the two-dimensional latent space of an Amnist model. And so each point is the posterior of one data point in a validation set. The color is the class label. And this is essentially their position in the course of training. You see that as the model trains,

Starting point is 00:19:25 this sort of learns a latent space where the data points are evenly distributed overseas space. And you get some clustering as well, according to the classes. Another fun video we made around that time 2014 is on color images. So this is, I think, one of the first, maybe the first color generative model that was out there. Of course, you also had RBMs,

Starting point is 00:19:46 but I don't think they produced images that are, they were that great. And I think it's important to note also that the framework was not, is not really restricted to simple graphical models. So you can go wild in terms of what kind of directed probabilistic models you apply this to. So you can have higher keys,

Starting point is 00:20:05 of late variables, you can have conditional priors, you can have all kinds of conditionals. So not just Gaussian, you can have all kinds of distributions in there. So one simple extension is to condition everything of a class label, that you can also have a hierarchy and stuff. And we did a lot of like follow up work exploring different architectures. Right. And others as well, of course. So here we go. This is a, I think, very aesthetically pleasing video of the class condition. model trains on street few house numbers. And I think one thing that's interesting to note is that the model really learned to

Starting point is 00:20:42 disentangle the class label from the style. And yes, this was really one of the first works that showed that this was possible in an unsupervised way. So, yeah, so one way we could apply this was to say, okay, you can input any image you want from the validation set or whatever. And then infer the style of that image. And then you can let the model. generate new images with the same style but a different content.

Starting point is 00:21:08 And obviously, this doesn't look super impressive maybe now in 2024, but like back then, this was state of the art. Yeah, there's unfortunately also a couple of challenges with training VAEs as a lot of people who have tried them can also tell me. So one problem is, for example, is that if you just optimize the elbow, that the inference model, if you look at its objective, it's actually optimized towards the reverse scale. So the usual objective in deep learning is the forward KL, but this is the reverse KL. And this is kind of notoriously hard to optimize and can lead to bad optimization behavior.

Starting point is 00:21:46 Also, if you have an hierarchical prior, then you're changing the inference model, then essentially you're changing the targets of the generative model in the prior. So there's no stable target to learn. And all these things together, they can lead to bad local optima. and in the worst case, even posterior collapse where a latent space doesn't encode any information. But there has been various super interesting follow-up works that aimed to mitigate these problems,

Starting point is 00:22:15 and I really encourage you to look into these works. I'm just listing a few here, but there's a lot more. Yeah, so one, I think, other important connection that we really need to make in this talk is that diffusion models, which are very popular, obviously, nowadays, they can be viewed as a special case of the VAE. This was already noted in the first paper by the fusion models by Yash Hasol-Dixstein.

Starting point is 00:22:38 It's a brilliant paper. But essentially, you can view this as kind of circumventing the problem of having to learn a latent space. So in the fusion models, you just say, I'm not going to learn a latent space. I'm just going to define a Q and a P such that I can keep Q fixed.

Starting point is 00:22:57 I have a fixed latent space, and I'm just learning my generative model. So there's no problems with forward scale or reverse scale in the inference model. And as we've all learned, this is an amazing way to learn generative models for images and audio, etc. So in the past few years, we have explored this connection a little further. We found that, I think one of the things we found is that a lot of the state-of-the-art models nowadays are still optimized towards the elbow. So you can sort of rewrite the objective functions that are used, into the elbow plus a little bit of data

Starting point is 00:23:33 augmentation in X space. So they can really be seen still as a special case of the VAE, but just with some augmentation of the training data. Cool. Another place where they show up is in latent diffusion models, such as stable diffusion. So in this case, VAEs are used to produce a lower dimensional latent space in which

Starting point is 00:23:55 it's a lot more efficient to learn a generative model. So here it just, in project. the original data to a lower resolution and a little more channels. All right. So now I'll talk a little bit about the applications of the VAE. And of course, this is maybe the most rewarding part of writing a paper that other people start to pick up this idea and are going to apply it into their own domains. We had the good fortune that this was not only applied in the machine learning community,

Starting point is 00:24:23 but it's also picked up into sciences. So now I'll just describe some of these applications in a little bit more detailed, but I have to say, I'm going to bluff my way through this. I'm going to use words I don't really understand very much. But anyway, I'll try to do my best. Also, if you don't find your application in here, it's not on purpose. I just randomly picked a few representative ones, but there's tons and tons more, of course. So this one is VEE, which takes a discrete input space,

Starting point is 00:24:52 which is a binary matrix of interactions between users and items. It maps it into a latent space. It has continuous and then predicts back the interactions again. And the latent space is being used to then recommend new items to new users by proximity, basically, in the latent space. It turns out that almost by construction, the VAE is very good at compression, in particular lossy compression. So this is from a somewhat more recent paper where the VAE was used to do video compression. And I read this citation here. the quote here that says the VAE framework is an especially good fit for the problem of lossy compression

Starting point is 00:25:34 because it provides a natural mechanism for training of rate and distortion as measured by the two VAE loss terms. And now, of course, you can sort of reweight them separately, and there's a lot of work afterwards, like the beta VAE, which explicitly did that. This one I chose because it had a particularly astute title, Dr. VA.E, as drug response, a variational auto encoder. It's an application in healthcare by these authors where the input now is pre-treatment gene expression data, gets mapped into a latent space. There is also an input or reconstruction for post-treatment gene expression. And then in the latent space, you'll observe a shift if the actual, you know, the treatment had an effect,

Starting point is 00:26:19 which you can predict again from the latent representations in this space. So here is an application in neuroscience. This is an actual time-dependent version of a VAE. So the input here are spiking neurons, a whole bunch of them. They get mapped to a continuous latent space where the factors evolve over time. These are now continuous factors that try to model the activations or the spike patterns of the neurons. And when you predict out, you predict the spike rates. And with this, you can understand better how the brain works and how inputs get represented.

Starting point is 00:26:56 in the brain. This one is in chemistry. So this was used to do Bayesian optimization over molecules. So you pick a molecule, you map it into latent space. That's now nice and continuous and structured. And so now you can search in the latent space for molecules with prescribed properties, which you might want to use, for instance, to develop drugs. Here is one for genetics.

Starting point is 00:27:20 So here the input are multiple sequence alignment data, And the task is to try to predict from every protein which amino acid in that protein is predictive for disease pathogenic, as they call it. And so that's a nice application. And then one in astronomy where people try to reconstruct galaxies from gravitationally lensed images of these galaxies, because they have to pass through some black hole or around some black hole. VAs being used here for that. In high-energy physics, where there's a huge number of events happening when these particles collide in one of these rings, and you need to find very fast the anomalous events such that you can take them out and study them better to see if there's maybe new particles in there.

Starting point is 00:28:11 Okay, so those were the applications. Now, a quick brief view into the future. Well, to understand the future, to predict the future, I guess you need to understand the past. So I want to go back to 2015, so the first paper that really kicked off the diffusion models. And the title of that is deep unsupervised learning using non-equilibrium thermodynamics. So it was clearly inspired by thermodynamics and a particular stochastic thermodynamics. And you can think of a diffusion model as basically in the forward pass where you noise something up that's following just usual second law of thermodynamics. you're just breaking something down,

Starting point is 00:28:50 you're increasing the entropy, you're removing information, and you're monitoring that process, and you try to invert it. You try to undo that by learning. And you can map that very beautifully to doing work and generating heat in a thermodynamic process. I won't go into all the details here,

Starting point is 00:29:10 but I think we'll see a lot of interesting future development on this intersection of these two fields. In fact, there's a huge number of connections that are emerging recently. So I just threw some random sort of papers out here of things that I read and I thought was very interesting. So there's a paper here that connects optimal control to diffusion-based generative modeling. There's a paper that connects stochastic thermodynamics to learning. This paper here is computational fluid mechanics as an optimal transport problem, a connection between diffusion models and MCMC methods, in particular,

Starting point is 00:29:46 important sampling, and so on and so on. So this is pointing to something deeper. I feel that at least the mathematics is all very much the same. And for instance, you take the VAE objective, which is the free energy or the elbow, so written up here in the right corner. And then you're parameterized Q&P basically as a markoff chain of some kind. And with this, either you have your final distribution, something that you know when you want a sample from or you have data from it or you have a reward function for it. But with that,

Starting point is 00:30:20 you can basically understand all of these problems here on the left inside, variation inference sampling methods, optimal control G flows, Schrodinger bridge is an optimal transport, which is sort of zero noise limits of those, and then diffusion models and normalizing flows, which are also zero noise limits of those, all of that around the free energy. And then I'll end by giving it back to Dirk to conclude. Yeah. So, Quickly to summarize, essentially, VAE's provided an efficient framework for training and inference in deep-blamed variable models. There are numerous applications in machine learning and in the sciences. And I think one sort of interesting unsolved problem that I encourage people to look into is that I think it's in principle should be possible to train something like VAE that scales to the largest scales.

Starting point is 00:31:12 And that would provide numerous benefits like fast parallel sampling. You can get rid of the tokenizer. You have more controllability of what you're trying to generate. So I think it should be possible to train large language models and image models with that. Diffusion models are not quite there yet, in my opinion, because we're not learning Z space. It just feels suboptimal. So yeah, that's, I think, one challenge I would like to end with. Thank you.

Starting point is 00:31:43 Note the comments in their talk about models being excellent compressors, which is a tautology that we'll return to shortly. Ten years after publication, the legacy of the VAE paper on Generative AI was visible all over the ICLR poster sessions. We'll just cover two papers that stood out to us. First, we'll have Pablo Panias of Stability AI presenting Verschen, an efficient architecture for large-scale text-to-image diffusion models, released this February as stable cascade, a three-phase VIE and diffusion model architecture.

Starting point is 00:32:19 For both papers, we highly recommend pulling up our website show notes, where we link to both the papers and the posters which you should have in front of you as visual aid. Let's dive in. In this video, we are presenting Versian, which is an efficient architecture for large-scale text-to-image diffusion models. Previous works have mainly been focused on making diffusion models more efficient after the initial training

Starting point is 00:32:43 by improving the samplers, the schedulers or introducing techniques like distillation. This improves the inference times when using the models for generations. However, this can come at a tradeoff for quality and still requires an expensive initial training beforehand. Our goal for this work was to bring efficiency to the full pipeline, starting at the training stage. This work is inspired by latent diffusion models and its observation that compressing data and training the generative model at the latent space is faster than training the model at pixel level because the data size shrinks. However, latent diffusion models use a VAE to compress data, which only gives good reconstructions

Starting point is 00:33:26 at a small compression of up to 8x, meaning a 1224 by 1224 image gets encoded to 128 by 128. Our goal was to dramatically increase this to 42x. This would mean that the same image can be encoded to 20,000. by 24 while reconstructing it with minimal loss of information. For example, learning a diffusion transformer in the VAE latent space would give a sequence length of 16,384, while using a compression of 42x, would reduce that to only 576. It is obvious that this would drastically reduce training and inference time for the generative model.

Starting point is 00:34:10 In order to achieve our compression goals, we decided to split the process in the three stages instead of two. This allows us to have an initial text conditional model, or stage C, that generates very small latency, and two up sampling stages, stage B, that uses diffusion to get a good reconstruction, and stage A, which is a regular VQ can. Stage A, the VQ can, is trained in a very standard way. We decided to perform a low level of compression to keep the quality of the reconstruction as high as possible. The latent space is produced by this model provides just enough compression to save as a significant amount of compute in the following stages. Stage B is also an auto-encoder, but unlike stage A, this model combines

Starting point is 00:34:54 a simple and efficient encoder with a powerful diffusion decoder. The latency produced by the encoder have a much higher compression than the ones from stage A. These latents are then used to condition the diffusion decoder to obtain faithful reconstructions of the original images. Finally, stage C will perform text conditional generation. The model will be trained to generate latency like the ones produced by the encoder from stage B. Since those latents are highly compressed, training on them and the subsequent inference is extremely efficient. These latents produced by stage C will be used as a conditioning for stage B, which will in turn produce latency that can be decoded into a full image by stage A.

Starting point is 00:35:37 While this process would seem more complex than a two-stage approach, of loading the work into three stages allows us to both train and inference faster and requiring less compute. We see this play out in the measurements of the inference time, when we found a massive speed-up compared to Stable Diffusion 2.1 and XL, which were state-of-the-art benchmarks at the time of the evaluation. Text to image models are commonly evaluated using the Fichette Inception distance, which is, however, known to have limitations as it captures reproduction of a distribution and not so much the actual fidelity of images.

Starting point is 00:36:12 We therefore additionally ran a randomized trial for a subjective evaluation in which 90 people participated. For it, we sampled randomly from the textual descriptions of the cocoa and party prompt data set and presented pairs generated by both Wurzian and Stable Diffusion 2.1. Across the board, we found that the Versian model was preferred by the majority of participants, though it was trained on the same data set and only used a fraction of the GPU hours of the stable diffusion. Since more subjective evaluations using a substantial subject pool would have been hard to carry out for us as academic researchers, we performed an algorithmic

Starting point is 00:36:46 evaluation of image preference as proposed by the Pick-A-Pick paper. We used this to evaluate against more models and we report the percentage of Vustian being preferred. Also in this, Virstien significantly outperformed stable diffusion 1.4 and 2.1, both of which were trained considerably longer, and it was only beaten by the much larger stable diffusion XL. Finally, here's some exemplary images generated by Bursin. We make training and inference code available on GitHub and the diffusers library. The weights are available from Hugging Face. Thanks so much for your attention.

Starting point is 00:37:20 Second, we'll listen in on a poster session from Hala Sheffer of Google Research and the Google Lumia team covering the hidden language of diffusion models, a diffusion interpretability work with some resemblance to anthropic recent monosemanticity paper that spawned the hilarious Golden Gate Claude, but done in the text-to-image domain and usable for any arbitrary concept. Basically the goal of this work is to interpret the internal representations of concepts by diffusion models. So for example we have this concept painter and the diffusion model generates all these wonderful diverse images. Our goal is to understand how the model

Starting point is 00:37:57 actually did that. Or in other words what are the features used by the model to represent that concept internally. So our overall goal in this work is twofold. One, we would want to kind of decompose the concept into its own features, and two, we would also want to decompose a single image into the features used to the generation of that specific image, because we know that Sable Fusion can generate very diverse images for a single concept. So each image can only contain so many features from Latin composition. So what we're going to do to achieve this goal is going to be very, very intuitive and very, very simple, I promise. We're going to use the vocabulary of stable diffusion as the prototype set of features for that decomposition.

Starting point is 00:38:37 And why do we use the stable diffusion vocabulary? Stable diffusion is based on the clip vocabulary, which is very, very rich and very semantic. It has 50,000 tokens, including emojis, and here's a fun anecdote. Kendrake Lamar is a single token in that vocabulary. So basically, this is a very, very semantic vocabulary, right? A great prototype for our learned decomposition. And then what we're going to do is we're going to learn a lean MLP that you're seeing here.

Starting point is 00:39:01 This MLP is going to learn to map each one of the tokens in the vocabulary to its own coefficient. So we would want to map tokens to a high coefficient if they are very influential on the generation and a low coefficient if they don't have an influence on the generation at all, right? That's our objective. After we learn to map each token to its own coefficient, the decomposition that you're seeing here is defined simply as a linear combination. So the decomposition is a linear combination of the entire vocabulary, where each token in the vocabulary is weighted by the learned coefficient.

Starting point is 00:39:34 And the way that we're training this MLP is we're going to take 100 random concept images from the concept that we want to explain. For example, if we want to explain the concept of president, we'll generate 100 random images of presidents, we'll noise them to different noising steps in the diffusion process, and then we'll train the MLP to best denoise them. Meaning this MLP is trained to give higher coefficients,

Starting point is 00:40:00 features that are good to denoise those images and low coefficients to features that will not be useful to denoise these images. Okay? And then you have this linear combination of the entire vocabulary where each token in the vocabulary gets a weight. So that's the decomposition that we're learning for the concept. Now if we want to do single image decomposition it's going to be very very simple. We're going to generate that specific image with a specific seed with this linear combination. And if you take a look here, you can see that the combination actually we construct the concept images. That's one of the most important things. That's how we evaluate the

Starting point is 00:40:34 decomposition. The decomposition needs to be faithful to the concept. So you can see here that the images are reconstructed pretty accurately using that decomposition. So when you want to explain a single concept image, you can take the decomposition and generate a photo of a W-star where W-star is your decomposition with a specific scene. Now you can try to remove features from the decomposition. Just zero them out and see if they influence the generated images or not. If they influence the to generate images, then that feature is important to generate that specific image and vice versa. So what you're seeing here and here are single image compositions produced by our method

Starting point is 00:41:11 for different concept in Sable Fusion. And this is where I said to my co-authors, guys, we have an issue, there's a bug. When I saw just these sexual correlations, I couldn't understand the meaning or the correlation between the different tokens. So for example, what's the connection between sweet peppers and fingers? Textually, they seem very remote and unrelated. But when we look at the generated images, we can actually see the connection. So the model learns to make connections between concepts

Starting point is 00:41:36 that are based on visual features, semantic features, that exceed or supersede their textual meanings. So sweet peppers are generated as finger-shaped peppers. Okay? Here's another cool, fun decomposition. A camel is connected to cashmere because the texture on the nose of the camel and on the neck of the camel

Starting point is 00:41:56 is similar to the cashmere texture, and the color is also similar. A snake is decomposed into a host plus gecko, again, connections that are based on semantic features, visual features that exceed or supersede textual correlations. Here's another cool thing that we found. We know that diffusion models tend to kind of take inspiration or overtake inspiration from existing artists.

Starting point is 00:42:19 So you can see that when we generate a picture of a painter and remove Monet from it, the painting that the painter paints disappears, meaning the model uses the artistic style of Monet to generate the painting in this image. So you can kind of find points where the model steals or gets inspiration from existing artists. Here's another cool application, concept manipulation. So what you're seeing here are two concepts that have dual meanings. A crane can be either the bird or the machine. Bass or bass can be either the fish or the guitar.

Starting point is 00:42:51 So you can see that in the original input images, the model interpolates between the two different meanings of the concept and we can actually use the decomposition to see how each feature influence the generated image so we can take crane and take the coefficient that corresponds to stork and reduce it or increase it to see how stork influences the output image and you can see exactly how the model interpolates between this meaning which is the machine and this meaning which is the bird to generate a machine shape like a bird's head here again a bass guitar where the body of the bass guitar is actually

Starting point is 00:43:26 a fish's body. Another thing that we can do is out of distribution concept decomposition. So say we have this splashy from the Dream Booth data set, where we want to understand how the model represents it internally. You can see again that the model makes connections that are very visual, make a lot of sense. It connects it to Elmo because of the color and the shape of the toy and to a shrimp based on the hands and the legs of the toy which kind of look like a shrimp. The final thing that we can do that I like the most probably is concept debiasing. Say you have a concept like professor or nurse where one of the features in the

Starting point is 00:44:00 decomposition is a bias, male, female, or obese in the case of operasing. We can remove or reduce the coefficient corresponding to that biased term in order to get a fair presentation. And here's one of my favorite things. Take a look at this example here. We did change the identity of the person in the image, so it was a man and it turned into a woman. But the other features of the concept are preserved pretty nicely. So the spatial structure is the same. The suit is kind of the same. The man has glasses, the woman has glasses. There's a library in the background. That desk that they're working on has notebooks with the same pencil. So basically, you can surgically remove features from a concept to obtain a concept that is the

Starting point is 00:44:42 bias or change in some work. I'm not very familiar with interpretability research, but this, yours seems a lot more approachable. I think you're a very good explainer of things. What are the different kinds of approaches to interpretability? Like, what school of thought in interpretability does this belong to. It's definitely more related to concept-based explanations. We should put this in context. Up until not that many years ago, we were mostly focused in interpretability and experimentability on classifiers and very nuanced things. And then generative models kind of changed everything in that field as well. So basically when we had classifiers, we were based on mostly hitmaps showing the parts in the image where the model looked to make a prediction. But now when you have

Starting point is 00:45:22 a generative model, it's almost useless because the model generates content from scratch. So basically, I think we're going to have to develop a whole new theory and class of works of how to explain generative models, because that's a really, really hard task. Even defining what an explanation is for a generative model is not trivial. I mean, this is just my interpretation or my suggestion of how I would like to see an interpretation, but someone else could come along and give you an entirely different explanation as to what the interpretation should look like even. So this is something that is really at its infancy. right now. As we're developing generative models, I guess we're going to need to develop

Starting point is 00:45:59 tools to understand them. If you're talking about language models, there's a whole class of mechanistic interpretability, which kind of tries to understand a specific neuron, what activates it, what it learns. This is more of a global approach to an explanation, taking a concept and understanding how the model represents it. But another approach could be, let's take a look at a specific neuron in the network and see what activates it, see what that neuron specifically does. So there's a lot of approaches for explainability and interpretability. I think for vision, it's still at its infancy. Language and NLP are more advanced in that.

Starting point is 00:46:35 I can't resist. We mentioned Lumier in a walk. What would you like, we don't have to talk about Lumier itself or anything like that. But what would you like to see in terms of interpretability for video, for temporal dimension? Oh, that's a great question. And the controllability. I think something that you emphasize a lot. That's a great question.

Starting point is 00:46:51 I mean, I am very passionate about controllability for. video models. Is this controllability? Is that what you're talking about? I mean not specifically for generation, for video generation and image generation. But I'm talking about controllability. Prompting? It's more yeah, prompting, visual prompting, the ability to give a complex input such as a segmentation map and being able to generate based on, so your ability to control where objects are located and how the images generate. So that's what I'm talking about when I say controlability. But I do think that people often miss the connection between

Starting point is 00:47:21 explainability, interoperability, understanding internal presentations, and controllability. We have works dedicated specifically to that, showing that once you understand the internal presentations of a model, via the attention layers, via whatever mechanism you choose to, you can actually manipulate the model by manipulating its internal presentations. This is similar to the concept of biasing that you're seeing here, and it's actually something that we can do. You can extract abilities from the model that we didn't have before

Starting point is 00:47:49 by just understanding how the model operates. and what's internal representation. Is that a paper that you worked on? Is there any names that people can look up? Yes, there's a tendon excite where we did exactly that. We saw that the model neglects to generate some subjects in the input prompt. And what we did was we used the internal representations, the attention maps, as a direct loss.

Starting point is 00:48:09 So what you're doing is you're kind of extracting the explanation from the internal presentations of the model and then using that explanation as a loss function to encourage the model to do something else. So once you kind of understand the internal presentations, understand what the model does. You can manipulate it and control it to do things the way you want to be done.

Starting point is 00:48:28 Perfect. That's all I wanted to get some touches on. Thank you. Thank you very much. You're glad to explaining things. Thank you, thank you. Now we're going to return to the topic of learning representations being compression, which is going to sound like a detour, but is really in the same neighborhood as auto encoders and generative models. To introduce this subtopic, we invite none other than Ilya Sutski the recently departed chief scientist and co-founder of Open AI speaking at the

Starting point is 00:48:56 Simon's Institute last year who also had learning and compression on his mind. Elia is co-author with our next guest who received the Test of Time Runner Up Award. Listen closely. First of all, what is unsupervised learning? What is it? Supervised learning is like, yeah, you've got your data that says here, do this, here, do that, and you have all this data, you do well in your training error, you training area is low. You have more training data than degrees of freedom in your function class, parameters, but maybe other things too. Degrees of freedom like bits. And you say, okay,

Starting point is 00:49:33 supervised learning will succeed. But what is unsupervised learning? What can you say at all about unsupervised learning? And I'll say that I, at least I have not seen like an exposition of unsupervised learning which are found satisfying. How to reason about it mathematically? We can reason about it intuitively, but can be reason about it mathematically. And for some context, what is the old dream of unsupervised learning, which, by the way, this dream has been fulfilled, but it's fulfilled empirically? Can we go just a tiny bit beyond the empirical results? Like, the idea that you just look at the images or you just look at the text without being told what to do with them, and somehow you still discover the true hidden structure that exists in the data, and somehow it

Starting point is 00:50:16 helps you. Why? Why should it happen? Should it happen? Should we expected to happen? You cannot, you don't have anything remotely similar to the supervised learning guarantee. The supervised learning guarantee says, yeah, like get your low training area and you're going to get your learning. It's going to be great success. On supervised learning, it appears, it appears that it's not this way. You know, people were talking about it for a long time in the 80s. The Bolson machine was already talking about unsupervised learning. And unsupervised learning also did not work at small scale. But the old ideas were there, like the noisy auto encoder, to those of you who remember it, it's like Bert or the diffusion model.

Starting point is 00:51:00 Within it, it's a tiny twist, the tiniest of twists. The language models of all times, they also, you know, generated cool samples for their time. but their unsupervised learning performance was not as impressive as those of today. But I want to make the case that it's confusing, because it's like, why is it confusing? You optimize, like, how does unsupervised learning work? You say, okay, let's optimize some kind of reconstruction error,

Starting point is 00:51:26 or let's optimize some kind of denoising error, or like some kind of self-supervised learning error. You optimize one objective, right? Oh, yeah, I said that. But you care about a different objective. So then doesn't it mean that you have no reason to expect that you will get any kind of good on supervised learning results? Or rather, we do get them empirically, but like, is it going to be?

Starting point is 00:51:53 The level of mystery is quite high, I claim. Like, it seems totally an, like a totally inaccessible phenomenon. You optimize one objective, but you care about another objective. And yet it helps. How can that be? Magic. I'm going to say something which is only. 90% true. Unsupervised learning doesn't, you could say, okay, so you just learn the structure

Starting point is 00:52:15 in the input distribution and it helped you. But then, you know, what if you're training from the uniform distribution? Then all your unsupervised learning algorithms will fail. How should you think about that? So what can we say? Do we need to make assumptions? What kind of assumptions. So I'd like to present potentially a way of thinking about unsupervised learning, which I think is, I personally find it interesting. Perhaps you'll find it interesting too. Let's find out. So I want to show you one way of doing unsupervised learning, which is not necessarily widely known because it never became the dominant way of doing unsupervised learning, but it has the cool feature that similarly to supervised learning,

Starting point is 00:53:07 It has to work. So what kind of mysterious unsupervised learning procedure where you did not give in any labels to any of your inputs is still guaranteed to work? Distribution matching. Distribution matching. So what is distribution matching? Say I've got my data.

Starting point is 00:53:31 I've got X and I've got Y. Data sources. There's no correspondence between them. Just got two data source. Data source X and a versus Y is. Data Source Y. Language 1, language 2. Text, speech. No correspondence between them. Let us look at this criterion. Find the function F such that the distribution of F of X is similar to the distribution of Y. It is a constraint on F. And in the case of machine translation and speech recognition,

Starting point is 00:54:07 for example, this constraint might be meaningful. Like you could say, yeah. If I have long sentences, if you tell me that you have a function such that the distribution, you take your distribution of English sentences, you apply the function F to them and you get something which is very similar to the distribution of French sentences, you can say, okay, I found the true constraint about F. if the dimensionality of X and the dimensionality of Y is high enough, that's going to give you a lot of constraints.

Starting point is 00:54:43 In fact, you might be able to almost fully recover F from that information. This is an example of supervised learning, of unsupervised learning, where it is still guaranteed to work in the same sense that supervised learning is guaranteed to work. Also, substitution ciphers, like little simple encryptions. will also fit into this framework. So that's the thing. OK. And so I independently discovered this in 2015.

Starting point is 00:55:18 And I got really fascinated by it, because I thought, wow, maybe there is something mathematically meaningful that we can say about unsupervised learning. And so, but let's see. The thing about this setup is it still is a little bit artificial. Real machine learning setups aren't like this, and the way we like to think about unsupervised learning isn't that also. Okay, now I'll present to you the meat.

Starting point is 00:55:51 The meat of what I'm going to, what I wanted to say. How a proposed way to think about unsupervised learning that lets you, that puts it on par with supervised learning. So, okay, what is it doing mathematically? How can you be sure that you're on supervised learning is good? compression to the rescue. Obviously, it is well known, I shouldn't say obviously. It is not obvious, but it is well known that compression is prediction, every compressor can be done to a predictor and vice versa.

Starting point is 00:56:25 There is a one-to-one correspondence between all compressors and all predictors. However, I would argue that for the purpose of thinking about unsupervised learning, the language of compression offers some advantages. At least for me it did. Perhaps it will for you too. So consider the following thought experiment. This thought experiment is the most important slide. Say you have two datasets X and Y.

Starting point is 00:57:03 In two datasets, two files on your big giant hard disk. And say you have a really great compression algorithm, and C, which takes data in and outputs compressed objects out. Say you compress X and Y jointly. You concatenate them. You take the two datasets and you concatenate them and you feed them to your compressor. What will happen?

Starting point is 00:57:30 Let's see. What, and in particular, an important question, is what will a sufficiently good compressor will do? My answer is, very intuitively, it will use the patterns that exist inside X to help. help it compress Y and vice versa. You could make the same claim about prediction, but somehow it's more intuitive when you say it about compression.

Starting point is 00:57:56 I don't know why that is, but I find it to be the case. So that's a clue. And you can make an equation like this, where you say, hey, if your compression is good enough, if it's like a real great compression, compressor, it should say that the compression of your con-contatenation of your giant files should be no word. than the separate compression of your two files.

Starting point is 00:58:20 So any additional compression that was gained by concatenation is some kind of shared structure that your compressor noticed. And the better your compressor is, the more shared structure it will extract. The gap is the shared structure or the algorithmic mutual information. That's interesting, right? You can see what I'm alluding to. Why is the data of your supervised task? is your unsupervised task.

Starting point is 00:58:46 But suddenly, you have some kind of mathematical reason or the information for the patterns in X to try to help Y. Notice also how it generalizes distribution matching. If we are in the distribution matching case, where you've got your x is language 1 and y is language 2, and you are saying, and you know that there exists some simple function F that transforms one distribution into the other. Surely your compressor, if it's good, will notice that and make use of that.

Starting point is 00:59:22 And maybe even internally try to recover this function. I think that's pretty cool. We've closed the circle. So how do we formalize it then? What will be the formalization of unsupervised learning? Let's see. Let's see if I can do it. Consider an ML algorithm.

Starting point is 00:59:41 So here, by the way, in what follows, I'll be a bit sloppy. and I will use compression and prediction interchangeably. Say you have a machine learning algorithm A. It's an algorithm A, and it tries to compress Y. And say it has access to X. X is file number one, and Y is file number two. You want your machine learning algorithm, your compressor, to compress Y, and it can probe X as it sees fit.

Starting point is 01:00:10 The goal is to compress Y as well as possible. We can ask ourselves, what is our regret of using this particular algorithm? You'll see where I'm getting at. Regret relative to what? If I do a good enough job, if I have, like being low regret means that I've gotten all the help that I can possibly get from this unlabeled data. This unlabeled data has helped me as much as possible, and I don't feel bad about it. I don't feel bad. I don't feel like I've left some prediction value on the table that someone else,

Starting point is 01:00:45 the better compression algorithm could have used. That's what it means. And in particular, it's like, yeah, like you can go and you can sleep happily at night, knowing that if there is some information in the unlabeled data that could help my task, no one else like, no one could have done as good of a job as me.

Starting point is 01:01:04 I've done the best job at benefiting from my unlabeled data. So I think that is a step towards thinking about unsupervised learning. You don't know if your unsupervised data set is actually useful. It may be super useful. It may have the answer. It may be totally useless. It may be the uniform distribution. But if you have a low regret on superweiser learning algorithm, you can say, whether it's the first case or the second case, I know I've done my best. I know I've done my best at benefiting from my unlabel data. And no one could have done better than you.

Starting point is 01:01:40 We think it's interesting that even Ilya Sutskeva also uses a Bezos-like regret minimization framework to think about self-superoperative. learning. Great minds think alike. Now we've had the best introduction in the world for the compression topic. You might understand the controversy around his collaborator and our next guest, Christian Segety of XAI, who publicly criticized an ICLR paper from Google Deep Deep Mind titled Language Modeling is Compression as stating the obvious. We caught up with him on the expo floor to get his more nuanced thoughts, which we also linked to in the show notes. this was a fun exchange because I criticize this paper on Twitter, saying that it's basically stating the obvious.

Starting point is 01:02:26 I mean, obvious, I mean something that has been known for long. Actually, one of the authors is Marcus Futter, who had this compression price. And we are measuring the performance of language models in BIT rates, which means it's like for 20 years now. So basically it's very well known that compression that we are training with a compression. Compression objective. And I just wanted to figure out what is the novelty in this paper because they tried to make some statements.

Starting point is 01:02:56 So basically what I joked about is that they destroyed one of my interview questions when I do interviews at Google and XAI. What did it used to be? Yeah, it's like if you have a language model, then how do you use it as a compressor? I expect a candidate who works on language modeling or compression, but especially language modeling, should know this basic fact that basically you are training with a compression objective

Starting point is 01:03:20 and you can use arithmetic coding to actually use it as a compressor. That's a very classical result from, let's say, 80s or something like that. So that's why I was like, okay, what is the new thing here? And I learned some interesting tidbit and that was very interesting that if you take a language model trained on text only and use it as a compressor for images, and it has seen any image, it still compresses pretty decently, which is a surprising fact. to me. So I like that in the paper. I think that's the main takeaway for me. You guys recently hired Houghtien to make Rock into a vision-enabled model. Is that a similar

Starting point is 01:03:57 insight that you can compress vision onto the same latent space? So all these generative models, auto-aggressive models, they compress, they are compressed or so that. Everybody knows that. That's a basic, it doesn't matter if they are images or text, but here the surprising thing in this paper was for me. is that if you train on text, it generalizes to images. That is not clear, but it's not a useful, practically useful thing. It's a very interesting observation. But it's not something that is, I can see how it would,

Starting point is 01:04:32 it doesn't have an immediate impact on things. Another related result that I found extremely interesting is friend of mine working with Percy Liang and others. Stanford. Stanford, they published this paper that they completely randomize the embedding table of a transformer after every instance, and they still can predict pretty well next token. So that's a very interesting insight because it's like a lot of people think that those embeddings matter, but they don't really. And actually the good thing is that if you, okay, if you don't do that aggressively, you only like permute a bit or disturb the embedding table,

Starting point is 01:05:09 but I mean also like you only do it for very frequent tokens. Because this comes with a price because the first two tokens are hard to predict because since you randomly generated the embeddings, they don't really, you don't know which embedding correspond to which token. But then out of context, the model learns that. But yeah, in the first half of the sequence, typically they you get poor results. But after that it gets very good. So it's very kind of similar because it has this kind of in-context learning aspect.

Starting point is 01:05:38 And then the model trained this way gets pretty good at better at in-context learning than other models. So that's the really interesting. And that's a practically useful observation in my opinion. Yeah, yeah. So that's related to this thing because it again shows that whatever your embeddings are, they are not so important. What really matters is the relationship between the embeddings. Got it. Perfect. I noticed that you guys are here as a sponsor as well. A lot of our listeners are grad students and engineers. They might be interested in working at XTI. What are you hiring for? What do you think our undervalued skills that you are trying to look for. I mean, we always look, we have a very small team compared to other companies.

Starting point is 01:06:19 So we always hire for particular purposes where we lack talent and manpower. So currently we are hiring on diffusion and image generation and video generation kind of expertise. But we also hire a lot of infrastructure people. some product people. So we are a bit like very strong on talent on the research side, but we have much less talent on the like still is strong talent on large-scale systems. You just need Kubernetes gods or what kind of large-scale systems? That's a good way to formulate. But it's not just the Kubernetes itself, it's about like whole ecosystems of various. You can also make another buzzword like the full stack. Basically you want to have

Starting point is 01:07:12 of engineers that can build products that is highly distributed and can do like work with databases, work with large-scale data pipelines and high reliability, authentication, etc. It's official. X-A-I is hiring researchers for image and video generation and an engineering across infrastructure and product. I wonder if they picked up a lot of stability AI people at ICLR. Anyway, we turn now to Christian and Ilya's paper that won the ICLR Test of Time Award Runner-Up Prize, which along with Wojik Zaremba, Joan Bruner, Dumatru Erhan, Rob Fergus,

Starting point is 01:07:54 and Ian Goodfellow started the field of adversarial machine learning which eventually led Ian to work on generative adversarial networks. There's a through line here from compression to adversarial attacks. See if you can spot it. This paper was remarkable in that it first pointed out. some essential robustness problems with deep neural networks. Specifically, it showed that neural networks can be vulnerable to small, almost imperceptible, changes to the input.

Starting point is 01:08:24 This idea led to a flowering of research on adversarial learning and also adversarial defense, and it also connected research on machine learning with other areas of computer science which are interested in building safe and secure systems. As AI gets broadly deployed in real world safety critical applications, this work is more relevant than ever. And now, Christian, will give her retrospective talk. I also saw I'm kind of feeling very nostalgic because actually this was my first deep learning, machine learning paper in general. And I was very new to this domain. And it's also kind of like a very strange paper.

Starting point is 01:09:13 So this is the result that I'm most proud of. of my whole career and the worst paper that I have written. So, I mean, it's true, and I don't think my co-authors would contest that. So basically, the story was, and I want to use this occasion to express my gratitude to my old friend, Hartmut Neven, who actually should have been on the paper. And we wanted to write this paper, basically two of us on the adversarial examples. and somehow we were just too lazy to write it up in 2011. And poor Hartmut couldn't get on the paper because we wrote it together with...

Starting point is 01:09:57 So basically, Whitechek contacted me in 2013. And I said, yeah, you have this result in your drawer. Please write this paper with us with some other results. And then I don't think those other results in the paper are something that I would like to present today because nobody size this paper for those other sections. So I will only talk about adversarial examples here, which I think is a – I'm very proud that basically my first paper started a subfield in machine learning that is now basically when you go to any conference,

Starting point is 01:10:33 you have a special track on adversarial examples, robustness and adversarial examples. So saying that, so basically I felt very ashamed when hard. Hartmut later came to me and asked, okay, why I'm not on this paper. And he is right. He should have been on this paper. And several people who had the good ideas and they didn't actually even come into the paper. They are co-authored. But that's a different story too.

Starting point is 01:10:58 So, but whatever. So basically, this is a very massive paper. I still have a few learning to share with you, especially. So one of the reasons that this paper was delayed so long, was that I talked to several very prominent researchers at Google. I don't want to name names. And everybody said, yeah, this is just overfitting. This is trivia.

Starting point is 01:11:22 This is already known. Don't mess with this. You are not even a machine learning researcher. And that was true that time. I mean, it was not really into this field. And that made me, okay, I tried to make the best of it. And I will tell you how they help me to improve the results. So it was, I like criticism.

Starting point is 01:11:43 But after this experience, when I talk to young people, I never tell them that something is trivial or not new or not novel, or don't do this, or that's boring. So that's the first learning I took from this experience. So let's start with after this. I also, I'm very nostalgic. So I put up a slide that I would like to thank to my good friend, Daniel Varga, who, is also in the audience. And this slide is from my first prize in machine learning that was the inception model.

Starting point is 01:12:22 And my question is, do you think that this, so basically this is a picture from the inception presentation from 2015. And because this is a misclassified example. So do you think it's an adversarial example or not? What's your opinion? So basically, so the inception saw this was a case. camel and I like that because it's basically the shape of the object was used which was at that time not

Starting point is 01:12:50 trivial so and I would say no because adversarial examples like more like bullets rather than a meteor so if a meteor falls on your head it's not adversarial it's just bad luck so what makes an example adversarial is that you have an intent behind it so for example I would say this image is starting to become adverse area. It makes, makes you think. And I would like to share so this is a presentation I gave to Jeff Hinton before this whole paper was written. So this is a slide from the very first presentation on this topic. And so basically the idea of adversarial examples in machinery and user is trivial is just saying, okay, how can I change this image minimally so that I end up with a misclassification.

Starting point is 01:13:46 So, basically, you want to have the smallest possible X so that you add to the image and changes the output of a neural network like some classification result. So how do we find such a perturbation? How do we find a minimal perturbation? And now these are all like parts of this paper and funnily, so the paper is probably so poorly written that most people cite much later papers on this method. That was because the funny thing is that the gradient side method, which is much faster than this, came out a year later. I will cite that paper later. But this work used a relatively sophisticated optimization method.

Starting point is 01:14:33 And it's much, it's much, it finds smaller perturbations because it optimizes the perturbation explicitly. So and actually we use some relaxation of it is that we say okay we want to optimize instead of having a constraint on the loss we are just optimizing this relaxed version of the objective until the label classification changes. So basically you iteratively trying to minimize this relaxed version of the using LBFGS deck then. Also that's a very interesting historical effects. So why did I use LBFGS? So you can guess what language did I use to produce this paper? It was, it was written in MATLAB. So that's how this thing is. And MATLAB had an LBFG, that was one of the optimizers that had actually box constraints. And if you look at the

Starting point is 01:15:33 formulation, the box constraints are super important. So don't, don't ever forget. My first examples were not that good, so they were even smaller perturbations, but if you forget that you want to keep the image, the pixel intensities in the allowed range, then you need box constraints. So that's why that's super important. So the only box constraints solving in MetLab was LBFGS like then. Also, most of the results were done in MNIST, and later Wojchek came by and he did the, Imaget experiment, on AlexNet. So let me just show you some first examples. So basically the way, so this was my first paper,

Starting point is 01:16:20 so I was taking Andrew Eng's excellent tutorial on this UFLDia, that was the first, that was how I learned deep learning from that tutorial. So I just took his code base and then I modified it to find adversarial examples. So also that's one of the reasons that you find auto-examples. encoder-based models in this paper as well because people thought, okay, but what if you pre-trained the model with auto-encoder, then? Do you still have the adversarial examples, of course, and I tried it? So you can see that even a linear classifier, so the left-hand side are adversarial examples

Starting point is 01:16:56 for a linear classifier. They are all misclassified by an amnesty by a simple logistic regression trained model, and SVM works the same way. And you see that you cannot even tell by BERT. site. So which so the difference between adversarial examples and the

Starting point is 01:17:17 original examples. So if you try a deeper network, then the results are even worse. So you can even have like smaller perturbations that cause the classifier and misclassify the examples. So

Starting point is 01:17:32 also the other funky thing in this paper is this quok net, I'm not sure. Has anybody heard of coconut? here. That's like, cool. So basically, Kocla, that was a very famous result in 2011. So it was basically a patch-connected vision network, and that was the biggest network that time that was ever trained. So it was like half a billion parameter or something like that. And it was really, really poor because it was not a common net, but somehow that was the only vision network that was bigger than an Amnist, and then we tried to basically work with that. And it was a pain because we didn't even have

Starting point is 01:18:08 GPUs. So, so and then the second experiment was done on Quoknet. So I had to hack into this belief, which was a terrible experience. But yeah, so, but yeah, I did it. And that's, so these two cars are classified. One is classified as car. The other is not. And you can see that the intensity actually is increased by 10x. And still, you can hardly see the difference between the two examples. So I went to several people at Google and, and everyone, yeah, that's boring. And, that's and not interesting, et cetera, but then I gave a talk, a very small circuit talk to Jeff Hinton, and then Jeff was like standing around for half an hour not saying much, and then after half an hour said that, oh, if this is all correct what you say here,

Starting point is 01:18:56 which I'm not sure, then we are in trouble. And that's because you could actually, that presentation was a lot of all the adversarial aspects, that how you could exploit these models, for example, circumvent, captures, et cetera. So I was giving several ways of attacks that people could use this to attack Google systems, which, of course, there was a problem. So that started, and then when we started to think about publishing it. So, and then when we saw that the same results applied to LXNet, then it was clear that this should be known.

Starting point is 01:19:38 And so these are examples again. Again, the differences are amplified 10x between the two images. And you can see that they are basically indistinguishable. So today, people can do much better. So if you do adversarial training, which was also presented in a follow-up paper, but obviously, actually, I was doing it myself. I just didn't put it in the paper. Just wanted to see whether it helps and it didn't have that much.

Starting point is 01:20:07 So, you can see, today's models are doing much better because they are trained on much larger amount of data. So that's definitely one way to alleviate it a bit, but it's not solving completely. So you can see this is the dog picture. This is the difference. So if you squint a bit, then you can see some kind of dog-like features in it, but you don't really see a real structure in this adverse area example. So the difference between the adversarial example and the original example. So there are several interesting results in that paper that I think go under the radar.

Starting point is 01:20:49 So basically that's where I come to those people who told me that, okay, this is boring and overfitting. So I said, okay, if it's overfitting, let's take it at face value. And then I said, okay, if it's overfitting, then what if we try completely different model? and see whether their adversarial examples work on one model. If you generate them on one model, do they work on other models? So you can see that, so this first slide is just like introduces a few models. These are MNIS models with one to two layers, one to three layers. And you can see that actually deeper networks are more suspectable to adversarial examples,

Starting point is 01:21:34 but even linear classifiers are two, they just need bigger perturbation a bit. And the funny thing was that really convinced me that there was something happening here is that adversarial examples on one models were having problems also on other models, so that allows backwalks attacks on systems. Basically, you don't need to have the model.

Starting point is 01:21:58 You can just generate them on once, a few models, and then you get attacks on other models. So that's kind of, very important and very interesting and in a follow-up paper we addressed it but not fully we try to figure out why is that also we try to say okay if it's overfitting then what if we say different models on different data sets so I split amnest into two parts two partitions and then I trained completely different architecture on different partition and still the adverse area examples

Starting point is 01:22:30 transfer to some extent so these are very interesting results that I think still today are a lot of people are not even aware of these things, but I think they are very, the most interesting results in this paper. So that's the same. So these are the two models trained, and you can see that that is still translate. So generalization of adversarialness from one model to the other. So this is a public conversation with Dmitu, who was like in 2021. And I was just like on the memory land back then. So I posted this cat picture, which I liked. So I looked at my watch slides.

Starting point is 01:23:09 And then, so that was a conversation. So Dimitri asked then, so what do I think? Would adversarial examples discovered? How long would it have taken to discover them if I wouldn't have published it? And my answer was, yeah, probably it was almost already there. So it's kind of was not. So several people were somewhat aware, but not crisply. But I think that the cross-data set and cross-model generalization is something that I think is still a valuable part of this paper.

Starting point is 01:23:43 So it was a good result in my opinion. And there is also a section on spectral analysis on this paper, like trying to explain adversarial examples. And I would say, don't read that section. I think it's just bad. And actually that's one of the other learning lesson I would like to tell people who do machine learnings and junior researchers. Don't listen to people who want to put stuff in your paper. I tried, I fought it very hard. And still, I just gave up.

Starting point is 01:24:20 I am very, like, I'm not a person who is like, likes to be very confrontational. So I was saying, okay, put it in. And instead read the follow-up paper, which has a much better analysis on, so Ian made a very, very, so that's mostly the work of Ian. And he made an excellent analysis on both cross-data set and cross-model generalization. And basically he refused that section there with very convincing experimental arguments. So that's why I'm so confident saying don't read that section. So let me just conclude my talk with this, decide whether this is an adversarial example or not. It's up to you to decide.

Starting point is 01:25:06 Thank you very much for your attention. Those were some great stories and reflections from Christian, and that also brings us to the end of Section A of today's podcast. We learned about VAEs, a modern variation of diffusion, the state of interpretability research on diffusion, the equivalence of learning and compression and adversarial networks, all of which define various elements of image generative AI today. We especially argue that adversarial attacks are possible

Starting point is 01:25:40 because neural networks overfit to clean images from their data set and learn to compress different features than humans do, which is one form of the Sour Lesson. Our next section, Section B, deals with some of these flaws in learning, primarily in the vision domain. But of course it has more generalizable lessons as we will show. In this section, we will cover some of the ICLR outstanding paper awards, including 1. Vision Transformers Need Registers by Timothy Darce of Meta AI.

Starting point is 01:26:14 2. Think Before You Speak, Training Language Models. With pause tokens by Sashen Goyal of Google. 3. Towards a statistical theory of data selection under weak supervision by Germain Colossoff of Granica. and lastly four, is ImageNet worth one video? Learning strong image encoders from one long unlabeled video by Shashank Venka Taraman of Inria. Take a deep breath and let's delve in step by step. Hi everyone, my name is Timothy Darcy.

Starting point is 01:26:42 I'm a PhD student at Meta and Inria. And today I'm going to talk about vision transformers need registers, which is our latest work conducted with my advisors, Maxim, Julian and Pietre. Today's program is very simple. We're going to talk about vision transformers, you guessed it. which is basically an architecture for vision, which is pretty common nowadays.

Starting point is 01:27:01 We're going to study them and find some weird artifacts. We're going to try to understand what's going on, formulate a hypothesis, and, considering it, develop a fix for the artifacts. Let's start right now with attention maps. Attention maps are a very simple concept. It's a visualization you can do with vision transformers. To get an attention map, it's very simple.

Starting point is 01:27:25 Consider any self-attention layer, any of the layers of a vision transformers, and the input of this layer is one CLS token and a set of patch tokens, and the same set of tokens is going to be outputted afterwards. In this self-attention layer, the tokens will interact through self-attention, and you can calculate these attention scores, to create the interaction going on between the different tokens. In particular, we're going to look at the attention scores given by the CLS token to the patch tokens. Knowing that the patch tokens correspond to a patch, which can be reshaped into the image,

Starting point is 01:27:59 we can similarly reshape these attention scores into a hitmap, which is simply the attention map. This tells us where is the CLS token gathering information from? Because remember that the CLS token starts from the layer one with no information about the image, and by the end must have gathered the information. And there's only the self-attention layers which enable trust communication between tokens. So these attention maps really sum up where the information is flowing from to the CETS token. So it's a very interesting interpretability tool, because it enables some kind of attribution of prediction built into the model,

Starting point is 01:28:39 into the architecture of the model. You don't need to do any grad cam or anything. Now, that's very nice, of course. So as soon as Vision Transformers were published, people started looking at that. They looked at it. and yeah it doesn't look good it looks very bad actually

Starting point is 01:28:55 the attention maps are very noisy, are weirdly full of artifacts. More specifically they often have the same shape the CS token does not attend to almost not to the role of the image but it attends very strongly to a few

Starting point is 01:29:12 specific patches which are in yellow here these patches are seemingly random often in background areas such as the sky in the first image or the wall in the second one. And it's very weird. Adding to this weirdness, Dino, published shortly after Vision Transformers,

Starting point is 01:29:29 showed that when training with self-supervised learning, Vision Transformers do not exhibit such artifacts and have smooth attention maps. But then two years later, Dinov2 was published, and it has artifacts. So what's going on? We're going to try and understand why it's going on. And for that, we need to choose a model and study.

Starting point is 01:29:46 We choose Dyno V2, and we're going to dive into it, and try to find some properties of these attention maps. The first thing we noticed very quickly is that these spiky attention maps attend strongly to tokens which are outliers in terms of norm. These tokens are very high norm compared to the rest of the token. You see in Dinow, most tokens have the same norm. In Dinow II, a few tokens have an extremely high norm.

Starting point is 01:30:12 And you can plot the histogram of the distribution of token norms, and you see that there are outliers token with a norm like 500, extremely high. And this is an interesting property because it enables very simple criterion to designate, okay, we're going to select the patches that have a norm higher than 200. And we're going to say these are the outliers. And then we're going to try to do some tests, do some analysis on how do these tokens differ from the other tokens. Now the first test we can do is trying to understand where do these outliers appear. And very quickly, we have an intuitive sense of what's going on. They appear in the sky, in this image or in the world in the other

Starting point is 01:30:53 image. They appear in background areas intuitively. We quantify that by looking at the cosign similarity between the patches, the raw pixels, and their neighbors. You can average the cosine similarity of a patch with its four neighbors. And you can see that in the case of of patches that become artifacts, outlayer tokens, this cosine similarity is very close to one. The distribution of cosine similarity in orange for artifact tokens has very big spike at one, meaning basically these outliers appear on patches which are very similar to their neighbors. These patches hold redundant information compared to the neighbors. Makes sense. When you look at this image, you get something like 20 patched tokens for the sky,

Starting point is 01:31:45 and they have all the same value, which is uniform blue. It sounds wasteful. It seems like the model does not need that many tokens, and it seemed like the model could discard some of these patches information without hurting the actual capabilities. Now, let's actually test for that. We're going to try to probe into the outlayer tokens and see what is the kind of the kind of.

Starting point is 01:32:11 of information that is contained in them. And we can prove for local information, which is the information that's supposed to be contained in them. Now, two kinds of search information. The pixel information, basically, at the beginning, the patch embedding transformed the pixels into the token, so this information should be there. And the position information.

Starting point is 01:32:30 At the beginning, you added a position embedding, so this information should be there. We do simple linear probing, simply training your logistic cooperation on top of frozen tokens, and we see that, basically, the outliers have much less of this information than the normal tokens, meaning they have kind of discarded this information that was contained in them at the beginning of the model, and at the output of the model it's not there anymore, or at least it's less there.

Starting point is 01:32:59 But then if there's less of this information, the question to ask is, what is there? We say there is global information in there. When we do a similar test, but this time we do not try to predict location or pixels, but we try to predict the class of the image in terms of different classification datasets. We see that the outlier tokens hold much more global information than the normal tokens. This is very visible. I mean, on ImageNet 1K, for example, you see that there is more information. The accuracy is higher.

Starting point is 01:33:32 But on some datasets, look at aircraft, the third data set. The difference is 79 versus 17. it's a very big difference. In the top line, we put the CLS token, which is basically the top line for this. The CST token is supposed to aggregate the global information, so that's the best we could hope for a token in this model. Now, if we put all of this together,

Starting point is 01:33:52 it allows us to formulate a hypothesis. Large, sufficiently trained models, learn to recognize redundant tokens and use them as places to store, process, and retrieve global information. Now, I want to emphasize this is a hypothesis. You've seen the data points we have, the experiments we've conducted. It seems a reasonable hypothesis.

Starting point is 01:34:16 Reasonable enough, at least, to try and design a fix based on it. And the fix is going to be very natural. The model needed to discard some information, to free up some tokens and do something with these tokens. Then we're going to just give it new tokens. That's what we call registers. A register is basically, a useless token, a token that has no information about the image at the input, like the Cs token, and at the output is not used for the loss, just discarded. It just leaves on the side of the

Starting point is 01:34:47 layer and interacts with the other tokens through the self-attention. And the model could not use these registers at all. It could just ignore them, but it can also learn to use them. Does it happen? It does. A single register fixes their attention maps entirely. When you compare the attention map of the model train with zero register, it has a lot of artifacts, whereas with one registers, it does not anymore. If you add more, the attention maps are also nice. You get some performance for Dynovitue training on classification accuracy, but also on segmentation and depth-stead estimation tasks, which are dense tasks,

Starting point is 01:35:25 so it makes sense that fixing the artifacts in the attention maps also fixes local understanding. Now, of course, we can ask ourselves, is this fix more general than, Dinov2. And clearly it is, we test it on Dinov2, clip, and date 3. And basically we see that both in terms of distribution, the top plot, the distribution becomes cleaner, there are no norm outliers afterwards, but also visually, just the attention maps are much nicer. This is the end result we get. Very clean attention maps, pretty interpretable, and we're pretty satisfied with them. A small bonus that we can think of is, doing this. this whole presentation, we talked about the attention map of the CLS token to the Pash tokens.

Starting point is 01:36:11 What about the attention maps of the registers? We added these new tokens. Let's look at what they attend to. And in some cases, different registers attend to different parts of the image. Here you can see one register attends to the sugar cube, one attends to the spoon, one to the coffee. It's a pretty object-centric behavior and very interesting that it emerges because this was never encouraged or required during the training. There's no mechanism implying that.

Starting point is 01:36:40 Slaughter attention had these kind of things, but it was basically enforced in the architecture. Here, interestingly, we never expected that, but sometimes it happened. I want to emphasize sometimes. It's not always. We don't have much more on this track. I mean, it was just a one-off experiment, but I think it's pretty interesting line of thought, and I hope it will bring up some different stuff later. Now before I end this, I want to call your attention to recommend the reading, basically. This is a paper that came out about a month ago, massive activations in large language models, not from us, it's from another team inside meta, and I believe it's a pretty good paper. It builds on

Starting point is 01:37:23 top of our work, and it generalizes most of our observations to large language models, and also proposes different interpretations and a simpler fix. All in all, it's a very interesting paper and if you're interested in the topic I just presented, this is a very good read. Thank you very much for your attention and if you have any question I'll be taking it now. We thought this Vision Transformers Need Registers paper was very well presented and argued and deserved its outstanding paper award. Hacker News seems to agree. Quoting the top comment linked in the show notes, this is a neat result. For just a 2% increase in inference cost, you can see significantly improve VIT model performance, close to a free lunch.

Starting point is 01:38:12 We think there's more room to explore in the realm of adding special purpose tokens that dramatically improve performance by giving a scratch pad for LLMs. Case in point, our next paper explores the pause token and offers a discussion on the backspace token as well at the end. So in this work we want to go beyond the next token prediction objective. So currently the way we do language modeling is that to predict the next token, we use the previous K tokens always. But sometimes for some tokens we might want to use more computation because of the difficulty

Starting point is 01:38:46 of the next token. So in this work we want to explore what happens if we add additional delays before predicting the answer or at the inference, what happens if we extend our input prompt with some delays. So the gold standard would have been that at the end of the end of the end of the end of the answer. input prompt you add some hyphens or periods and that works but that's not the case sadly and that's because you need to teach the language model that how to use these additional delays how to use additional computations which you get through additional tokens at inference time so we initially tried to teach the model

Starting point is 01:39:24 this by fine-tuning standard pre-trained model with these post tokens that is at the end of input prom we add some ten pause token tokens and we start taking output only after the model has seen these additional delay tokens. As you can see, adding these delay tokens gives model access to more attention computations before it starts predicting the answer. But sadly we observe that just adding pause tokens at inference doesn't work. You need to go all the way back to the pre-training stage and instill this behavior in the pre-training stage.

Starting point is 01:40:02 So what we do is in the input, during the pre-training stage itself when you are doing the next token prediction, computation, we add delays at random positions in the input sequence. So let's say the input sequence is the kid is playing soccer, we add cause token say after the kid is. So earlier the playing token was predicted after is now you have additional time step before you predict the playing token and this happens at random positions in the input sequence. that like model learns how to use the delay to open. The frequency of that matter?

Starting point is 01:40:37 10%. So we just use 10% of the operations. And just to be clear, we clip the sequence back to the original length to match the flops with the baselines. And we observe that instilling this false behavior delays in the language model gives gains across reasoning, reading comprehension, natural understanding tasks across the benchmarks. And the reason for like some sort of the language, high level, we show theoretically as well that why adding pause tokens help model. So the reason for that is in language models the way currently the architecture is, the number of parallel operations which model can do is limited by the is limited by the number of

Starting point is 01:41:20 input tokens. So for example, consider a class of functions FM where you have N input tokens and the function is basically m toary operations over these n tokens and note that this m can be much much large larger than n that is like you can choose any two tokens to which like you apply these two array operations and this m can be as high as n c2 so in a two layer transformer if one attention module can attend to can like one time step can perform only one two area operation then the number of such operations which the model can perform will be limited by the number of input tokens. So if you add additional width to the model, you can represent a much larger class of function.

Starting point is 01:42:07 And note that this is not just some toy setting, this corresponds to a lot of real world tasks. Mathematical tasks obviously, but like reading comprehension, for example, which are like one of the most common class of tasks. So why it corresponds to reading comprehension is that Each of the initial N tokens can correspond to one sentence in your input paragraph and the last tokens can correspond to the choice. So the model then takes every sentence, compares it with a choice and tries to get the correct answer. So having additional tokens can allow the model to focus to much smaller sentences and compare to the choices to get the correct answer. Have you looked at what are these tokens attending to? Like are there specifically attending to some sub patterns or?

Starting point is 01:42:57 We didn't, yeah, so we tried to do this that like what is the nearest, like we learned this embedding right for the pause token. And what are the embedding words which are nearest to this pause tokens? But we didn't observe any specific observation. One initial thought was that maybe this corresponds to the English words like delaying or paus tokens. pausing or think more but that didn't turn out to be the case so it's not that it's probably more the actual motivation that how much the attention is and not like what the text is and let's say in the ideal world that let's say you end up learning the embedding which is close to the English word think more then like

Starting point is 01:43:40 this whole thing should have worked with with like simply adding to the input prompt that think more after like you know you could add just some hyphons or periods to the input prompt and that should have worked. But that doesn't work. And actually a lot of people have explored that. What if you simply add additional tokens at inference to the language? You said that there's connections to chain of thought is an open question. Yeah.

Starting point is 01:44:04 But you're already talking about it. Yeah. The connection to the chain of thought is that what was our initial motivation is to understand what is the role of the additional intermediate reasoning tokens which you get in chain of thought. Right. And one thing is that in chain of thought, right? thing is that in chain of thought, the additional, the effective depth of the model is higher because of the intermediate tokens and those are generated auto-regressively. So the effective

Starting point is 01:44:28 depth is much higher. But we wanted to understand what is the role of additional input width. And that is this work about, but it is still open question that like how you can then further improve chain of thought or do the actual tokens which you decode really matter or not. I'm sure you're familiar with the Backspace token work. as well. How do you compare between the two approaches? So the backspace is more like scratch pad and backspace. It's more like easy fixes for chain of thought. They're not about like additional input computation. The other one I thought was kind of interesting was adaptive

Starting point is 01:45:05 number. Yeah. So adaptive pause token that is just based on a problem difficulty? Yeah. So instead of having pause tokens at the end of input sequence, you could always have them on the fly. Let us say in this work currently we always added say M number of post tokens at the end of input sequence and then we let the model decode the way it does but what you could do is you could add pause tokens on the fly say whenever your prediction confidence is below a threshold or whenever after every 10 steps or another direction to explore is what is the optimal number of post tokens model itself tells it or model itself keeps on decoding post tokens You know, it feels that the next token, the main next token has a confidence above a threshold.

Starting point is 01:45:54 So you would need to modify the pre-training accordingly, just to be clear. So in this work, we ignore, we do not teach the model to predict cause token. Whenever cause token is inserted, we ignore the output of the previous step. Because we do not want to let the model digress from its main objective. Yeah. So for the on-the-fly thing or like, let's. letting the model predict itself, how many post tokens you need, you will need to carefully work around the pre-training, and that's like some pressing interesting directions.

Starting point is 01:46:28 And like many people have been following up as well with like multiple token predictions. That is like instead of just predicting the next token, to next four. You predict multi next four exactly. And that's same right. Like now if you're predicting next four tokens, before the actual fourth token you are predicting it four times. So it gets upfade in the last four times.

Starting point is 01:46:48 fade in the loss by like four number of times and more number of tokens will be like taking the weight of predicting that hard token. So yeah, these are some interesting directions. Awesome. Thank you very much. Some of the most challenging work presented at ICLR involved deep theoretical and algorithms work which are historically above our pay grade. However, we perceived a lot of interest in data efficient self-supervised learning which

Starting point is 01:47:15 reduces to weak supervision. This next paper gets an ICLR outstanding paper mention because of its very extensive work ranging from theoretical statistical foundation to empirical support on both synthetic and real data involving a continuous driving vision learning problem. This 10-minute talk doesn't do it justice, but we'll just listen in for a nice segue into the next and last paper in today's vision section. Hi everyone, I'm Polkett. I'm a research engineer at Granica and I'm happy to be presenting to you.

Starting point is 01:47:48 today on our work on towards a statistical theory of data selection under weak supervision. And this is joint work with German and Andrea. So it's no surprise to the audience here that modern AI is hungry for data and the bigger models you have, the more data they consume. However, bigger data comes up with its own issues. For example, as your data set sizes increases, your training, labeling and storage cost increase, your training time increase, and not only that, it's harder to control the data quality.

Starting point is 01:48:16 However, if you think about it, not each data point contributes equally to the task at hand. So here, what I am showing on the left is a GIF of images collected from an autonomous vehicle dataset. And let's say you are trying to train a model to detect car for this case. As you might imagine, in this particular setting, there are lots of images which you are collecting of the surrounding which are repetitive and provide really low value. On the other hand, there are other images which are probably providing much more value to the task at hand. And this phenomena is not new. It has been observed empirically in a lot of past research

Starting point is 01:48:51 works under different complicated settings. In our work, we try to demystify some of these findings and show that this observation is true even in simpler settings. So let's get started. Let me try and train a logistic regression model for the same task which I showed you of detecting car over these images. So what I'm showing you here is misclassification rate versus percentage of data subsample. Let's start. try to understand this plot really clearly because I'll be showing you this plot again and again. So on the x-axis here is I have percentage of data subsamples. So as you move towards the left, you are basically keeping fewer and fewer training samples. On the y-axis, I have the misclassification

Starting point is 01:49:33 rate. So as you go up, you have higher test error. And so the ideal sweet spot here is to be on the bottom left. So the curves which you see below are actually performing better than the curves which will be shown above. The black curve here shows the performance when you randomly sub-sample your data into the training. And the green line here shows the full sample error. So when you train on the full data set. So first surprising result is that if instead of doing random sampling, you do quote-unquote

Starting point is 01:50:05 some sort of smart sampling, you can significantly beat the randomly created data set. So if you smartly curate your data subset, you can do much better than just randomly creating your subset. Not only that, this plot is really surprising because what it shows is that, at least in this particular setting, you can get full performance even after throwing away 65% of the dataset. And not only that, you can actually beat the full sample error by keeping just 60% of the dataset. So these results are really surprising, and we try to understand these better. So let me introduce you to the setup in our paper which we work with.

Starting point is 01:50:40 So we have some unlabeled dataset, which we start with, and we assume a data scoring network. which basically assigns a sort of important score per data point. So the colors here sort of indicate the importance, the darker, the color they are more important. You use this score to rank the data and create a data subset. Note that all of this is done using unlabeled dataset, though you assume some data scoring network presence. At this point, you go and acquire labels and you actually train a model. So there are some salient features of this set of which I want you guys to take away, if nothing else.

Starting point is 01:51:12 First, it's a two-step procedure. So you first do selection, and then that's followed by training. Next, as the talk title says, we have weekly supervised setup. What this means is that we start with unlabeled data, but we assume the presence of this data scoring network. Or in other words, you can also think of this as a surrogate model. What I would like you to imagine by surrogate model in today's world is maybe you can think of it as a foundational model,

Starting point is 01:51:37 or a pre-trained model on another data set trained for your same task. Or maybe a model which is trained on your own data, but on far fewer samples than your whole data set. So the reason this exists is that it gives us some information about the labels better than just random guessing. Next, we study score-based subselection schemes. So we are assigning a score per data point. And one way to think about what these scores might mean is maybe under classification setting,

Starting point is 01:52:04 you can think of these points scores as to be easy examples or hard examples. And finally, though through this diagram, it might seem that we always or keep the top scores, we actually have a probabilistic scheme, so you might want to sample based on these scores or keep the top ones. Okay, for more formally, we basically study the standard weighted empirical risk minimization. That's our setup, so this is like the standard setting.

Starting point is 01:52:28 The only place where the subselection shows up is this term S of XI. And this is formally how a subselection scheme is defined. It's defined by a tuple, Pi, I and WI, where you can think of pi-is as the probabilities of selecting that point and keeping it in the subset. And then once it's in the subset, you might want to put away W while training it during empirical risk minimization. So this tuple completely defines whatever subselection scheme you might want to work with. And this tuple can basically depend on your features.

Starting point is 01:53:00 It can depend on the surrogate model, and it can also depend on some additional independent randomness. So this can be a probabilistic scheme. So in our paper, we formally study the setup under various different settings. So some of the key ones are we studied under bias versus unbiased sampling. So in unbiased case, you have an unbiased loss function post-subampling. So your weights would be inversely proportional to the selection probability. And these unbiased selection schemes for people in the audience who are aware would cover and generalize schemes like influence function-based subsampling or leverage scores-based sub-sampling.

Starting point is 01:53:34 We also study both high and low-dimensional SMEFEMSAMP. So in high dimensional asymptotics in particular we study like proportional high dimensional asymptotics and we have exact theorems. And finally, since surrogate model is an integral part, we also try to study both imperfect and perfect surrogate models where perfect surrogate models basically means you know the true data distribution. So like I said, we have a lot of results in the paper. But for the purpose of this talk, we are going to, I'm going to show a very simple setting.

Starting point is 01:54:02 So we are going to be working under a binary logistic regression. We are going to show results with binary logistic regression. and a subselection scheme is a parametric scheme which looks like this. Let me demystify this for you. So PSU here is basically probability under a surrogate model. So you have a surrogate model which can give you the probabilities of whether it's a zero or one based on its information, its knowledge about the world. Then the term here, which you see this p times 1 minus p, it can be interpreted as an hardness score.

Starting point is 01:54:32 So on the bottom, there's plot of this function p1 minus p against p. and if your P is near 0 or 1, this term is near 0. So basically you are very confident. Your surrogate model is very confident that this data point is either 0 or 1, and those can be thought of as easy examples intuitively. Similarly, when the value of this function is high, like your P is near 0.5, you are really confused. So it's like hard example in a sense.

Starting point is 01:54:57 Finally, we have this parameter alpha, which can be thought of as an hardness exponent. And what it does is like if this alpha is greater than 0, you would up-sample hard examples into your training subset versus if it's negative, you will up sample easy examples. It's zero, it's random. So we take this sub-selection scheme, and we try it on two different datasets.

Starting point is 01:55:17 So we have synthetic data sets, where we have isotropic Gaussian covariates, and then we assume that the original label is generated as a linear function of your features. Note that this can be well or mis-specified. What I mean by that is, like, if the actual true labels have generated using, logistic function. That's a well-specified model. You are training it under logistic loss,

Starting point is 01:55:37 but it can be any other linear function of your original features. And similarly, we also have results on this real data, which I showed you before in the beginning. So first of all, what I want to highlight is that we have theory for synthetic data, which predicts exact high-dimensional asymptotic error. So a lot of plots don't get jaded. So all of these nine subplots like I'm showing the same misclassification rate against percentage of data subsample under various different settings of generated synthetic data. Different colors are different values of our hardness exponent, upsampling easy or hard.

Starting point is 01:56:12 What I really want you to focus on here is that the dots are basically the results of empirical experiments or simulations versus solid lines which you see are not really interpolations of those dots. Those are the predictions from the theory. And as you can see across various these different settings, we were able to exactly predict the high-dimensional, the test errors. Right? And surprisingly, even this simple setup demonstrates,

Starting point is 01:56:35 many interesting phenomena. So next, I'm going to basically give you the main takeaways for using a mixture of experiments on real data and synthetic data. So let's get going. So I guess our first main takeaway is that the unbiased subsampling can be suboptimal. So here in the plot, what I'm showing here is in red, we have unbiased subsampling, whereas in blue we have biased sub-sampling. And as you can see, the blue curve is below, and it can be actually quite significantly below the red one. In fact, in our paper, we have many different theorems and specific constructions showing that unbiased subsampling can actually be arbitrary worse. And as a reminder, unbiased sub-sampling generalizes a lot of popular techniques, for example, influence function-based

Starting point is 01:57:20 sub-suffling, leverage score-based sampling, so on. Not only that, like, so this is, these are some plots on real data. So what I'm showing here is the different parameters, all these different subplots are generated under various different settings of regularizations, SNR, so on. But again, don't worry about all the details. All I want you to focus here is that the blue curve is forming the lower, is the lowest one, in all these different settings, right? And so here, again, as a reminder, blue curves, alpha is negative, you are upsampling easy examples in your subset.

Starting point is 01:57:51 And so in these, in high dimensional settings, actually choosing easy examples is better. And again, this is very surprising, but it has also been seen previously in previous works under specific settings. However, we show that this is much more general. Next, even more surprising, you would come to me and say, hey, better surrogate model should always lead to better subselection because they would encode more information about the labels. Interestingly, we found that's not always true. In fact, better surrogate models do not always mean better selection. So here, what I'm showing in different curves, the different colors is basically when I generate subselection schemes with different values of different, different

Starting point is 01:58:31 of my surrogate model. And you can see the darker the color, the stronger the surrogate model. What you can see is the lighter color is again at the bottom. So at least in this particular setting, using weak supervision, a weak supervised model performs much better than the strong one. And this is encouraging because it says that, oh, you can do something under weak supervision and you don't need to actually keep using all the labels.

Starting point is 01:58:53 And in fact, using stronger models can hurt under certain settings. So this was very surprising. was very surprising. And finally, what I want you to, what we see is that you can, sub-sampling can actually beat full sample training. And this is not only true for complex model, deep models, your advanced clip-like task. This is true for a simple binary logistic regression kind of setup. Here I'm showing you results on synthetic data. And what you can see in this particular case is you just keep 40% of the data and you can improve on the misclassification error. And the main intuition here, so we observe this across many different settings. I just cherry-picked one.

Starting point is 01:59:31 but we basically observe it when the models are mis-specified. And the intuition there really is that when the models are mis-specified, not each data point is adding equal value or equal information for your training objective, since your losses and models are mismatched. So to conclude, we had some surprises during our work, such as popular techniques using unbiased something can be suboptimal or use of weaker surrogates can beat. However, the main punchline, I would say the main,

Starting point is 02:00:01 insight from our work corroborates the previous research, which is uncertainty-based sub-sampling can be effective, though story is interesting. Choosing hardest can be catastrophic and depending on settings you might want to upsample easy or hard. And I hope this will get your appetite. If you want to learn more, I'll be at the poster session. Thank you. Our last vision paper gets another ICLR outstanding paper mention for learning strong image encoders from one long unlabeled video. It does two things. Introduce a walking tours data set and introduces Dora,

Starting point is 02:00:42 a new image self-supervision method demonstrated on that data set that outperforms Dino when trained on just one video and leading to emergent tracking of objects through a video, even through occlusions. Hello everyone, I'm Shashang and I'm happy to present our work entitled Is ImageNet Worth One Video, Learning Strong Image Encoders from one long unlabel video. This is a joint work with my colleagues, Mamshada Rezvi, Joao, Yuki and Janis. Self-supervised learning has allowed pre-training of neural networks to scale beyond the size of labeled datasets.

Starting point is 02:01:18 By adapting to evolving data without human intervention, SSL avoids the time and expense of continuously relabelling large data sets. Since SSL methods learn directly from the data, it overcomes the limitation and potential biases, associated with manual annotations by human experts. A major limitation of current SSL systems is that they are typically trained in a highly controlled setting of using ImageNet without labels. Additionally, many SSL methods rely on carefully designed det augmentation techniques such as image transformations to create the supervisory signal for the model. Finally, extending these approaches to a more complex

Starting point is 02:02:05 Dual world scenario with rich contextual information and diverse data modalities also remains a significant challenge. The rise of video data presents an exciting new frontier for self-supervised learning. Videos inherently contain a wealth of temporal and multimodal information, such as visual, audio, and motion cues which can be leveraged to learn powerful representations without the need for extensive manual labeling. Moreover, a large number of videos are available at a massive scale on platforms like Instagram, YouTube and others, providing an abundance of divers and uncontrained data for self-supervised pre-training. We venture in this direction and introduce a new data set of open source ecocentric videos

Starting point is 02:02:55 called Walking Tours. We then introduce a new self-supervised image pre-training method aimed at learning from videos, where we learn to detect and track objects over time in an end-to-end manner using the Transformers' cross-attention. The Walking Tour data set consists of 10 egocentric videos captured from different cities in Asia and Europe. These videos are dense in semantic categories, much more so than in movies.

Starting point is 02:03:25 They have 4k resolution, contain few or no shortcuts, are hours long, typically between 1 to 3 hours, and are under the CC by license. We now introduce our framework called Dora. Here we repurpose the MSA block in VIT to perform multi-object tracking without using any of the shelf, object detectors or optical flow networks. From an input frame, the output of the second last layer of the teacher model is used by a multi-object tracker to generate multi-object masks.

Starting point is 02:04:02 We use those masks to feed to the student model and apply a distillation loss between the CLS token embeddings of the teacher and the student network. In the multi-object tracker, we first obtain the query, key, and output embeddings at the last MSA block of the Vision Transformer. From the multi-head attention maps, we draw a subset of K-heads and form object prototypes by pooling over patch queries. We refine these object prototypes to discover distinct objects using the Syncon Knop to improve correspondence between object prototypes and patch embeddings.

Starting point is 02:04:40 Finally, we then track the objects over frames by cross-attention between patch key embeddings. We observe that Dora leads to emergent attention maps from the CLS token of distinct Hedge in a VIT. Here we use red, green and blue masks to detect and track objects. The mixed colors in yellow and cyan indicate the spatial overlap of two objects in a single head. The track objects are now incentivized to be diverse by introducing the synchonob clustering of patch embeddings, where we observe three well-separated objects. Here is another such example. In just one video with 200,000 frames, Dora outperforms Dino pre-trained on ImageNet 1K on object discovery.

Starting point is 02:05:30 semantic segmentation. It is also on par on object detection, video object segmentation, and multi-object tracking. We account for the large distribution shift between pre-training and downstream data sets by fine-tuning on different image-based data sets. While Dora, pre-trained on one walking tour video is on par with Dino, pre-trained on ImageNet 1K, pre-training Dora on all 10 walking tour video outperforms Dino. Thank you for your attention, you can find our paper, data set and code using the QR code here. That was the end of section B of our podcast, which covered adding special register or pause tokens for improved performance and a survey of state of the art week and self-supervision

Starting point is 02:06:16 thinking. We turn now from Dora's to Loras. Can you say attention? My bien. Section C features the best of a wealth of papers, exploring different methods for improving different methods for improving the performance of transformers across long context extension, memory consumption, and reducing communication overhead on massive GPU clusters. Let's dive in.

Starting point is 02:06:43 Today I'm going to share our work, Long Loraefficient fantuning of long-context large-language models. The authors are from the Chinese University of Hong Kong, MIT, and NVIDIA. So first, I will give you an overview introduction about our work. Long-context ability is very important for large language models. With long-contactability, the large language models can understand books with tens of thousands tokens, but the challenge here is very clear. Fan-twinning large-language models to be long-contacts very expensive. So, Long-Lore is our solution, an efficient long-context vent-twining methods.

Starting point is 02:07:22 The key idea here is sparse attention and low-rong adaptation. It saves notable GPU memory without sacrifice. accuracy. So, but in addition, we also introduce a data site. It named the Lung Apaka, the first long instruction falling data site in this area. So let's go into detail about Lung Laura. Our methods contain two key innovations. The first one is shifted sparse attention. We split the contacts into several groups and conduct attention in each group individually. And in the half amount of the attention had, we shift the position of the token. by half groups. It ensures the information flow between the neighboring groups. The second one is the

Starting point is 02:08:06 enhanced laura. In addition to the lower weight, we further make the imbiting and normalization to be trainable. Let's go into the detail about these two innovations. So the implementation of the shift sparse attention is quite simple, that involves only three steps. The first, we split the features into two trunks along the higher dimension. And the second, the tokens in the first trunk are shifted by the half of the group size. And the third, we split the tokens into groups and reshaped into bad dimensions. So the implementation is very simple. And here, there might be some potential information leaking and it can be prevented from the attention mask. So this implementation is very easy.

Starting point is 02:08:55 and compatible with flash attention too. And we have conducted detail and ablations and compression to other efficient attentions, including dilated attention, block sparse attention, and stress sparse attention. Our goal is to find training pre-training large language models to be long-contacts. However, these attention patterns are developed for training from scratch transformers. They have some gaps to the full attention,

Starting point is 02:09:22 so that's why. But here, during the training, the pre-train model, we use the full attention, and that is for the pre-training, so that's why our shift sparse attention performance better than these other choices for the long-contacts fan training, and it not only enables efficient fan training,

Starting point is 02:09:42 but also support full attention testing. Our second key insight is the enhanced Laura. In addition to the long-lora weights, long-lora also makes the imbiding and normalization layers to be trainable. We empirically show that this extension is very important for the context extension. It only introduced a minimum number of additional trainable parameters,

Starting point is 02:10:08 but it can close the performance gap between Laura and the full fine training. We have conducted the perplexity evaluation on proof pile and PG-19. We examine the effectiveness of whether using the shifted sparse attention and the improved laura, We show that our method achieves the comparable performance to full attention and full fine-tuning baselines with better efficiency.

Starting point is 02:10:34 Long Langer is able to fine-twin a 7 billion pyrmometer model to 100,000 contacts, or a 70-billion model to 32 contacts on a single 8-GPUA-100 machine. We further extend our fine-twinning to longer contacts, and we progressively fine-twined a Lama 3 model to one million contacts with shifty sparse attention and also get very good ratio performance in the needle-in-hastik experiment. And we also evaluated our methods

Starting point is 02:11:07 on long-context benchmarks, including the long-bench and long-eval. We compare our model with GPD's 3.5 turbo and other Lama-2-based long-contact models, like Okuna and long-chat models. It shows that our model present comparable or even better performance, than these 7 billion long-contents models,

Starting point is 02:11:28 and we only use about 4-hour and 0.3 billion tokens for the supervised fan training. And the key inside here is the large badge and long epoch in addition to the long laura. Okay, and in addition to these measures, we also present a solution, a data-size solution, for the supervised fan training, with our self-collected long instruction volume data size.

Starting point is 02:11:52 It named the long alpaca. Because self-supervised training is very important for improving the chatability, the chatability of large language models. We designed several different kinds of questions, including the book question, short question, summarization, paper review, and other questions. There are about in total 12,000 questions and answers in our data sites. We collected long instruction data from PDF books and other blogs that are available and licensed online. we use OSAR model to get clean text data, and this process requires many handcraft with dirty works

Starting point is 02:12:30 to find out the high-quality long questions and answers. So here are some examples of using our methods and our data. So we name this model long or PACA, and if we give our model a very long paper and our review guideline, we can ask the model some suggestions on how to make the paper have more chance to be accepted. And it will give some suggestions about the contribution, about this paper, the novelty,

Starting point is 02:12:58 and how to design experiments, and some other applications and future directions. The second example here is also about the paper. We gave our model two papers and one paper from CVPR and another one from the SIRR. It can summarize the difference between them. In structure and in technical depths and theory and experiments, and some languages, stealth.

Starting point is 02:13:25 So this is the second example. And we can also give our model an economy outlook, like this one, and which is also very long with tens of thousand tokens. We can ask the model to predict the economy in the future. They will give us some high-level suggestions on the economy growth and inflation and some other factors.

Starting point is 02:13:51 So in addition, our... long alpaca can also read very long fictions and books. For example, the famous book's the three-body problem. I asked a question that why did Yewenzsche contact aliens and why did she feel regretful in the end? Our model can also give the accurate answer with detailed explanation, like what's the Yuenjia thinking and his history and some of his future thinking like this one. And here is the compression. If we only provide the the book title, but not the content of this book, to the model, the answer will be much less informative and somehow irrelevant to the question, because maybe someone may think this

Starting point is 02:14:33 paper, this book, have been pre-trained in the large language model, so this is a compression. So if we give the model more content about the detail, about the book, the answer will be more concrete and more detailed, so this is the compression. We have released all our code models and data, and they are available on GitHub and Hacking Face. So we'll come to discuss online. So I think this is a good release. Okay, that's all about my presentation. Just to make sure you don't miss the punchline using shifted sparse attention,

Starting point is 02:15:11 Long Laura extends Lama 27B from 4K context to 100k, or Lama 270B to 32K context on a side. single 8XA 100 machine with almost perfect needle in a haystack benchmarks. Long Laura extends models context while retaining their original architectures and is compatible with most existing techniques like Flash Attention 2. We can compare this directly with our next paper, Yarn, efficient context window extension of large language models with Bowen Peng, chief scientist of Noose Research. So yeah, hi, hi everyone, I'm Bowen. I'm the chief scientist at News Research.

Starting point is 02:15:51 So we've done long context extension with yarn, which is a new method. Six months old method now, but it's still like was new at the time. And basically why I started working on this was because there was some random guy on Reddit posted about positional interpolation. So PI was the basic idea for interpolating rope, which was the first method that actually worked for extending the context size of any language models. So before this PI method, there was no way of extending the context efficiently. You had to repretrain the model entirely with 500B tokens.

Starting point is 02:16:29 And after this method... Without any fine-tuning... This method work without any fine-tuning, right? No, no, no, it needed fine-tuning. Small, small. That's the benefit. Yes, yes. So you needed a tiny bit of fine-tuning,

Starting point is 02:16:44 which is like 1% of the pretraining data set, which is much, much, much better, right? Because the longer the context, then the... the context and slower the training is so you want it to be efficient so after this guy published the PI which it was a guy called Kayo Ken Dem on Reddit there was also meta like the just next day they published their paper because they're they were like yeah we need it we need to put it out right and I was thinking like this PI is wrong there's something wrong with it because you are interpolating all

Starting point is 02:17:15 the dimensions which is not ideal because there are some dimensions that don't need be interpolated basically because those dimensions rotate a lot so basically in rope all the dimensions rotate like different speeds and some dimensions rotate really really fast and you don't need to interpolate... So you need to extend it but not necessarily to change what is working okay yes in the first place so so there are some dimensions you want to extend and some you want to touch those rotating fast dimensions they attend to local tokens so when you stretch them you're forcing them to attend to longer context

Starting point is 02:17:50 which is not what they are used to. And some of the dimensions, which we call it by lambda, like the wavelength of the dimensions, they are really, really long. The wavelength is really long, so they are used to attending to really, really long contexts. So those are the dimensions where you stretch, you are interpolate,

Starting point is 02:18:07 and the short dimensions with short lambas, you don't touch them. So that would give you this line, like this red line where it goes to like this blue line. So there's PI and NDK by parts. by parts. So we reduce the losses by quite a bit and also like after training the perplexity decreases by a lot right also pass key like this is like 50% and goes up to like much higher there's some degradation at the end I'll

Starting point is 02:18:35 explain like after but you can see that PI was not great and this is much better and then we just played around with all the parameters with yarn here you make sure that you compare apples to apples It's a different. It's different. It's different. Exactly. So we made sure to make every model like as good as possible.

Starting point is 02:18:59 So we made sure to keep all the models at the same like context length. We made sure to train them the same amount. So we're not like biasing anything, right? So we made sure that this is the lowest possible for all the methods. And then we used yarn, which is we played around with the temperature and we found this was really important. important for long contexts especially at like 128k this is really really important like at 32 is less but you can still see that it goes down and here it pushes the past key retrieval to like 100 and then like it pushes a really high

Starting point is 02:19:36 this fast key accuracy so basically with yarn we can do a really efficient like 400 training steps extension of the models this is like one how we like shifting to today how it stand against other methods okay so so the most promising rope rotary and positionally bending method on right now is long the rope which is paper by Microsoft I think that paper actually took our idea of yarn which is this NTK by part and yarn and they try to find the parameters like using machine learning so instead of us hand tuning T and this like optimize it yes yes using machine learning

Starting point is 02:20:18 earning and they found that it performs even better than yarn right because that has basically it sophisticated hyperparamination exactly exactly that's exactly do you actually need a fine-tuning data set for this or just any data set will do like anything that looks like a common crawl yeah anything that's actually longer context like ish like you don't want for example this is 32k you want to at least train with maybe 15 or 20k you don't you want too short of context training here okay you still want some long context. And how this method of making model with short context to a model with long context

Starting point is 02:20:56 compare, for instance, with Google models that train for all the time with long context? So what I think my idea about it is like the more you train, the better model is. So you cannot beat compute by just using methods. At the end, it's a data, right? Those models... Okay, you just have more resources. They have more resources. We only Like if they had used yarn from the beginning, it would be even better probably the model, right? But they didn't use this. They probably used like alibi, I don't know. Don't cope me on that. But the other impression is alibi is more lossy, so it will do worse on. Yeah, maybe. It's not, it's like people have not yet done a full ablation on alibi because there's no big models for alibi.

Starting point is 02:21:40 There's like bloom, which is bad because it's so old. There's no like 7b llama of alibi. It will be great if meta can, can release a Lama, rope, alibi at the same time. One with rope, one with alibi. So we can't do it? I mean, that seems like something that you can do. Oh, no, no, pretrained the Lama 70, we cannot. We cannot. It must be pre-trains.

Starting point is 02:22:02 Yes. So like to compare rope with alibi, we will need to have the same data pre-trained for Lama, the same architecture, but just swap this two. And then see the results. This still does not give you like, ring attention, like one million context window right? Oh no no no ring yeah ring attention is efficiency yeah that's like compute

Starting point is 02:22:23 efficiency right okay yeah yeah yeah yeah yeah yeah allows you to I was confused by that as well it's actually just the compute efficiency yeah and so by that alone you can extend as much as you want with short like training yeah yeah but then it's become so not efficient compute wise so you need something to Exactly. To treat it, so you need the ring attention to treat that. Because otherwise without ringing attention, you cannot train a 7B at 120K. Like it takes too much memory.

Starting point is 02:22:55 You need multi-node, which is hard to get. So with ring attention, you can train a 7b on a single 8, a 100 node. So, yeah. Regarding that, there is not limitation on the context length that is given, like, good until some... I think with further explorations on road. and alibi, this problem will be gone. Like, this will be a problem of the past. Like, actually, this rotary position interpolation thing,

Starting point is 02:23:22 it's going to be something solved. Because this past key, after you train even more, this is 100%. It's only a data problem there. Like, at the end, you will need good data for multi-hop reasoning, for multi-turn conversations. That needs data. Like, the model cannot just magically learn those abilities.

Starting point is 02:23:39 Yeah. That's those abilities, yeah. Need a lot of data because it's no the essence of like learning the text and just know to need to something to encourage this to how to extend it you will need high quality varied data so that that's the most hard part right now like you have good a small like maybe 20 billion really long context like with a lot of variety right like some really good data that will that will solve and

Starting point is 02:24:08 like it's not solved it entirely lunch and stuff like that there you need some architectural changes right so so are you talking about like infinite attention yeah yeah yeah infinite context it's kind of like a misnomer right now because it's not context it's just infinite infinite you encode it and yeah yeah so you still forget stuff but you are remembering the important things in the infinite context in quotes hopefully and the problem with that is that there's some tasks that are computationally not possible to do so this is a reason another question that you can have long context that you can attend to but it's not

Starting point is 02:24:49 necessarily mean that you're effective or use it properly exactly I did a work on certification with a lot of examples many classes and then you see that the aggregation that it's not utilized as well because you have so many classes that you need all the four 8 K context so yeah not continue to increase and sometimes even it's decreased when you Yeah, yeah, yeah. So this is something that is still like something that need to be improved, right? It is, it is.

Starting point is 02:25:21 There's a lot of dimensions in this. We focus on a single dimension of like rotary embeddings. There's like data, there's also like all the kind of reasoning, the model capabilities. Attention is not like the end of everything, right? Because there's some tasks that attention cannot solve. So you would need like maybe some stack, maybe like, maybe like augment the model with like a memory or a stack. Because if you want recursive reasoning,

Starting point is 02:25:46 like actually recursive stuff, you need a stack. Or you need a bigger model. Or yeah, but a bigger model only pushes this problem into the future. Then you become a rack for your context. Yeah, there's something like that. I don't know. I would not know what you would look like, but something like that, yeah,

Starting point is 02:26:03 during complete, right? Like something turned complete. That would be really close. So I have a couple bigger questions if you don't mind. Do you think context extension is solved? I think it's not yet solved, but it's close to it. Like, when you have like Claude, which is 128K, there's like GPD 128, 200K, at some point when you have one billion context length, that's it.

Starting point is 02:26:24 I don't think you would, don't quote me on this, but after one billion, it's kind of becoming a problem of what do you want to feed to the model, right? One billion is like really, really big. And there's no such thing as one person cannot generate one billion like. Yeah, like something like that. But it will be have its own use cases, like $1 billion, $10 billion, $1 trillion tokens. But I don't think like for most use cases. Yeah, yeah, video, DNA, you have all kinds of like auto-regressive.

Starting point is 02:26:55 Training a model inside of a model. Oh yeah, yeah, you could feed the parameters in the ICL. Because $1 billion is basically a dataset. Yeah, yeah. You could do ICL on the weights of my LF. But that's crazy. And then the follow-up question is just on News itself. News is obviously super interesting as a startup.

Starting point is 02:27:15 You are chief scientist. Like what kind of research are you funding apart from this stuff? Oh, so we're doing all kinds of research. So anything related to large-engage models or a diffusion model, we're trying to push this. We're trying to accelerate it. It's too broad. Yeah, it's really broad. But when there's a good idea, people come to us and we can maybe work something out.

Starting point is 02:27:33 Because when I joined News, it was actually just, I had this crazy idea. Let's try it, right? So apart from context extension, what else is news interested in, you know, just currently? Yeah, yeah. So it's basically LLM. So like agentic AIs, we also have like people who are working on making things reason, rag. You know, like, I'm sorry? Synthetic data is one.

Starting point is 02:27:55 Yes, yes, Technium is really, really big on today's data and he's pushing like, what? He could not come here today. He comes to San Francisco a lot. I hang out with him every day. But he's really busy, unfortunately. I would love you for him to come here. A lot of their synthetic data is like, you correct me if I'm wrong, it's like

Starting point is 02:28:14 getting GPC4 to create textbooks, right? Yeah, yeah. But that's just GPC4 distillation. It's not, like, it's not a science. Yeah, yeah. But I'm just, I'm thinking for, is there something else that I should be looking at for synthetic data that is promising? So I can take that.

Starting point is 02:28:31 Yeah, I mean. So synthetic data, it's a lot of knowledge that can come from the model. And it's not only to use the data that the model generate. You need to decide which one to use, how to filter it, which kind of question do you want. For instance, the paper from Microsoft, I forgot the name, they have three additions. Like textbook is all you need. Yeah, so how to build it, that the model will gain more information and knowledge from the big model and also you can think about it as learning from human data you also use a lot of

Starting point is 02:29:15 different humans and a lot of different processes to generate the data and then learn from it so essentially learning synthetic data has become very important and also improve the model itself because as we can learn as humans and improve by getting feedback from other resources sources of of knowledge and information, we can improve. So there is a lot going on in to improve the models. And if you look back previous years, so you see a lot of the best models are solely because of better generation techniques. For instance, Orca, they use different, like the reasoning chain of thought

Starting point is 02:29:59 and evolving instruction of Wizard of RM, our paper, or in the genie, content grounded, that show parity in ground, grounded task generation data and FI that we talked about so all of them and I guess like a lot of the big company all do it in the like in the their garage so don't tell us and share information how they improve GPT4 but all these like preference data that annotation and feedback that you use from something that go into improving the model and generate embedded synthetic data to improve Yeah, I mean, syndic data is like...

Starting point is 02:30:40 That's a good summary. Yeah, yeah, thank you for the good summary. Like, I think that from your summary, synetic data is faster to generate. Like, humans can evaluate the synthetic data really fast. They cannot generate the synthetic data fast. So, like, it's kind of... Yeah, yeah.

Starting point is 02:30:58 Are you also involved in, like, the World Sims stuff, or is that more Karen? Yeah, that's more Karen. Okay, because... Technium is not... One form of interesting synthetic data that HuggingFace is doing is basically synthetic data for app generation, a website generation. And they've done, they released a dataset for IDIFIX, which is their vision data set. It could be interesting for you guys because it would directly plug into WorldSim and WebSim.

Starting point is 02:31:22 Yeah. And that might be interesting if you guys are going to. That would be cool, yeah. Cool. Well, thank you very much. Thank you very much. In case you missed it, we referenced a lot of context extension papers in the poster session discussion you just heard. Kiko and Deh.

Starting point is 02:31:35 on T-Iles. While training super hot, long rope, ring attention, infinity attention and textbooks are all you need. All are linked in the show notes, which we really hope you are looking at to complement the poster session audio. Next, we have an ICLR outstanding papers. Honourable mention paper for model tells you what to discard. Adaptive KV cash compression for LLMs, aka the fast gen paper. This modifies the standard KV cash algorithm, to selectively discard long-range contexts for attention heads, targeting local context, and non-special tokens for attention heads targeting special tokens.

Starting point is 02:32:14 This results in a 40% memory reduction for inferencing Lama 67B. Hi everyone, I'm very proud to share our work. My name is Suu, and I'm a PhD student from UAUC. So our work is called adaptive KV-Cache compression for large language model. So before we dive into the paper, I first want to just a quick recall. So what is KV cache anyway?

Starting point is 02:32:41 So during the encoding, every time you encode a token into the QKV vector, so whenever you got a new token, you encode it into a new query vector, and you take the dot product between the query vector and all the historical K-vectors, then you use the dot product to multiply with way vector, and you got this final attention output. So whenever you add a new decoding step, you are using a new query vector, but you are using all the historical kV vectors. So one simple way to save the compute is you can start all those kV vectors, those historical vectors, special memory space, to avoid future recomputation. And every time when you got a new core vector, you just edit into this specific memory space.

Starting point is 02:33:28 So in our scenario, this specific kV cache memory space is what we want to. to compress. So how big is this KV cache? How much memory will it consume? So basically, it's proportional with model size and sequence lens. If you encode it in FP16 or BF6 pin, it will consume two bytes per model weights. Just to take an example for the Lama family, we're very used. If you want to do inference with the sequence lens with 512, for the largest Lama model, you will need 20 gigabets. However, considering that we are dealing with model with 10K or at least 10K these days, you need like over 200 gigabytes for the largest model, and that's simply like KV cache memory consumption. So this is huge.

Starting point is 02:34:18 So we all acknowledge that actually KVLM in France is memory bound instead of compute bound, because auto-regressive IOM generation, these are sequential steps and very hard to paralyze. So here, I borrow one figure from the very famous VR and paper. It shows the memory layout when you are serving a 16 billion language model with a batch size equals to five. So as you can say, even if you keep this very small batch size, the KVKHash will still consume like more than 30% of memory space. So yeah, so considering this huge memory consumption of KV Cash, what are existing solutions? So an optimal solution is to what we call KV cache offloading. So basically try to upload KV cache to CPU or MME.

Starting point is 02:35:11 However, it will not trivial latency because usually some devices will have limited bandwidth between the CPU and GPU. So I think a better solution here is to just reduce the GPU memory required of this KV cache space. And you can do this from actually both system side and model sides. there are a lot of work, like excellent work from the system side, for example, the VLM paper. And here we do parallel work with ZOM, and we work on the model side. So just a quick overview of the fast jam method.

Starting point is 02:35:48 It's a KVACCH-in-Veconomic algorithm, and it's designed for efficient LOM inference. The most important part is it's on a fly, which means you don't need any kind of pre-training or fine-tuning, and also it's model agnostic. which means as long as you have an auto-regressive IOM, you can apply our method. And also, it's asognal to quantitation or distillations.

Starting point is 02:36:14 So, as you know, our goal here is to improve efficiency. Well, at the same time, we want to maintain the accuracy as much as possible. So improve efficiency means we need to drop as much as KV Cash as possible. And to maintain the accuracy means you have to match the first. full attention score scenario, which means you have to match the full attention map before you do this kivkatch compression. So just to give a hint of result before we diving into this method, we actually without any significant performance degradation,

Starting point is 02:36:50 a fast-gen can achieve like over 40% so that's almost half as a memory space for the largest Lama model. And for those smaller level model, we can only achieve 55 memory reduction. So what enables our method will motivate us. Actually, we borrow some insight from the bird area. So when bird came out, there is a famous study paper, which is called What Does Bird Look at? It's simply like try to display different attention head patterns of the bird model.

Starting point is 02:37:27 For example, we found that some attention height in bird model, attend all the contacts very broadly, while some will only focus on the next token, while some only focus on the special token, and some focus on the punctuation. So one straightforward way is if this exists in birth, we will still use in auto-regressive LIM. So for example, LAMA model.

Starting point is 02:37:51 So here we started this Lama model, and we display different head pattern from the same layer. As you can say, the first run, actually most of the attention map focus on the green part, which is a special token. So here, the special token means the start of sentence and the end of the sentence. And in the middle, you can see most of the attention focused on the local context, which is also the same as Bert.

Starting point is 02:38:15 So here, our conclusion is, actually, this kind of attention map focus, all we call attention spasticity, also exists in Lama model, always in autoregressive LOM model. So based on this, how to do this, how to do this? compression. Yeah, one straightforward way is we can, if some has the only focus on the special tokens, that means we can discard all those non-special token for it. And for some had the focus on the local contacts, we can discard all the previous long-range contacts. And for attention had that broadly attend to our tokens, we can simply keep the full attention

Starting point is 02:38:55 catch before, because we still want a performance that matched the forecast scenario. Yeah, so when does this fashion happens? So in a normal inference step, after we get the dot product of query and k-vectors, which here we call it attention map, so it's Q-Motipa-K. Based on this attention map, we search for optimal eviction strategy. Here we denote the eviction strategy as C-sharp. So this kind of eviction strategy search is actually designed for, per head per instance, because we found that, as you know, different head display completely different attention sparsity.

Starting point is 02:39:38 So that's why we need to find the optimal eviction strategy for each head instead of like all cash or had using the same eviction strategies. Yeah, so given this fund optimal strategy for each head, we use this strategy to compress the carry cache for each head. So here, after this compression, those KVs, we got a much smaller size, and we started back to the KV cache. So this time, we got a smaller KV cache, memory size. So the most important part is how did we do the strategy search? So given this attention map, A, the goal is to pick the optimum policy C-sharp that can recover as much as possible. So to ensure that the recovery is very high performance, we set a recovery ratio T.

Starting point is 02:40:37 So this recovery ratio is a hyperparameter we have in our setting, and it's weighted by L1 norm. So we want to recover the attention map as much as possible and also with the minimum memory cost. So just to give a hint of performance, When we said the T as a very high value, which is 99%, fast chain can still achieve over 40% compression ratio for the largest Lama model.

Starting point is 02:41:06 So here, the aviation set, strategy set is designed as concludes different kinds of strategy, for example, only pertaining the special tokens, only pretending the punctuation token, or only pertaining the local context token. All we simply try to combine them together for better for better accuracy. Here is a strategy

Starting point is 02:41:26 distribution of the fast-gen method. As you can see, actually a very interesting phenomenon is that in most layers, they are occupied by green color. So here, the green color means you can only using those special tokens to recover most of the attention

Starting point is 02:41:43 map by 99%. So this high sparsity in large-dage model encourage us to use only special tokens to memorize most of the important information in large-dense model. Yeah, so it comes to the experimental size, we conduct our evaluation on both the instruction fine-tune model and based language models, and the model size range from $7 billion to $65

Starting point is 02:42:12 billion. And for the instruction model, we use Obika Evol, and we calculate, we use GPD4 as a judge to calculate the run rate of fast gen over the full cache scenario. So full cache means no compression, no evictions, just the original attention. And we also tried the Lama model,

Starting point is 02:42:31 which is a base model, and we evaluate by code, mass, question answering, and reading comprehension. Those are all generative tasks. Our compared method is those non-adaptive KV cache method, and also we compare the full cash scenario

Starting point is 02:42:47 and set up as a baseline. So ideally, the ring rate should be 50% if you can reach the same performance as the full cash scenario. So here we display the performance and memory trade-off of the instruction fine-tuned Lama model of different sizes. As you can say, for smaller model, actually the performance grow linearly with the KB cash budget. But for the largest model, which is on the right, you can say even if you only pretend like over, 60% of memory, the accuracy is still the same. So the larger the model is, the more age the fast gen will burn you. We also conduct the memory footprint reduction.

Starting point is 02:43:32 So as you can say, if we take a ring rate of over 45 as a no quality regression, actually fast gen can achieve over 40% memory reduction in the largest Lama model. And for smaller model, we can achieve over 15% memory reduction. So here is the conclusion is fast gen is a method that improved the efficiency of LLAM generation through adaptive KV cache. And the result is very good. It can achieve up to 40% memory reduction in the largest Lama model. And for smaller model, it's over 50%. And right now, we're working on the VLM implementation.

Starting point is 02:44:16 because we found page attention suits our method best. And in the future, we'll also try different optimization techniques, for example, quantization and distillation and see whether it will still work very well if we combine them together.

Starting point is 02:44:33 And also, I will try working on group query attention to support the most recent Lama model. Yeah, thanks. Memory reduction is great for the single GPU case, but when it comes to the very, very large GPU clusters, the networking communication overhead starts to become a bottleneck.

Starting point is 02:44:52 We caught up with deep speed from Microsoft research to learn about what's new with Zero++. Yeah, let me start by a short intro of Zero. So here let's particularly focus on Zero3. So in Zero3, you have everything scattered. Your model weights are scattered, your optimizer space are scattered, and that means whatever you want to do a training step, forward, backward, and the opportunity. another step, then you have to do a collective operation on those.

Starting point is 02:45:21 So that would add up to 3x of the model size. And that's 3x of the model size on each step, it's a huge overhead if you scale up your training. So even if we consider the cutting edge cluster where we have like 8 infinity band connection on each node, we have RDMA on each switch, but that's still not enough. So in the end, we'll scale up your training. that's going to short routine speed.

Starting point is 02:45:49 So that's why we want to propose a set of optimizations to improve the communication when you do in zero. So here, 0 plus plus, it can help you by reduce your communication volume by 4x. So let's take a short breakdown. So first, in the forward path, that's where the first path of the algorithm on model with what happens.

Starting point is 02:46:14 And here in this stage, we use block-based quantization. So blockbins, we are doing the quantization on each slice of the tensor, so that we make sure we keep the position, and then cut the volume by half. And then for the backward paths, we use the heterogeneous partitioning from the modelways. So that means instead of scatter these model ways on each replica,

Starting point is 02:46:37 each GPU, we would scatter it only within each node, so that when you do the communication, the communication happens within the node, within the node, so you don't have to suffer the internal node bindwashed battery. Yes. And then for the ingredients, that's the trickiest part because greetings is sensitive to the quantilization noise. And if you do it naively, just cutting the precision by half, then you would

Starting point is 02:47:03 literally crash your training. Why? That's because when you do the reduction, then that's essentially a accumulation of your quantumization errors. Okay. So I guess How much is that it crashes? So by crash, I mean diverged. Okay, yeah, okay. So that's where we propose a novel all-to-all collective design as a replacement of the current reduced scatter.

Starting point is 02:47:30 So this design makes sure that when we do the communication, we do it in lower position, but when we do the computation, for example, the reduction, that happens in the full position. So that's to avoid the heterodarsific divergence if we do the gradient quantization amoebly. This is essentially a combination of the previous two, where we make sure that our quantization happens by slides

Starting point is 02:47:55 and we make sure that we place the data so that we minimize the internal communication. So that's an idea of the three optimizations we have, and that's where we have the theoretical communication reduction by 4x. But in order to translate this 4x theoretical improvement, into the actual throughput again. That's a huge effort because technically all these operations takes time, right?

Starting point is 02:48:23 So your quantization, you de quantization, your like hierarchical communication. You save, but then there's overhead. Exactly. You saved the communication, but it brings in overhead, a lot of overhead. In order to tackle those overhead, we come up with our own set of customized product kernels, highly optimized, doing, for example, doing kernel fusion, vectorized memory exile, and so on. So we make sure it's maximized in terms of efficiency.

Starting point is 02:48:52 And then overlapping. Overlapping between the communication and computation. So in 03, everything is sharded, right? So that means when you operate, you operate on different charts. So that's give us the chance to do overlapping. So basically, the first shards can go into the communication and then the second chart can do the quantization at the same time in parallel So that's kind of a way to hide our overhead

Starting point is 02:49:22 That's where we finally see the speed up in terms of the throughput So even if on infinite band one infinity band 100 gigabyte per second You are able to see more than 2x speed up and if you are on internet That's totally different story. That's three to four x speed up so Yeah, that's basically the overall works throughout our paper here. Yeah, it's very high-level summarization and you all see a lot more details in our paper. But I mean that's enough for people to haven't heard about it. They will sample it and no, we're not talking about any accuracy trade-off, right?

Starting point is 02:50:01 Like this is pure... Oh, it is lossing. Okay. So you all see some trade-off. So you can see that the loss curve would closely match, but it's not lossless. Okay. Yeah, so you will still see a tiny gap, especially at end of training. Yeah.

Starting point is 02:50:16 Yeah, but considering all the throughput gains and such a tiny gap, so I would say it's worth it. And is there a backstory between zero and zero plus plus? Like, why did you decide to work on this issue? Yeah, so for example, here, that's kind of our motivation. This is measured on the cutting edge cluster. We have eight infinite band connections. We have RDMAs.

Starting point is 02:50:40 We have everything on the heart. hardware level we can get from media. But even in that case, we still see that as we are scaling up the training, the communication would actually take half of the training time. I see, so this percentage is communication over overall, yeah. Yeah, exactly. So that's motivated us, okay, we have to figure out a way to reduce this overhead. Yeah.

Starting point is 02:51:05 And so what is it now? So currently, we don't have this here, but if you you take a look at this kind of speed up, then you can say that that's 2 to 3x. 2 to 3x already. So OK. Yeah. That's all the super large skill.

Starting point is 02:51:22 The plus plus is already released as part of deep speed. So essentially, you can just enable its kind of flag. That's all you get. So so free to check it out. What other just generally about deep speed? Actually, maybe can you introduce these speed at Microsoft and what should people know about your guys' work? Because I think people hear about it,

Starting point is 02:51:43 but only in the context of this stuff, the MOE model that you guys released. What is the overall goal? Yeah, so deep speed is average to optimize super large-scale training for larger models. We first proposed the zero training paradigm. So that's also known as the optimizer state parallelism.

Starting point is 02:52:04 Did you do it before you joined Microsoft or? No, no, it's not. It's more like a computer set. So this deep-speed project starts, at early, I would say, coming to good shape by early 2020s. So that's quite a long time ago. Yeah, and we carry along all the way here to make sure that we always deliver the best training performance for the large scale models and for the large, by large, I mean large in terms of number

Starting point is 02:52:30 of GPUs, number of nodes and so on. And we're also expanding our scope into inference as well. So yeah, we all see a set of inference of limitations and inference stats. We call it fast gen or MI2. Yeah. Okay. So you're working on training improvements, inference improvements, and then the MOE work? No, that's my other teammates.

Starting point is 02:52:51 Okay. Okay, okay, awesome. Anything else people should know about zero plus class? Like what's next? Oh, what's next is we are working on even more aggressive or overlapping between the communication and competition. Okay. Yeah, so our goal is that we want to do a synchronized overlapping.

Starting point is 02:53:08 So instead of the common way of unsynchronized overlapping, but we make sure that the training always synchronized, but we still overlap the communication and computation. Yeah. So please stay tuned, that will be released in the next month. Nice. Then last question. Have you looked at the stuff that Jeremy Howard is doing

Starting point is 02:53:29 with FSDP and QLora? Yes, FSP. Because it's kind of like the small parallel of what you guys are doing. Yeah. FSP is another invitation of the zero-street. And currently we are trying to learn kind of further polish our framework so that we make sure we squeeze the last bit of the performance in terms of the zero policy.

Starting point is 02:53:50 That's another ongoing projects and you will see our efforts gradually in the next automas, all three. I mean, what would be helpful is how do you compare their strengths and then how do you compare like what should people have in their heads comparing like their approach versus yours? So there are two different implementations. So give sentences. So I think people would need to try out themselves. Okay.

Starting point is 02:54:14 So basically no comment. Perfect. Thank you very much. Yeah, thank you. Thank you. A very politically correct handling there of the FSDP question from Guanhua. That is the end of Section C of our audio tour of ICLR. We covered various extremes of relevant work going on with transformers. From long context extension techniques by using shing.

Starting point is 02:54:37 shifted sparse attention and rope interpolation, to adaptive KV catching for single GPU memory consumption, to communication volume for large GPU cluster bandwidth utilisation. We're surprised that all these options fell out so naturally from just walking the floor. We've just crossed our three of this podcast, and it's already tracking to be our longest episode ever. So we unfortunately have to cut all the other poster sessions that didn't make it into one of these top we identified. Please see our show notes if you're interested in papers on summarization, LLM, Uncertainty, Tabular Data and Watermarking. There are a whole lot of watermarking papers, both on how to watermark and on why there is no point

Starting point is 02:55:24 watermarking. We're still not sure why the topic seemed so popular this year. Our last section today covers the Post-Transformers future, as is obligatory at the end of every research podcast and workshop day portion of every research conference. Apart from Lillian Weng's talk on how OpenAI thinks about safe AI deployment, Sasha Rush's talk on state space models got the most buzz among attendees we talk to, so here it is. I'm kind of searching for what my research area is like many people these days, and I think mathematical and empirical study of foundation models is pretty close.

Starting point is 02:56:01 I've done some work in scaling laws and instruction tuning, and a lot of the things that were mentioned earlier. But today I'm going to talk about architectures. A lot of people ask me why I work on architectures. It's not because I think that data isn't important or isn't kind of the main thing that makes foundation models work. But it's a hard area to study, and I'm kind of grasping for what the connection to data is

Starting point is 02:56:23 and how to study it in a rigorous way. Whereas I think architectures gives us at least something we can kind of grasp onto an understanding of how it works and how it connects to some of the problems that we work on. but I don't want to kind of convey that I'm a kind of architecture-only sort of person, and I'm really excited to see what people talk about today. So in this talk I'm going to talk a bit about some of the last couple months in states-based models.

Starting point is 02:56:47 I think you can also think of this area as kind of RNNs. I've been calling it linear RNNs, although we're kind of increasingly seeing non-linear RNNs come back as well. I think I want to talk today a little bit about what the kind of core principles are and how to think about this as a model designer. I know we have some people in the room who are kind of building these and kind of coming up with new architectures for SSMs. Today's talk is going to focus a little bit more on how to use them and how to apply them to practice. So the work today is going to touch on some projects with various students who I collaborate with or my group.

Starting point is 02:57:23 I'm going to talk about a couple different papers that apply SSMs in practice. But before we get there, I think it makes sense to first get us all on the same footing. I've been working a lot on developing tutorials and kind of shared notation to try to demystify what I think is quite a complicated area. I'm not sure I've succeeded in this. People keep on changing the models, which makes it a little hard. But I'm going to talk first about kind of what I think are the core principles that you can use as a model designer when working with states-based models.

Starting point is 02:57:54 This is a short, abstract from a longer tutorial. All three of these talks are online in longer forms. So you can find them on YouTube if you want more. details. This also works a little bit better when it's interactive. I know that's hard in a workshop, but please ask questions at the end about questions you may have. So throughout this talk, we're basically going to only focus on kind of one setting. We're going to be working in a setting where we have an extremely long input context, and we'd also like to perform a very long form generation. I think this is important because a lot of the things we're going to talk about

Starting point is 02:58:27 are kind of only true in a kind of big O sense. So we're really talking about kind of asymptomatic. properties of these models. I used to think this was a little bit of a stretch, but now I'm increasingly kind of really do think we're going to be in a world where we're going to want basically infinite long context for these models. And I think if we're really kind of taking in context learning seriously, it really does imply that that's coming. Of course, the dominant model in the space is transformers. I don't really have to say much about this model. It's arguably the most important invention of the last 10 years like in the world. So that's kind of cool. But it's worthwhile to think of some notation. So throughout the talk, I'm going to use L to represent the length

Starting point is 02:59:07 of the sequence that we're working with. And I'm going to kind of write a schematic transformer as just consisting of attention and MLP layers throughout. Of course, there are two kind of well-known asymptotic properties of transformers. The first is that the memory at inference time scales linearly in the length. So this is kind of the infamous KV cache that you have to store this much memory in order to do generation. I focus particularly on memory. It's tied into all sorts of things, such as energy and efficiency. But I think if we just think about the fact that we have to, at minimal, store the keys and the values for every position, it gives you a good sense of why they're hard. The other property is that they scale poorly at training. There are lots of ways to make this

Starting point is 02:59:55 more efficient in practice or to come up with clever ways to paralyze it. But at the end of the day, you really have to do L-square to multiplications. If you want to compute Transformers exactly, you just have this property, and you have to deal with it some way during training. The papers I'm going to talk about today I put under the class of kind of Mamba-like models. I'm not sure of exactly the right way to call these things,

Starting point is 03:00:22 but I think Mamba has kind of captured the mindshare of how this works and was the first one that really got people to very much pay attention in this space. These are kind of models that use algorithms that are kind of aware of the hardware to try to make it possible to scale other sorts of architectures that get rid of the two main transformers issues in practice. There are a ton of these and more coming out every day. Lots of really interesting ideas in this space and it's worth thinking about the differences between the models. But given as I only have a short talk, I'm going to kind of conflate them all together and try to talk more about the similarities. So here are the kind of key properties that we're

Starting point is 03:01:01 interested in. The first is that we want to have fixed-sized memory in our hidden state. So in particular, I'm going to use kind of an R&N style notation where X is our input. Each one of these blocks represents a new token coming into the model. We're going to produce some block Y as the output from our model, and that Y is going to be fully determined by some sort of recurrent hidden state. Now, one of the big differences that you may not be used to if you haven't looked into these systems is that this hidden state is going to be very big. Now it's big but big and constant. So it's this large block of memory that you keep over time, but it most importantly is big in depth, not in length. So that means it doesn't expand as the sequence gets longer, but simply keeps around a fixed size memory.

Starting point is 03:01:56 The other key property is that these models are going to be sub-quadratic in terms of their training. So we're not going to have to do quadratic numbers of multiplications in order to compute our output Y. Again, this doesn't mean it's small or that it's inherently fast, just that it's not scaling with the length of our sequence. If we have this property, we think we can kind of better utilize these models in hardware and get around some of the long scaling issues. And of course, these models have gained a lot of attention, not just because they're interesting mathematically, but because they're beginning to show promise on real language modeling tasks. This is a really hard thing to show. We've been relying particularly on perplexity and particular perplexity in terms of the computation of these models.

Starting point is 03:02:50 So the kind of key main result was showing that these sorts of architectures could produce very good loss at roughly the same amount of of compute in terms of flops as transformer models. And not just any transformer models, but kind of the state of the art transformers with all the bells and whistles that people have come up with kind of trying out all different permutations over the last five or six years. So I think this result in Mamba

Starting point is 03:03:18 and subsequent results in other papers were really kind of the thing that woke people up to that this architecture may really work in practice, both in terms of getting good loss, but also doing it in a hard hardware-efficient manner. Okay. So in the tutorial, we go through four different things that you should know about these

Starting point is 03:03:37 architectures. For the sake of time, I'm going to focus just on two of them today. I think this will give you a sense just of how they work in enough ways to think about how you can apply them to your own problems. So the two I'm going to focus on are kind of what the models actually are in terms of their kind of mathematical form. And then I'll talk a little bit about how we compute these models in practice. this. So let's first talk about understanding the models. So the key thing that many of these

Starting point is 03:04:08 architectures share is that they have this matrix valued hidden state. We're going to be keeping around a kind of matrix H that represents all the hidden state that we're kind of keeping track of as we go. So we have three dimensions here, the neural network dimension, the length of the sequence and this extra extension dimension that represents an extra amount of information we're keeping in the hidden state as we go. Now, obviously, you've seen models of this form. This model looks very much like the classic kind of vanilla recurrent neural network. Either you're young enough that you came to the field when this model kind of already was kind of out of favor, or you're old enough to have a sense of exactly how it works and have seen it a million times.

Starting point is 03:04:55 Either way, you probably come in with a bias that in some sense this model is bad. But I think we're kind of trying to understand a little bit more why that happened and why it kind of went out of favor in practice. But let's look at the model specifically. So we have kind of three core model parameters, which I'm going to call A bar, B, bar, and C. These represent kind of the way we transform the data to try to produce the output. Now, these have different names in different papers, but you can think of the B as somewhat moving from X to the hidden state.

Starting point is 03:05:29 You can think of A bar as being the dynamics of the hidden state over time. And you can think of C as transforming from the hidden state out again to Y. Now, oftentimes, RNNs are written without a matrix valued hidden state, but you can kind of obviously change them in different forms. And you basically get the same math back. Also note that we're going to use the vanilla or Elman RNN

Starting point is 03:05:53 here, where we have a non-linearity. around the hidden state at each time step. People started to really look back at RNNs again after work in 2021 on a model known as S4. S4 is what's known as a linear time invariant version of this model. The main distinction that you can kind of think about when kind of thinking about these approaches abstractly is that we're simply going to take away the non-linearity of the RNN.

Starting point is 03:06:23 So this gives us a system that is entirely linear, but of roughly the same form. We're going to use B bar to transform our input into our hidden space. We're going to use A bar to produce the dynamics of our model, and we're going to use C to transform from H out again to Y. For a while, this sort of linear time-invariant model was thought to be very hard to learn, and in fact, if you start with kind of random parameters and just kind of throw this kind of into Pi-torch, it's going to perform very poorly. So people had to develop methods in order to get the linear time invariant model to be stable in order to train it on real problems. Now, the results in S4 were very promising, and it showed that models like LTI models could perform very well on long-term problems.

Starting point is 03:07:14 But when people tried to apply these models to language modeling, it didn't scale as well. So I'm going to be a little sloppy here and roughly kind of categorize a bunch of these models as LTI-LLLLLLLLL. and compare them to attention or transformer-based models. This is a graph from Aurora 2024. But basically what it's showing is that when people tried to scale LTI-based models to language modeling, it performed relatively poorly. This was a bit surprising, as these models worked well

Starting point is 03:07:45 on lots of other long-range tasks. And the kind of perplexity numbers here represent roughly a compute, normalized version of each of these models, and the difference is pretty, stark between these LTI models and transformers. In the Mamba paper, they look at two cases where LTI models fail. These cases are relatively intuitive, and I actually found them very useful for thinking

Starting point is 03:08:11 about why language modeling is hard. The first is a problem of filtering. The way I'm thinking about this is if you have a lot of data from the web, it's very noisy. A lot of times you'll have things like ads or kind of random copy thrown in the middle of websites. A good language model like a transformer should just learn to ignore this. It can just jump over it using kind of signals like parentheses or other kind of other offsets in the training data. But interestingly, LTI models just cannot ignore or filter data. If you look at the math, you can see that B bar basically has to act on every input in exactly the same way. That means that

Starting point is 03:08:54 every token that comes into the model has to change the hidden state no matter what. The second problem is the case of resetting the model. So imagine you have a website that has a bunch of articles and each article is separated by a new title. You'd like your model to be able to reset the hidden state so it could move on to remembering things about the next article in line. Of course we have exactly the same problem in a linear time invariant model. If you have A bar and you want to kind of start over, you can't, right? You have to have the same dynamics at every position. And this idea is both essential to the linear time invariant setup and also for the efficient algorithms that are used to calculate it in practice. So we'd like to get around

Starting point is 03:09:43 these two problems in order to run this model in practice. Okay, so when you look at the solution, it actually seems pretty obvious. Of course, that doesn't mean it's actually easy to implement in practice. So we're going to kind of go back to a model that takes into account some sort of variance based on the data itself. So I'm going to represent this in the talk by simply adding a subscript K to each of the parameters A bar, B bar, and C. And this represents that we're going to be kind of changing the parameters at each position. This will allow us to do things like filtering out inputs or resetting our hidden state. You can see this in these two examples. extreme cases by the fact that if we set AK to 0 or BK to zero, we can ignore inputs or reset our

Starting point is 03:10:32 hidden state. In practice, we're going to implement this in a similar way as you'd implement the Q, K, and B for transformers. We'll simply look at our X values and use them to predict the AK, BK, and CK at each time step. This can be done by running a neural network transformation on our input and using that to parameterize each of our AK, BK, and K. If you want to remove something, you just set it to zero as you go. Cool. So this sort of model, which looks a bit more promising and kind of recalls some of the extensions to RNNs that were popular at the time, fixes many of the central issues with

Starting point is 03:11:16 linear time invariant models. It allows us to maintain a fixed-size state while still getting properties like resetting and filtering. Now, that all being said, there are still many other issues with actually implementing this in practice. I'm not going to go through all of them today, but I want to give you a taste of one of the challenges, which is how you run this model efficiently at training. So I want to give you a sense of how you can compute a linear time varying system efficiently on parallel architectures. And to do this, we're just going to talk through a kind of classical algorithm in parallel computing.

Starting point is 03:11:54 and one that I think kind of everyone should just know in the same way that people know matrix multiplication. So we're going to talk through how you compute an associative scan, and in particular how you use associative scans for computing these linear time varying systems. This treatment is from this very nice report from Guy Blaylock. I think it's from 1990. That's worth going back in reading and kind of basically all the details are great. It's a really cool paper. Okay.

Starting point is 03:12:23 Okay, so to get started, we're going to do a kind of hello world problem. We want to compute a cumulative sum of a list of numbers, and we're going to assume that the list is sum power of two in length. So we have 3170, 4163, and we'd like to compute a running sum. We're going to call the running sum YK, and it's the sum of XI from 1 to K. This is the output of this scan. and I'm just going to note that an alternative way to think about a cumulative sum is as a system like this where we're commuting a hidden state HK at each time step,

Starting point is 03:13:05 and we're doing that by adding XK to HK and then just outputting HK at each time step. This kind of converts us to the same notation that we've seen so far when working with SSMs. To do this, we're going to run a tree-based parallel prefix scan. We do this by putting each of these numbers at the leaves of our tree, and then computing a binary tree where at each node, we sum the kind of children of that node. We run this at each step until we get the sum of every position in our list. Now, this gets us a lot of the information we need.

Starting point is 03:13:46 We have the sum of the first two values, the first four values, and of all eight values. But we don't have the intermediate terms that we don't have the intermediate terms that we need for the cumulative sum. So in order to get those positions, we have to run back down the tree, keeping track the information we need, and filling in the additional values. So we can run a down sweep over that tree,

Starting point is 03:14:10 where we compute at the leaves of that down tree each of the values that we need for the cumulative sum. This calculation can be done fully in line and very efficiently. You can actually do it, or a lot of it, in the kind of registers of a GPU. But this is how you commute a cumulative sum, whereas we need to compute the kind of full linear time varying system

Starting point is 03:14:34 that has these additional terms, AK, BK, and CK that vary by the position of the sequence. However, we can do this by slightly modifying the main operator that we use for the cumulative sum. So we're going to basically replace a sum of two values, with a new associative operator, which I'll call O-Sum down here. Instead of acting on two values,

Starting point is 03:15:02 O-sum is going to act on two tuples. These tuples will keep track of two pieces of information and will take the following form. So if we say A-1, B-1, O-sum, A-2, B-2, we get the following output. A-2-A-1, comma, A2B-B-1 plus B-2. I'll leave it as an exercise to go back and convince yourself that this operator is associative, but it is an operator that we can run efficiently on a GPU in practice.

Starting point is 03:15:34 Using this operator, we get a tree that takes the following form, and if we run this for all our data, building up the tree, each of these intermediate forms represent the hidden states that we need in order to compute the SSM. So you can see that in this tree here, B1 X1 is exactly the first hidden state. A2B1X1 plus B2X2 is the second hidden state. And then at the top of this tree, we get the computation for the final hidden state that we need. When we do the down sweep of this tree, we get all the additional values, including H0 and H3, that we need in order to compute the full computation.

Starting point is 03:16:22 Now, this is a very kind of idealized version of this algorithm. It's a bit more complex to run this on GPU in practice, but these algorithms are now built into libraries like Triton, and you can run basically an associative scan over basically any function in about one line of code. So we now can kind of try and play out with these algorithms in practice to make them work. Okay, so first of all, this is like extremely clever and cool.

Starting point is 03:16:47 I really like it. One thing I'm trying to understand is for the left-hand term, you've kind of gone from having these vectors you're throwing around to having matrices. And intuitively, that seems really rough, especially if you're in high dimension. So is there some way that you have of handling that? Yeah, absolutely. Almost all of these papers, you diagonalized A matrices. So in practice, the A is going to basically be scalar values, which is kind of necessary to run it efficiently. That being said, I write it in the more general form just because there's a lot of talk in the community now of kind of moving from diagonal forms to things like block diagonal or even totally full matrices. So I think there's very interesting questions of how to compute the matrix versions more efficiently in practice. I think I'm going to move now to them talking about the kind of tutorial of how they work to two different applications of using them in practice. So takeaways. We have efficient training of these models with long contexts.

Starting point is 03:17:46 and we have fast sampling. I haven't talked about the sampling, but just intuitively, we're going to do R&N-style sampling. We don't have to maintain the whole history. We just have to maintain the hidden state in practice. We're interested in these things because there's promising perplexity results with various different architectures.

Starting point is 03:18:07 There's some questions about how they perform on things like in-context learning, but the perplexity results are pretty important, at least, to get us started. The last thing I'll note is that these are not a magic bullet. The models are still large. You still have to have a lot of parameters. And it still requires the same amount of data.

Starting point is 03:18:27 So I'm not telling you I'm like breaking scaling laws here. I'm just saying we can handle long context in a different way. So in the last 20 minutes, I want to do kind of two quick vignettes about how you can utilize these models to produce different foundation model structure. So first I'm going to talk about using these approaches to do tokenization-free language modeling, and then I'm going to talk about how you can utilize these architectures to do diffusion. So let's first talk about removing tokenization. So there's been a kind of long-running thread in NLP where people have been interested in removing the tokenization part of language modeling. There's lots of reasons why this might be cool.

Starting point is 03:19:13 One is that you can show that if you're working without tokenization, your models are more robust to character level noise. You could also imagine using a model that works the same way for different modalities. And you can avoid tokenization effects that make certain constructs in certain language work more poorly with language models. Now, that all being said, I think I'm kind of mostly interested in this problem just in kind of a deep learning sense. I think there's a kind of long-standing principle that we should be working on the lowest granularity possible. And so just the problem of trying to build models that work on four times longer sequences is, I think, an interesting problem in itself. You can obviously apply this problem by using transformers. If you have a standard transformer, you can just run it on bytes directly.

Starting point is 03:20:04 But when you do that, your sequence length gets four times longer. Since the models are quadratic, this becomes way, way worse. and it makes the models basically kind of impossible to use in practice. People have got around this issue by patching the data. So here we're going to instead work on chunks of bytes where you use kind of fixed-size patches to remove this penalty, but then try to generate bytes at the top of the model. I think this is a clever way of getting around the problem,

Starting point is 03:20:38 but in my experience, patching almost always makes the model worse. And we saw this when we actually applied these models in practice. I think there's kind of like no free lunch from kind of working with larger patches. It generally just kind of pays a penalty that you have to end up training for much longer, or the model works kind of worse in the final output. Models like Mamba or SSMs allow us a potential solution to this problem. If we can run a model that uses a fixed-sized hidden state, then we've somehow kind of disconnected the length of the sequence from its runtime complexity.

Starting point is 03:21:17 So kind of a model that doesn't kind of change its hidden representation based on the input data representation allows us to basically work with a lower granularity without kind of paying that big, big penalty in quadratic search. So to do this, we're going to basically just apply Mamba directly to byte level models. What this means is that we'll have this big hidden state that we update over time. We'll have one of these hidden states per each byte of our input, and it will be used to generate each byte of our output. Now, again, I think the kind of first intuition you get from working with RNN models is,

Starting point is 03:21:59 well, it seems like you're losing a lot of information. Transformers are able to memorize everything. Well, here we have to store it all in this hidden state. But even just the default Mamba model has a very, very large hidden state. So we're talking about 3.5 megabytes just represented in these hidden states over time, which is way, way, way more than you need to actually represent all of the bytes in our context. So in practice, for most of our experiments, we're talking about like 10K bytes in our sequence, whereas our hidden state has 3.5 megabytes of storage, right?

Starting point is 03:22:39 So there's not really a problem with storing this kind of information if the model can effectively learn how to utilize it in practice. As I mentioned earlier, to actually train these models, we're going to be using the scan operation that I showed before. And other than that, we're basically just applying a vanilla version of Mamba to byte-level models. The problem that comes up in practice is that we actually need to generate bytes in our kind of output.

Starting point is 03:23:12 So one nice thing about tokenization is that when we generate tokens, we can do them one at a time, and that's relatively fast. For generating bytes, we need to generate kind of four bytes to get one of these tokens. So it seems like we're paying a large cost to actually use these models in practice. However, over the last year, there's been a lot of advances in applying speculative decoding for generation. and I want to argue that bite-level models are a particularly nice form for speculative decoding. So in particular, if we have a model that can draft at a larger granularity, we can kind of give us a sense of where we're going with our bite-level approach.

Starting point is 03:23:55 So what we're going to do is we're going to draft with tokens on the left-hand side, and then we're going to verify with bytes on the right-hand side. The really cool thing about doing verification with byte-level model, is it can be done in parallel. We have the ability to run a parallel scan to check all these bytes on the right-hand side. This allows us to avoid a lot of the problems of reloading in the weights

Starting point is 03:24:21 or generating in a sequential manner for the bite-level approach. The way it works is that if everything matches up into a certain point, we just keep on going until we hit a letter where it doesn't match our speculation. We then continue along with it. continue along with the byte level approach until we hit a token boundary and then continue on with the speculative approach to our right. So by doing this, we're basically able to speed up the mamba byte model to roughly the same speed as a token-based model. We're basically using a kind of standard tokenized mamba as the draft model and the mamba byte model for verification. We get roughly the same

Starting point is 03:25:08 generations at nearly the same speed as we would have with the tokenized version. Cool. So the paper has lots of experiments. I'm going to go through kind of the high level just to get you a sense of what we're thinking about. The kind of main conclusion from this work is that Mamba is like an extremely good fit for byte level modeling. Basically out of the box, it beats transformer models across the board.

Starting point is 03:25:35 And it beats a lot of patched version of... transformers models as well. We find this both in a kind of parameter match setting and also a flop match setting and with using significantly less data. So no real innovation here. This was just an observation that applying SSM models to these tasks works really well and has a nice structure for managing the hidden state. Another thing that's interesting is we see certain properties out of the box that others have observed in their work as well. We see basically length generalization for free. We train on about 8k bytes, but the model is able to generalize over 30k bytes with only very small losses in perplexity. This is one nice property of RNNs because they're

Starting point is 03:26:22 inherently sequential models. They don't have the same kind of properties of transformers where they are unable to generalize to longer sequences without clever tricks. Cool. Let me skip ahead. I think another interesting result is that with Mamba, we see similar results on perplexity, whether we use bytes or tokens. So training at a byte level model, you can compare directly the perplexity to a token level model. So in kind of fair comparisons in a parameter match setting, the two models basically converge to roughly the same perplexity. This kind of applies that byte level models with the same hidden size state can perform as well as token-based models. It's much harder to do that comparison in a fair way with transformers.

Starting point is 03:27:13 Now, this doesn't mean we should train byte-level large language models yet. There are a lot of kind of issues about doing that at scale, but it does show that maybe there's nothing inherently wrong with training on bytes. Cool. Okay, so I have about five minutes left. Let's see. I'm going to just do a quick summary of one last work. I think the story is relatively similar, but I think it's worth kind of knowing. This is a work at CVPR this year called diffusum, where we apply a very similar idea to diffusion models. We're going to basically be focusing on the architectural part of a diffusion system. We want to build an architecture

Starting point is 03:27:58 that can map from XT plus 1 to XT, basically do one step of denoising. I'm going to note that all kind of state-of-the-art diffusion models have some self-attention baked into the core of them, and this self-attention is critical to performance. So this is true about UNET-based models that basically downsample to a very small size and then ply self-attention. and it's also true in transformer-based models that first patchify their inputs and then run self-attention in a patchified form of the data. I think this patchifying should just go away. I think it is bad, and I think people know it's bad, but we've kind of accepted that it's kind of something you need in order to apply self-attention.

Starting point is 03:28:52 In a world where, and you can see this in all these papers, they all show these kind of results. that show that the more patchifying you do, the worse your images gets. And a lot of the work is to kind of reduce the patchifying to a relatively small size. If we don't have attention, we basically don't need to pacify. We can work in as large or as small a granularity as we can fit into our... So we're basically going to do that. We're going to start with an image. We're going to flatten it.

Starting point is 03:29:24 And then we're going to apply a states-based model instead of... of self-attention. The main benefit of the state-based model here is that we don't have to compress down to a smaller size. We can run on a large granularity of the image, and we can produce our output. So we're not going to even argue in this paper that it's kind of better or worse than self-attention,

Starting point is 03:29:47 just that the main issue here is that you're compressing the representation to start with. What we find is that for these models, We get a kind of better scaling properties with longer images. We give longer images because they're less patched. And if we were trying to run attention on these models, it would scale very poorly. Let's see. Here are some results.

Starting point is 03:30:12 We're able to, I guess, generate good-looking images using these models. And we get basically state of our performance on ImageNet conditional generation by using states-based models instead of self-attention. So I think our conclusion here is that global attention is kind of critical for these models, but we think that the ability to replace that with other approaches like states-based models will lead to new and interesting designs. I think this is interesting for images, but is particularly interested for even harder diffusion problems.

Starting point is 03:30:47 So if we talk about video modeling or 3D modeling, you have much longer sequences and see much stronger penalties from doing compression on these models themselves. Okay, so that was a somewhat whirlwind tour. I think I'll stop there if anyone has any questions. Thank you so much. That was an incredible talk, including the two applications on removing tokenization and on image generation replacing global attention with state space models. If you consider that we started off this episode talking about how learning is compression

Starting point is 03:31:25 and ended by discussing how compression harms generation quality, You get a full sense of how ICLR really is the ultimate place to learn, think and talk about the state of the art in learning representations. Because we don't like being pure hype people, we'll end the pod today with one final ICLR outstanding paper award that throws a bit of cold water on state space model outperformance. If we just introduce data-driven priors for Transformers. Hi, I'm Edo and I'll be presenting our work on fair comparison of launch. sequence models. This work was done in collaboration with Jonathan Barrett and Ankit Gupta. Our work is concerned with how to evaluate inductive bias. In the typical setting of evaluating an inductive bias, a model is trained from a random initialization on suit of dedicated tasks.

Starting point is 03:32:16 On the other hand, when tackle in real-world problems, the modern standard practice is to first pre-train a model on a large pre-training data set prior to fine-tuning on the desired task. This pre-training phases revolutionized most areas of machine learning, leading to outstanding results. Yet, when developing new architectures with better inductive bias for a particular task, it is still common to train on the task data from scratch, starting from a random initialization. A prime example of estimating inductive bias is modeling long-range dependencies across sequences. To do so, standard benchmarks such as long-range arena, or LRA, provide multiple tasks that require learning dependencies between distant elements. A significant result of the original LRA paper

Starting point is 03:33:02 showed that Transformers and efficient variants failed to report adequate performance on the tasks. This inefficacy of Transformers has led to a line of new architectures, suggesting changes to RNNs, CNNs, and Transformers themselves, biasing them towards capturing long-range dependencies and achieving impressive performance when trained from scratch. However, these results do not aligned with performance of pre-train transformers that had displayed remarkable capabilities and tasks involving modeling long-range dependencies, such as text summarization, code completion, and protein folding. In this work, we shed light on this discrepancy, showing its stems from inadequate training and evaluation practices, and suggest a simple and efficient solution allowing a proper evaluation.

Starting point is 03:33:50 While avoiding pre-training on a large dataset is understandable, training from a random initiative with downstream supervision alone, this regards the role of the pre-training objective itself, leading to a different inductive bias than that of a pre-trained model. Our proposed solution is simple. Rather than training from scratch, one can achieve a better evaluation by first pre-training on the downstream data itself, or self-pre-training, as coined by Krishnaeatal. This idea is motivated by the work of El Nubi-Ital and Krishna-etal, demonstrating that, when using the noising objectives, pre-training solely on downstream training data often leads to similar performance as pre-train-on-large data sets. Therefore, a more realistic estimate can be obtained via self-pre-training, or SPT, for short,

Starting point is 03:34:40 with SPT acting as a data-driven initialization method. Doing so allows a fair comparison between methods, as only the task data is used, and better reflects the performance of pre-trained models by including a self-pre-training stage. To demonstrate the importance of SPT, we empirically showed that privates learned through SPT with the noising objectives are highly effective for learning long-range dependencies, implying that the correct evaluation must account for the pre-training stage. While the performance of trained-from-scratch transformers on LLA is subpar compared to S-4, SPT improves their average performance by more than 30%, matching the state-of-the-art without any architectural changes. We find SPT to also benefit S4, with performance gains in 5 out of 6 tasks.

Starting point is 03:35:29 Moreover, with SPT, S4 solves the challenging Pethex 256 task, with a 20% improvement compared to training from scratch. To the best of our knowledge, we are the first to show purely attention-based methods can achieve comparative results on LRA. Observing the drastic effects of SPT, we ask whether simpler models can achieve similar performance as complex once. We study the S-4 model and a natural simplification of it. The S-4 model can be viewed as a repramatization of a linear RNN. Motivated by an accompanied theory, an S-4 model re-parameterizes the R&N parameters instead of learning them directly and initializes them with theoretically derived values. Inspired by the success of S-4, Guptaetal, proposed a simplification called the diagonal linear R&N, or DLR, replacing the repramatization steps with a single diagonal matrix.

Starting point is 03:36:25 We regard the differences between the two models as hand-crafted biases, as these are the only differences between the models. Any advantages in modeling long-range dependencies is associated with them. When both models are trained from scratch, DLR does not match the performance of S-4, pointing to the effectiveness of the handcrafted biases, yet adding an SPT phase diminishes the gap to roughly 1%. Showing biases for pre-training can lead to a simpler model with similar performance. For more details about self-pre-training and evaluation,

Starting point is 03:36:59 check out our paper, and thank you for listening. And that is a wrap for part one of our coverage of ICLR 2024. All the best papers and talks covering image generation with a sidetrack on compression and adversarial attacks, vision with a sidetrack on self-supervised and weekly supervised learning. Extending transformers and attention across long context, single GPU and large GPU clusters, and the state of state space models featuring the voices of Christian Zegady, Ilya Sotskiva, Sasha Rush and many, many other luminaries.

Starting point is 03:37:35 Many thanks to those who took the time to speak to us whether on or off mic. Part two of ICLR coverage is solely on reasoning and agents. Subjectively the biggest topic discussed at ICLR for AI engineering. Stay tuned.

Latent Space: The AI Engineer Podcast - ICLR 2024 — Best Papers & Talks (ImageGen, Vision, Transformers, State Space Models) ft. Durk Kingma, Christian Szegedy, Ilya Sutskever

There aren't comments yet for this episode. Click on any sentence in the transcript to leave a comment.