Hello, everybody! Today, we're gonna try to figure out how Word2vec works under the hood. I'll remember the basic idea. Computer can work with numbers, and we'd like him to work with words, and ideally not just words, but also the meaning of those words. What's the meaning of the word? Aah, at the domestic level, we could have answered that: we have a vocabulary that defines each word, yes, where for each word it says what it means. Can you, um, try to make sense without that, find the meaning of the word? Turns out I am. Ah, we're gonna have a little experiment with you. To do this, I will revert to, I hope, the already known site of Elena Voight. I'll leave a reference to him, so I'll come over and see.
But now we're going to go through the main points, um, and I hope you'll get a better understanding of how Word2vec is under the hood. So let's move on to the demonstration. So we need to understand, can we, uh-a-a-a-a-a-a understand the meaning of the word if we don't have a vocabulary? How could that be done? There's a little experiment here. In fact, we're being asked, "Do you know what the word means, "Why don't you just let it be a zguigo?" I don't know how it reads. I hope you don't know that word either. But how could you possibly understand? Let's say we have four proposals. Ah, I'll translate it in English for your comfort. Those who know English can read right off the screen. Ah, we have a bottle of this very zguigño on the table, respectively. Everybody loves that tzeguino. Tezguiguinho makes you drunk and, accordingly, we're making a zeguino out of the corn. That is the only thing that knows, yes, knowing that the word is in these contexts, is in such a way that you can assume that word means it? Mm-hmm, you could put it on the pause and think, but in general, I think that many people are going to come to this conclusion that thesis is some alcoholic drink that comes from the corn. Look, we don't know the definition of this word, we don't know it, but we know what context it's in, right? I mean, if there's a context we know, then we can guess what that word is. Ah, let's see. Yeah, that's the example. There are four such proposals, which, um, can define the word., we can try to frame the various words in these passes instead of the zesguinyo and see if this word is satisfactory or not. There are four options, five options, right? A-a-a-a-a-a-a-a-a-a-a-a-a-a-a-a-a-a-a-a-a-a-a-a-a-a-a-a-a-a-a-a-a-a-a-a-a-a-a-a-a-a-a-a-a-a-a-a-a-a-a-a-a-a-a-a-a-a-a-a-a-a-a-a-a-a-a-a-a-a-a-a-a-a-a-a-a-a-a-a-a-a-a-a-a-a-a-a-a-a-a-a-a-a-a-a-a-a-a-a-a-a-a-a-a-a-a-a-a-a-a-a-a-a-a-a-a-a-a-thin this, this demonstration, in this, this, this, this, this demonstration There are more than one word that either satisfys or does not, or does not satisfy, individual words, individual proposals. For example, wine satisfies the first. You could say a bottle of wine is on the table. Everyone loves wine, and wine makes you drunk, but, accordingly, wine doesn't come out of corn, does it? So three satisfies, the fourth does not. The concept, in fact, not the context, makes sense that it's close enough to the wine and the zeguino, but not entirely concomitant. Mm-hmm, another interesting version of the cake, huh? These are the Mexican food chips. So, here we, uh, have a bottle of cake and a bottle.
Tortilla doesn't make you drunk, but you can say that everyone likes tortilla, and the tortillas are made of corn. And there are some other examples, respectively. You can put every word in your mouth and try to determine, and for which of these four proposals this word will be satisfied. You might find a way that for everyone, maybe just for some. Now look, uh, how can we make an interesting conclusion? When we have some contexts, we can understand what words might be close and what might be far-reaching, right? So, words that are in contexts, yes, that is, contextuals are being followed by one word and another word, they are in the sense of a close, like, the zeguino and wine. For example, motor oil, it is therefore farther, even though in some contexts these words can be used equally.
If we have a large number of these contexts, we will be close to the word, and the far away, respectively, will be far., proximity means what the number of contexts these words will be appropriate. Accordingly, let's see what we're getting from this. We can imagine any text and context of this word that will be defined from the words around it. We'll take a few words before we take a few words after. It's a condition. The broad of the context window you can ask anyone. It could be two words left, two words to the right, maybe ten words left, ten words to the right, and so on. They can be symmetrical, non-symmetrical. This example is not important, but for simplicity, it is here for Elena to demonstrate that we take a central word and context, two words before it, two words after it. We can see, in fact, that each word will be defined by some context or influence some context. So, for example, if we take the word "cock," yes, the context we're gonna look at for him is "mile," "serious," on the one hand, and on the other hand, "plays in." Yeah, I'm just saying, I see a nice, grey cat playing in the garden. Uh, what can we do about it? We can try automatically for a large number of texts to consider contexts. That is, and instead of defining it, we can count and in what contexts the word is used. Aah, what's that gonna look like? I'm gonna need a little vocabulary of all the words I'm allowed to say. Uh, we can take a vocabulary on the condition. Ah, there's a vocabulary in this orthographic vocabulary, and there's a hundred and fifty thousand words. For example, there's a Russian language orthographer, between 100 and 200 thousand words, depending on the publication. Well, let's have a hundred and fifty thousand words. What do I want to do next? For every word of this dictionary, I want to define another vocabulary and count: this is the word, how is it likely that I will meet with the other one? I'll have another vocabulary for every word, and there I'll set out the numbers, the odds that I'll meet the word, you know, the cat. Yeah, let's get some cats. How much is it possible to see the word "bone" with the word "bear" and see the word "bone" next to "bear" next to "play." And that's why I have all the 100, fifty thousand words written here, and for a cat, I'm actually gonna have a certain chance, as often I can see those words.
Yeah, like, with the word "motor" in there, it's motor oil, I don't think I'll be able to see a cat too often, and with the word, like, "mortal," I'm probably gonna be there a lot of times, right? I mean, for every word, I have to make another exactly the same vocabulary of 100 thousand words, and, accordingly, for each one of them, I will write how often I meet with you. But it's actually automatic. Oh, a little math, don't be afraid, yeah, we're gonna be a little bit more rephrasing right now, but the point is, I can count on how often these words will be dating each other. And then when I run the texts, we take the huge texts, "War and Peace," "Anna Karenina," math textbook, literature doesn't matter, yeah, that's all-- all the "Vickipedia" that is, we take all the texts we have. There are, and for each word, we will update this dictionary. If that word was nearby, his number, his probability is a little bit more. If-- and if all the other words are said, I'll be a little shorter. What's going to happen? When I'm all the texts I'm available, I'll run, the words that are next to the word "cock" will be likely. All the other words will be very low, right? And, in fact, this is the distribution of words that are being met by the cat, and I will consider it a definition. So how can we work on this? We come for example, King Minus man plus woman is queen. If you didn't hear that example, you'd actually look at the video, right? What do we get? We have a dictionary for the king, and we're reading a dictionary for a man. We have a vocabulary of 100,50,000 words that we've learned. So we're from the numbers that correspond to a specific word, we're gonna count the numbers, then add, and, uh, the vocabulary that comes to the woman, and so we're getting another vocabulary. And then we start looking, and what's the closest thing he's gonna say? And it turns out he's very close to the word "Korolewa." That's how it's actually video-addition to the previous video. If you didn't see him, look at him. And if you've already looked at him, you can actually play the quiz I'm talking about. And I'm sure that the examples for you will be won a little bit of a new one, yes. And if you're in trouble somewhere,
Yeah, and if you had any questions, why would the answer be that, not the other one? You might think that in the texts where the word2vec model was taught, we had the distribution of words, respectively.
And that's why we did it, uh, that was the version that we had. It is worth a quick saying that, uh, word2vec is, of course, a turning-in velocity in NLP, in natural language processing, yes, in the processing of natural language. But if we're talking about modern LLM, then, uh, there's a little other mechanisms under the hood. Actually, they're conceptually similar, but we'll talk about them in more detail in the next editions. Stay on the canal with us. I hope you were curious and a little more understandable. Beat the liki, write the comments. Almost everything, uh, I've responded. In fact, we're happy to see you in the next releases of our sub-cadast.