Elon Musk's approach to problem-solving | Lex Fridman Podcast - Sous-titres bilingues
Can you just speak to what it takes for a great engineering team for you, what I've saw in Memphis, the Super Computer Club?
cluster is just this intense drive towards simplifying the process, understanding the process, constantly improving it, constantly iterating it.
Well, it's easy to say simplify and it's very difficult to do it.
You don't have this very basic first principles algorithm that I run kind of as like a mantra,
which is the first question, the requirements, make the requirements less dumb, the requirements always dumb to some degree.
So if you want to solve by reducing the number of requirements and know about how smart the person who gave you those requirements,
this will dumb to some degree.
If you, you have to start there because otherwise you could get the perfect answer to the wrong question.
So, try to make the question the least wrong possible.
That's what question and requirements means.
And then the second thing is try to delete the, whatever the step is, the part or the process step.
Sounds obvious.
people often forget to try to leading it entirely,
and if you're not forced to put back at least 10%
of what you delete,
you're not deleting enough,
and it's somewhat illogically,
people often,
most of the It feels though they have succeeded if they have not been forced to put things back in,
but actually they haven't because they've been overly conservative and have left things in there that shouldn't be.
So only the third thing is try to optimize it or simplify it.
Again, these all sound, I think, very obvious when I say them, but the number of times I've
made these mistakes is more than I care to remember.
That's why I have this mantra.
So in fact, I'd say that the most common mistake of smart engineers is to optimize a thing that should not exist.
So, like you said, you run through the algorithm and basically show up to a problem, show up
to the supercomputer cluster and see the process and ask, can be deleted?
Yeah, first try to delete it.
Yeah.
Yeah.
That's not easy to do.
Yeah.
No, and actually
there's What generally makes people uneasy is that you've got at least some of the that you'd lead you will put back in.
But back to sort of where our limbic system can steer us wrong is that we tend to
remember with sometimes a jarring level of pain where we've where we deleted something that we subsequently needed.
And so people will remember that one time they forgot to put in this thing three years ago and that caused them trouble.
And so they overcorrect and then they put too much stuff in there and over-complicate things.
So you actually have to say, we're deliberately going to delete more.
than we should, so that we're putting at least one in 10 things we're going to add back in.
And I've seen you suggest just that that something should be deleted and you can kind of see the Oh, yeah, absolutely.
Everybody a little bit of the pain.
Absolutely.
And I tell them in advance, like, yeah, there's some of the things that we delete, we're gonna put back in.
And people get a shook by that.
But it makes sense,
because if you're so conservative as to never have to put anything back in, you obviously have a lot of stuff that isn't needed.
So you got over correct.
This is, I would say, like a cortical override to an Olympic instinct.
One of many that probably leaves this astray.
There's like a step four as well, which is any given thing can be sped up.
Out of a fast, you think it can be done.
Like, whatever the speed is being done, it can be done.
faster.
But shouldn't speed things up until it's off until you try to delete it and optimize it,
although you're speeding up something that shouldn't exist as a absurd.
And then the fifth thing is to automate it.
And gone backward so many times where I've automated something, sped it up, simplified And I got tired of doing that.
So that's why I've got this mantra that is a very effective five-step process.
It works great.
When you've already automated, deleting must be real painful.
Yeah.
See, it's like, wow, I really wasted a lot of effort there.
I mean, what you've done with the cluster in Memphis is incredible, just in a handful of weeks.
Yeah, it's not working yet.
So I want to publish my bank books.
In I have a call in a few hours with the Memphis team because we're having some power fluctuation issues.
So, yeah, it's of a, when you do synchronized training, when you've all these computers that
are training,
that where the training is synchronized to,
you the sort of millisecond level,
you It's like having an orchestra, and the orchestra can go loud to silent very quickly, you at sub-second level.
And then the electrical system kind of freaks out about that, like if you suddenly see giant shifts 10, 20 megawatts several times.
This is not what electrical systems are expecting to see.
So that's one of the many things you have to figure out.
The the power, and then on the software as you go up the stack, the distributed could be all of that.
Yes, today's problem is dealing with extreme power jitter.
Power jitter.
Yeah.
The nice bring to that so that's okay.
I you stayed up late into the night as you often do there last week.
Yeah last week Yeah, finally finally good good good training going at Alina roughly 4 4 20 a.m.
Last total coincidence.
Yeah, I mean, it a 4-22 or something.
Yeah, yeah, yeah.
It's that universe again with the jokes that they just love it.
I mean,
I wonder if you could speak to the fact that you,
one of the things that you did when I was there is you went through all the steps of what everybody's doing,
just to the sense that you yourself understand it.
and everybody understands it so they can understand when something is dumb or something is inefficient or that kind of stuff.
Can speak to that?
Yeah, so I try to do whatever the people at the front lines are doing.
I to do it at least a few times myself.
So connecting fiber after cables,
diagnosing a poly connection, That tends to be the limiting factor for large training clusters is the cabling, so many cables.
For a coherent training system where you've got RDMA or remote direct memory access,
the whole thing is like So,
if you've got any to any connection, so it's the any GPU can talk to any GPU out of 100,000.
That was a crazy cable layout.
it looks pretty cool yeah it's like it's like a human brain but like at a scale that humans can
visibly see it is yeah brain yeah I mean the human brain also has a massive amount of the brain
tissue is the cables yeah so they get the gray matter which is the computer and then the white matter which is cables.
but big percentage of brain is just cables.
That's what it felt like walking around in the supercomputer center is like, we're walking around inside the brain.
One build a super intelligent.
Super, super intelligent system.
Langue de traduction
Sélectionner
Débloquez plus de fonctionnalités
Installez l'extension Trancy pour débloquer plus de fonctionnalités, y compris les sous-titres IA, les définitions de mots IA, l'analyse grammaticale IA, la parole IA, etc.
Compatible avec les principales plateformes vidéo
Trancy offre non seulement le support des sous-titres bilingues pour des plateformes telles que YouTube, Netflix, Udemy, Disney+, TED, edX, Kehan, Coursera, mais propose également la traduction de mots/phrases IA, la traduction immersive de texte intégral et d'autres fonctionnalités pour les pages web régulières. C'est un véritable assistant d'apprentissage des langues tout-en-un.
Tous les navigateurs de plateforme
Trancy prend en charge tous les navigateurs de plateforme, y compris l'extension du navigateur Safari iOS.
Modes de visualisation multiples
Prend en charge les modes théâtre, lecture, mixte et autres modes de visualisation pour une expérience bilingue complète.
Modes de pratique multiples
Prend en charge la dictée de phrases, l'évaluation orale, le choix multiple, la dictée et d'autres modes de pratique.
Résumé vidéo IA
Utilisez OpenAI pour résumer les vidéos et saisir rapidement le contenu clé.
Sous-titres IA
Générez des sous-titres IA précis et rapides pour YouTube en seulement 3 à 5 minutes.
Définitions de mots IA
Appuyez sur les mots dans les sous-titres pour rechercher des définitions, avec des définitions alimentées par l'IA.
Analyse grammaticale IA
Analysez la grammaire des phrases pour comprendre rapidement le sens des phrases et maîtriser les points de grammaire difficiles.
Plus de fonctionnalités web
En plus des sous-titres vidéo bilingues, Trancy propose également la traduction de mots et la traduction intégrale de texte pour les pages web.