Elon Musk's approach to problem-solving | Lex Fridman Podcast

Elon Musk's approach to problem-solving | Lex Fridman Podcast - 이중 자막

Can you just speak to what it takes for a great engineering team for you, what I've saw in Memphis, the Super Computer Club?

cluster is just this intense drive towards simplifying the process, understanding the process, constantly improving it, constantly iterating it.

Well, it's easy to say simplify and it's very difficult to do it.

You don't have this very basic first principles algorithm that I run kind of as like a mantra,

which is the first question, the requirements, make the requirements less dumb, the requirements always dumb to some degree.

So if you want to solve by reducing the number of requirements and know about how smart the person who gave you those requirements,

this will dumb to some degree.

If you, you have to start there because otherwise you could get the perfect answer to the wrong question.

So, try to make the question the least wrong possible.

That's what question and requirements means.

And then the second thing is try to delete the, whatever the step is, the part or the process step.

Sounds obvious.

people often forget to try to leading it entirely,

and if you're not forced to put back at least 10%

of what you delete,

you're not deleting enough,

and it's somewhat illogically,

people often,

most of the It feels though they have succeeded if they have not been forced to put things back in,

but actually they haven't because they've been overly conservative and have left things in there that shouldn't be.

So only the third thing is try to optimize it or simplify it.

Again, these all sound, I think, very obvious when I say them, but the number of times I've

made these mistakes is more than I care to remember.

That's why I have this mantra.

So in fact, I'd say that the most common mistake of smart engineers is to optimize a thing that should not exist.

So, like you said, you run through the algorithm and basically show up to a problem, show up

to the supercomputer cluster and see the process and ask, can be deleted?

Yeah, first try to delete it.

Yeah.

That's not easy to do.

Yeah.

No, and actually

there's What generally makes people uneasy is that you've got at least some of the that you'd lead you will put back in.

But back to sort of where our limbic system can steer us wrong is that we tend to

remember with sometimes a jarring level of pain where we've where we deleted something that we subsequently needed.

And so people will remember that one time they forgot to put in this thing three years ago and that caused them trouble.

And so they overcorrect and then they put too much stuff in there and over-complicate things.

So you actually have to say, we're deliberately going to delete more.

than we should, so that we're putting at least one in 10 things we're going to add back in.

And I've seen you suggest just that that something should be deleted and you can kind of see the Oh, yeah, absolutely.

Everybody a little bit of the pain.

Absolutely.

And I tell them in advance, like, yeah, there's some of the things that we delete, we're gonna put back in.

And people get a shook by that.

But it makes sense,

because if you're so conservative as to never have to put anything back in, you obviously have a lot of stuff that isn't needed.

So you got over correct.

This is, I would say, like a cortical override to an Olympic instinct.

One of many that probably leaves this astray.

There's like a step four as well, which is any given thing can be sped up.

Out of a fast, you think it can be done.

Like, whatever the speed is being done, it can be done.

faster.

But shouldn't speed things up until it's off until you try to delete it and optimize it,

although you're speeding up something that shouldn't exist as a absurd.

And then the fifth thing is to automate it.

And gone backward so many times where I've automated something, sped it up, simplified And I got tired of doing that.

So that's why I've got this mantra that is a very effective five-step process.

It works great.

When you've already automated, deleting must be real painful.

Yeah.

See, it's like, wow, I really wasted a lot of effort there.

I mean, what you've done with the cluster in Memphis is incredible, just in a handful of weeks.

Yeah, it's not working yet.

So I want to publish my bank books.

In I have a call in a few hours with the Memphis team because we're having some power fluctuation issues.

So, yeah, it's of a, when you do synchronized training, when you've all these computers that

are training,

that where the training is synchronized to,

you the sort of millisecond level,

you It's like having an orchestra, and the orchestra can go loud to silent very quickly, you at sub-second level.

And then the electrical system kind of freaks out about that, like if you suddenly see giant shifts 10, 20 megawatts several times.

This is not what electrical systems are expecting to see.

So that's one of the many things you have to figure out.

The the power, and then on the software as you go up the stack, the distributed could be all of that.

Yes, today's problem is dealing with extreme power jitter.

Power jitter.

Yeah.

The nice bring to that so that's okay.

I you stayed up late into the night as you often do there last week.

Yeah last week Yeah, finally finally good good good training going at Alina roughly 4 4 20 a.m.

Last total coincidence.

Yeah, I mean, it a 4-22 or something.

Yeah, yeah, yeah.

It's that universe again with the jokes that they just love it.

I mean,

I wonder if you could speak to the fact that you,

one of the things that you did when I was there is you went through all the steps of what everybody's doing,

just to the sense that you yourself understand it.

and everybody understands it so they can understand when something is dumb or something is inefficient or that kind of stuff.

Can speak to that?

Yeah, so I try to do whatever the people at the front lines are doing.

I to do it at least a few times myself.

So connecting fiber after cables,

diagnosing a poly connection, That tends to be the limiting factor for large training clusters is the cabling, so many cables.

For a coherent training system where you've got RDMA or remote direct memory access,

the whole thing is like So,

if you've got any to any connection, so it's the any GPU can talk to any GPU out of 100,000.

That was a crazy cable layout.

it looks pretty cool yeah it's like it's like a human brain but like at a scale that humans can

visibly see it is yeah brain yeah I mean the human brain also has a massive amount of the brain

tissue is the cables yeah so they get the gray matter which is the computer and then the white matter which is cables.

but big percentage of brain is just cables.

That's what it felt like walking around in the supercomputer center is like, we're walking around inside the brain.

One build a super intelligent.

Super, super intelligent system.

번역 언어

번역 언어 선택

비디오 요약 자막 내보내기 구술 평가

더 많은 기능 잠금 해제

Trancy 확장 프로그램을 설치하면 AI 자막, AI 단어 정의, AI 문법 분석, AI 구술 등을 포함한 더 많은 기능을 사용할 수 있습니다.

Elon Musk's approach to problem-solving | Lex Fridman Podcast - 이중 자막

더 많은 기능 잠금 해제

인기 있는 비디오 플랫폼과 호환

시작 준비되셨나요?