No Priors Ep. 65 | With Scale AI CEO Alexandr Wang - Çift Dilli Altyazılar
Hi listeners, and welcome to No Priers.
Today I'm excited to welcome Alex Wang, who started Scale AI as a 19-year-old college dropout.
Scale has since become a juggernaut in the AI industry.
Modern is powered by three pillars.
compute, data, and algorithms.
While research labs are working on algorithms,
and AI chip companies are working on the compute pillar,
scale is the data foundry, serving almost every major LM effort, including OpenAI, Meta, and Microsoft.
This is a really special episode.
So for me, given Alex's started scale in my house in 2016, and the company has come so far.
Alex, welcome.
I'm so happy to be talking to you today.
Thanks for having me.
No new goal for quite some time.
So excited to be on the pod.
Why don't we start at the beginning just for a broader audience?
Talk little bit about the founding story of scale.
Right before scale, I was studying AI and machine learning at MIT.
And this was the year when DeepMind came out with AlphaGo, where Google released TensorFlow.
So was the of the deep learning hype wave or hype cycle.
And I remember I was at college.
I trying to use neural networks.
I was trying to train image recognition.
neural networks.
And the thing I realized very quickly is that these models were very much so just a product of their data.
And I sort of played this forward and thought through it, and these models or AI in general is the product of three fundamental pillars.
There's the algorithms, the compute and the computation power that goes into them, and the data.
And at that time, there companies working on the algorithms, labs like OpenAI or Google's Labs or a of AI research efforts.
There were, in video, was already a very clear leader in building compute for these AI systems.
But there was nobody focused on data.
And it was really clear that over the long arc of this technology, data was only going to become more and more important.
And so in 2016,
Tropical MIT did YC and really started scale to solve the data pillar of the AI ecosystem and the organization that was going to solve all the hard problems
associated with how do you actually produce and create enough data to fuel this ecosystem and really
this was the start of scale as the data foundry for AI.
It's incredible foresight because you describe it as like the beginning of the deep learning hype cycle.
I don't think most people notice that hype cycle was yet going on.
And so I just distinctly remember,
you know,
you working through a number of early use cases,
you know,
building this company in my house at the time,
and discovering, I think, far before anybody else noticed that the AV companies were spending on data.
How did you think about like talk a little bit about how the business has evolved since then because it's certainly not just that use case today?
AI is an interesting technology because it is at the core mathematical level such a general purpose technology.
It be,
you know,
it's basically functions that can approximate nearly any function including like intelligence and so it can be applied in a very wide breadth of
use cases.
And think one of the challenges in building in AI over the past,
you know, we've been out of it for eight years now has really been.
Then what are the applications that are getting traction, and how do you build the right infrastructure to fuel those applications?
So as an infrastructure provider, we provide the data foundry for all these AI applications.
Our burden is to be thinking ahead as to where are the breakthrough use cases in AI going in B,
and how do we basically lay down the tracks before the freight train of AI comes When we got started in 2016,
this was the very beginning of the autonomous vehicle cycle.
I think right when we were doing YC was when crews got acquired,
and the beginning of the wave of autonomous driving being one of the key tech trends.
And I think that we followed the early startup advice.
You have to focus early on as a company.
And so we built the very first data engine that supported sensor fuse data.
So support a combination of 2D data plus 3D data.
So LiDARs plus cameras that were built on to the vehicles.
And then that very quickly became an industry standard across all the players, working folks like General Motors and Toyota and Slantas and many others.
In the first few years of the company we're just focused on autonomous driving and a handful of other robotics use cases.
But that was sort of the prime time AI use case.
And then starting in about 2019,
2020, it was an interesting moment where it was actually pretty unclear
where the future of AI use cases where AI applications were going to come.
And this is obviously pre-language model, pre-gender to AI, and it was a period of high uncertainty.
So we then started focusing on government applications.
That was one of the areas where it was clear that there was high applicability,
and it was one of the areas that was becoming more and more important globally.
So we built the very first data engines to support government data.
This was support mostly geospatial and satellite and other overhead imagery.
This ended up fueling the first of record for the USDOD and was sort of the start of our government business,
and that technology ended up being critical years later in the Ukraine conflict.
And then also around that time was when we started working on gender events.
So, we partnered with OpenAI at that time to do the very first experiments on RLHF on top of GPT-2.
This was like the primordial days of RLHF, and the models back then were really rudimentary.
Like, they truly did not seem like anything to us.
But we were just like, you know, OpenAI, they're a bunch of smart people, we should work with them, we should partner with them.
We partnered with a team that originally invented RLHF,
and then we basically continued innovating with them from 2019 onwards, but we didn't think that much about the underlying technological trend.
They integrated this,
all of this technology into GPT-3 with there was a paper instruct GPT,
which was kind of the precursor to CHACHUPT that we worked with them on.
And then ultimately, in 2022, Dolly 2 and CHACHUPT rolled around, and we ended up...
I'm focusing a lot of our effort as a company into how do we fuel the data for gender BI?
How do we be the data foundry for gender BI?
And today,
fast forward to today,
our data foundry fuels basically every major,
large language model in the industry, work OpenAI, Meta, Microsoft, many of the players, partner them very closely in fueling their AI development.
And in that timeframe, the ambitions of AI have just, you know, totally exploded.
I mean,
we've gone from,
you know,
GPD-3 I think was, it was a landmark model, but it was, you know, there was a modesty to GPD-3 at the time.
And now, you know, we're looking at building, you know, agents and very complex reasoning capabilities, multi-modality, multi-leguality.
I the infrastructure that we have to build
to just support all the directions that developers want to take this technology has been really staggering and quite, quite incredible.
Yeah, you've basically way surfed multiple waves of AI.
And of the big shifts is happening right now is there's other types of parties that are starting to engage with this technology.
So you're obviously now working with a lot of the technology giants,
With automotive companies, it seems like there's emergence now of enterprise customers in a platform for that.
There's emergence of Sovereign AI.
How are you engaging with these other massive use cases that are coming now in the generative AI side?
It's quite an exciting time because I think for the first time in maybe the entire history of AI,
AI truly feels like a general purpose technology, which can be applied in a very large number of business use cases.
I contrast this to the autonomous vehicle era where you've really felt like we're building a very specific use case that happened to be very,
very valuable.
Now, its general purpose can be it can be encompassed across the broads.
span.
And we think about, what are the infrastructure requirements to support this broad industry?
And is the broad arc of the technology?
It's really one where we think, how do we empower data abundance, right?
There's a question that comes up a lot, you know, are we going to run out of tokens?
And what happens when we do?
And I think that that's a choice.
I think we as an industry can,
or data scarcity, and we view our role and our job in the ecosystem to be to build data abundance.
The key to the scaling of these large language models, and these language models in general, is the ability to scale data.
data.
And I think that one of the fundamental bottlenecks to, you know, what's, what's in the way of us getting from GP4 to GPD 10.
you know, data abundance.
Are we going to have the data to actually get there?
And our goal is, you know, how do we ensure that we have enough tokens to do that?
And we've sort of, as a community, we have easy data, which is all the data on the internet.
And we've kind of exhausted all the easy data.
And now it's about, you know, forward data production that has high supervisory signal that is basically very valuable.
And we think about this as, you know, frontier production.
And the kinds of data that are really relevant and valuable to the models today, there's a, you know, the quality of data.
increased dramatically.
It's not any more the case that these models can learn that much more from various comments on Reddit or whatnot.
They need truly frontier data.
And does this look like?
This is reasoning chain of thoughts from the world's experts or from mathematicians or physicists or biologists or chemists or lawyers or doctors,
this is agent workflow data of agents in enterprise use cases or in consumer use cases or even coding agents and other agents like that.
This is multi-lingual data, so data that encompasses the full span of the many, many languages that are spoken in the world.
includes all the multi-modal data to your point,
like how do we integrate video data,
audio data,
start including more of the esoteric data types that exist within enterprises and exist within a lot of industrial use cases into these models.
There's this very large mandate, I think for our industry.
to actually figure out what is the means of production
by which we're actually going to be able to generate and produce more tokens to fuel the future of this industry.
And I think there's a few sources or there's a few answers to this.
So.
The first is we need the best and brightest minds in the world to be contributing data.
I it's one of the things that I think is actually quite interesting about this technology is very smart humans.
So, PhDs or doctors or lawyers or experts in all these various fields actually have an extremely high impact into the future of this technology by producing data that
ultimately.
If you think about it, it's their work is one of the ways that they can have a very scaled society level impact.
You know,
there's an argument that you can make that producing high quality data for AI systems is near infinite impact,
because, you know, even if you improve the model just a little bit,
If you were to integrate that over all of the future invocations of that model, that's like a ridiculous amount of impact.
So I think that's something that's quite exciting.
It's kind of interesting because Google's original mission was to organize a world's information and make it universally accessible and useful.
And they would go and they would scan in books, right, from library archives.
And were trying to find different ways to collect all the world's information.
And effectively, that's what you folks are doing or helping out.
You're effectively saying,
where all the expert knowledge and how do we translate that into data
that can then be used by machines so that people can ultimately use that information?
And that's super exciting.
It's exciting to the contributors who are in our network as well,
because they think there's obviously a monetary component that are excited to do this work,
but there's a very meaningful motivation,
which is how do I leverage my expert knowledge and expert insight and use that to fuel this entire AI movement,
which I think is like a deep, that's kind of like the deepest.
is, how do I use my knowledge and capability and intelligence to fuel humanity and progress and knowledge going into the future?
I think it's a somewhat undervalued thing where this is going to age me,
but like there was a decade or so where like the biggest thing happening in technology was digitization of different processes.
And I think there's actually some belief that like,
oh, that's happened, right, like, know, interactions are digital and like information is captured in relational database systems on, you know, customers and employees or whatever.
But one of the big discoveries as a investor in this field over the last five years has been like the data is not actually captured.
Yeah.
almost any use case you might imagine for AI, right?
Because I have multiple companies,
and I'm sure a lot does too,
and you and your personal investing,
where the first six months of the company is a question of where are we going to get this data?
You go to many of the incumbent software and services vendors,
and despite having done this task, you know, for years, they have not actually captured the information you'd want to teach a model.
And like that, you know, that knowledge capture era, I think is happening in scale is a really important part.
To make a dune to analogy, I mean, I it really is spite, you know, data production is very similar to spice production.
It is the, it will be the lifeblood of all the future of these AI.
systems.
And you know, so I think Fast and Brightest People is one key source.
Proprietary data is definitely a very important source as well.
You crazy stat, but JP Morgan's proprietary data set is 150 petabytes of data.
GP4 is trained on less than one petabyte of data.
So there's clearly so much data that exists within enterprises and governments
that is proprietary data that can be used for training incredibly powerful AI systems.
And then I think there's this key question of what's the future of synthetic data and how synthetic data needs to emerge.
And our perspective is that the critical thing is what we call hybrid human AI static data.
So how can you build hybrid human AI systems such that AI are doing a lot of the heavy lifting,
but human experts and people,
you the basically best and brightest,
the smartest people,
the sort of best at reasoning can contribute all of the to ensure that you produce data that's of extremely high quality of high fidelity to ultimately
feel the future of these models.
I want to pull this around a bit because something you and I were talking about both
in the context of data collection and e-vals is like what do you do when the models are actually quite good,
better than humans on many measured dimensions?
And can you talk about that?
from both the data and perhaps, you know, we should talk about evaluation as well.
I mean, I think philosophically, the question is not...
is a model better than a human unassisted from a model?
The question is, is a human plus a model together when people produce better output than a model alone?
And I think that'll be the case for a very,
very, very long time, that humans are still, you
human intelligence is complementary to machine intelligence that we're building and they're gonna be able to combine to build,
you know, to do things that are strictly better than what the models are gonna be able to do on their own.
I have this optimism a lot and I had a debate at one point that was challenging for me.
philosophically about whether or centar play or like machine and human intelligence were complementary.
My simple case for this is when we look at the machine intelligence,
like the models that are produced, you we always, you know, you see things that are really weird.
You know,
there's like the ROT 13 versus ROT 8 thing,
for example, where the models not already ROT 13, they don't know how to There's the reversal curse.
You know,
there's all these these artifacts that indicate somehow that it is not like human intelligence Or not like biological intelligence and I think that's a that's the bull case for humanity
Which is that you know, there are certain qualities and attributes of human intelligence which are somehow distinct from the very separate?
and very different process by which we're training these algorithms.
And so then I think, you what it look like in practice?
It's, you know, if a model produces an answer or a response, how can a human critique that response to improve it?
How can a human expert, you know, highlight where there's factuality errors or where there's reasoning errors to improve the quality of it?
How can the human aid?
in guiding the model over a long period of time to produce reasoning chains that are very correct
and deep and are able to drive the capability of these models forward.
And I think there's a lot that goes into,
this is what we spend all of our time thinking about,
what is the human expert plus,
plus model teaming that's going to help us keep pushing the boundary of what the models are capable of doing.
How long do you think human expertise continues to play a role in that?
So I look at certain models,
Med Palm 2 would be a good example, where we've released a model where they showed that the output was better than the average position.
You still get better output from a cardiologist,
but you just asked a GP a cardiology question, the model would do better as ranked by physician experts.
So it showed that already for certain types of capabilities,
the model provided better insights or output than people who were trained to do some aspects of that.
how far do you think that goes in terms of, or when do you think human expertise no longer is added up to these models?
Is never?
Is it three years from now?
I'm of curious of the time frame.
I think it's never because I think that,
you know, the key quality of your intelligence or biological intelligence is this ability to reason and optimize for a very long time horizons.
And is biological, right?
Because our goals as biological entities is to optimize over our lifetimes, optimize for reproduction, et cetera.
So we have the ability as human intelligence is to produce long-term goals,
continue optimizing, adjusting, and reasoning over very long, very long time horizons.
You know, current models don't have this capability because the models are trained on these like little nuggets of human intelligence.
So very good at like,
almost like a shot glass full of human intelligence,
but they're very bad at continuing that intelligence over a long time period or a long time horizon.
And so,
this, this fundamental model biological intelligence, I think it's something that will only be taught to the model over time
through, you know, through direct transfer via data to fuel these models.
You don't think there's like an architectural breakthrough in planning that solves it.
I think there will be architectural breakthroughs that improve performance dramatically.
But I think if you think about it inherently,
these models are not trained to optimize over longtime horizons in any way,
and we don't have the environments to be able to get them to optimize for these amorphous goals over longtime horizons.
So think this is a somewhat fundamental limitation.
Before we talk about some of the cool releases, you guys have for scale.
Maybe we can zoom out and just congratulate you on the fundraise that you guys just did.
A dollars at almost 14 billion in valuation with really interesting investors like AMD, Cisco, Meta.
I want to hear a little bit about the strategic.
Or mission is to serve the entire AI ecosystem and the broader AI industry.
You know, we're an infrastructure provider.
That's our role is to be as much as possible, supporting the entire industry to flourish as much as possible.
And we thought an important part of that was,
how can we
be an important part of the ecosystem and build as much ecosystem around this data
foundry which is going to fuel the future of the industry as much as possible which
is one of the reasons why we wanted to bring along a other
infrastructure providers like Intel and AMD and folks who are also laying the groundwork for
the future of the technology but also you know key players in the industry like like meta folks like Cisco as well,
you know, our view is that ultimately there's the stack that we think about.
There's the infrastructure, there's the technology, and there's the application.
And our goal as much as possible is how do we leverage this data capability,
this data foundry to empower every layer of that stack as much as possible.
And in a broader industry viewpoint around what's needed for the future of data.
I think that this is an exciting moment for us.
I mean,
we see our role going back to the framing of what's holding us back from QB-10,
what's in the way from QB-4 to QB-10.
We want to be investing into actually enabling that pretty incredible technology journey.
And there's tens of billions, maybe hundreds of billions of dollars of investment going into the compute side of this equation.
And one of the reasons why we thought it was important to raise the money and continue investing is,
you know, there's real investment that's going to have to be made into the data production to actually get us back power comes great responsibility.
If these AI systems are what we think they are in terms of societal impact, trust in those systems is a crucial question.
How do you guys think about this as part of your work at scale?
A lot of what we think about is how do we utilize, how does the data foundry enhance the entire AI lifecycle?
That lifecycle goes from A,
ensuring that there's data abundance as well as data quality going into the systems,
but also being able to measure the AI systems,
which builds confidence in AI and also enables for further development and further adoption of the technology.
This is the fundamental loop.
The company through,
you they get a bunch of data or they generate a of data,
they train their models, they evaluate those systems and they sort of, you know, go again in the loop.
And evaluation and measurement of the AI systems is a critical component of the lifecycle,
but also a critical component I think of society being able to build trust in these systems.
know, how are governments going to know that these AI systems are safe and secure and fit for broader adoption within their countries?
How are enterprises going to know that when they deploy an AI agent or an AI system,
that it's actually going to be good for the consumers and that it's not going to create greater risk for them?
How are labs going to be able to consistently map?
what are the intelligences of the AI systems that we build, and how do they make sure they continue to develop responsibly as a result?
Can you give our listeners a little bit of intuition for what makes ethos hard?
One of the hard things that, because we're building systems we're trying to approximate and build human intelligence.
Grading one of these AI systems is not something that's very easy to do automatically,
and it's sort of like you have to kind of build IQ tests for these models,
which in and of itself is a very philosophical questions like how do you measure the intelligence of a system?
system, and there's very practical problems as well.
So of the benchmarks that we as a community look at for the academic benchmarks that are
what the industry use to measure the performance of these algorithms fraught with issues.
Many of the models are over fit on these benchmarks.
They're sort of in the training data sets of these models.
And you guys just did some interesting research here.
Yes.
So one of the things we did is we published DSM-1K, which a held out eval.
So we basically produced a new evaluation
of the math capabilities of models
that there's no way it would ever exist in the training data set to really see how much of the,
how were the performance of the models, were the reported performance of the model capabilities?
versus the actual capability.
And what you notice is some of the models perform really well, but some of them perform much worse than the reported performance.
And so this whole question of how we as a are actually going to measure these models is a really tough one.
And answer is we have to leverage the same human experts
and kind of the best and brightest minds to do expert evaluations on top of these models to understand,
you know,
where are they powerful, where are they weak, and what's the sort of, what are the sort of risks associated with these, with these models?
So, you know, one of the things that, um,
And going to, we're very passionate about it is there needs to be sort of public visibility and transparency into the performance of models.
So there needs to be leaderboards,
there to be evaluations that are public that demonstrate in a very rigorous scientific way what the performance of these models are.
And then we need to build the platforms and capabilities for governments,
enterprises, labs to be able to do constant evaluation on top of these models to ensure that we're always developing it.
Thank you.
in the world is deploying it in a safe way.
So this is something that we think is just in the same way that our roles in infrastructure provider is to support the data needs for the entire ecosystem.
We think that building this layer of confidence in the systems through accurate measurement
is going to be fundamental to the further adoption and for the development of the technology.
You want to talk about state of AI at the application layer because you have a viewpoint into that that very few people do.
You know,
after GPD4 launched,
there was sort of this frenzy of sort of an application build out and And I think that there was,
you know,
there were all these, like, agent companies, or excitement-around agents, there all these, like, you know, a lot of applications that were built out.
And I actually think it's an interesting moment in the lifecycle of AI,
which that,
you know,
GPD4, I think, as a model was, a little early of a technology for us to have this entire hype wave around and I
think we you know the community very quickly discovered all the limitations of
puberty 4 but you know we all know gb4 is not the terminal model that AI's that we are going to be using.
There better models on the way.
And I think there was an element by which it was sort of a classic hype cycle.
Gb4 came out,
lots of hype around building applications from Gb4,
but it was probably a few generations too early of a model for the 1,000 flowers to bloom.
And so I think in the coming models we're going to see this sort of like trough of
disillusionment I think we're going to come out of because the future models are going to be
much more powerful and you're actually going to have all of the fundamental capabilities you
need to build agents or all sorts of incredible things on top of it.
And we think what we're very passionate about is how do we empower application builders,
so whether that be enterprises or governments or startups to have to build self-improvement into the applications that they build.
So what we see from like OpenAI and others, is that self-improvement comes from data flywheels.
So how do you have a flywheel by which you're constantly,
you know,
getting new data that improves your model,
you're constantly evaluating that system to understand where there's weaknesses, and you're, you're sort of like continually hydrating this workflow.
We think we think that fundamentally,
every enterprise that's workflow,
or government or startup is going to need to build applications that have the self-improvements loop and cycle, and it's very hard to build.
And so,
you know,
we built this product,
our Ginii platform,
to really lay the groundwork and the platform to enable the entire ecosystem to be able to build the self-improvement and it.
into their products as well as possible.
I was just curious,
I one thing related to that is you mentioned that,
for example, JP Morgan has 150 petabytes of data relative, you know, that's times what some early GPT models trained on.
How do you work with enterprises those loops or what are the types of customer needs that you're seeing right now or application areas?
One of the things that all the model developers understand well,
but the enterprises understand super well is that not all data is created equal and high quality data or frontier
data can be 10,000 times more valuable than just any run of the mill data within an enterprise.
challenge in, you know, or a lot of the problems that we solve with enterprises are, how do you go from this?
giant mountain of data that sort of is like all over,
is truly all over the place and distributed everywhere within the enterprise to what are the,
how do you compress that down and filter it down to the high quality data that you can actually use to,
you know, fine tune or train or continue to enhance these models to actually drive differentiated performance.
I think one of the things that's interesting is that there's some papers out of meta,
which basically shows that actually narrowing the amount of data that you use creates better.
So the output is better.
The are smaller, which means they're cheaper to run, they're faster to run.
And to your point,
it's interesting because a of people are sitting on these massive data sets
and they think all that data is really important and it sounds like you're really working with enterprises to sort of narrow that down.
And the data that actually improves the model?
So what's that information theory question in some sense?
What are some of the launches that are coming from scale now?
You know, we're building evaluations for the ecosystem.
So one is that we're going to launch these private held-out evaluations and have leaderboards
associated with these EVALS for the leading LLMs in the ecosystem.
And we're going to rerun this contest periodically.
So every few months we're going to do a new set of held-out EVALS to basically consistently make
that can sure the performance of our models and continue adding more domain.
So we're going to start with areas like math and coding,
instruction following, adversarial capabilities, and then we're going to over time continue increasing the number of areas that we test these models on.
We think about as kind of like an Olympics for LLMs, but instead of every four years, we'll be right back.
a few months.
So that's one thing we're quite excited about.
And then we have an exciting launch coming with some of, with our government customers.
So one of the things that we see with,
in the government spaces,
they're trying to use LLMs and they're trying to use these capabilities as,
there's actually a lot of,
there's a lot of cases where even the current agenda capabilities of the models can be extremely valuable to the government.
And it's often in pretty boring use cases like writing reports or filling out forms or pulling information from one place to another,
but it's well within the capabilities of these models.
And We're excited launching some agentic features for our government customers with our Donovan product.
These are applications you build yourselves or an application building framework.
So for our government customers, we basically build a AI staff officer.
So it's a it's a full application, but it integrates with whatever model.
our customers think is appropriate for their use case.
And do you think scale will invest in that for enterprise applications in the future?
Our view for enterprises is fundamentally like,
how do we,
how do we, For the applications that enterprises are going to build, how do we help them build self-improvement into those products?
So we think about much more as a platform level for enterprises.
Does the new OpenAI or Google release change your point of view on anything fundamentally, multi-modality, the of voice agents, etc?
I think you tweeted about this, but one very interesting element is, the direction that we're going in terms of consumer focus.
And it's fascinating.
I I multimodality, well, taking a step back.
First off, I think it points to where there's still huge data needs.
So multimodality as an entire space is one where for the same reasons that we've like exhausted a lot of the internet data,
there's a lot of scarcity for good multimodal data that can empower these personal agents and these personal use cases.
So I think there's,
you know,
as we want to keep improving these systems and improving these personal agent use cases, there's, you know, we think about this a lot.
What are the data needs that are actually that are going to be required I think the other thing that's fascinating is the convergence,
actually, so both labs have been working independently on various technologies and, you know, Astro, which shockingly similar and sort of demonstrations of the technology.
And there's, I think that was very fascinating.
The labs were converging on the same end use cases or the same visionary use cases for the technology.
I there's two reads of that.
One is like,
there's an obvious technical next step here, and very smart people have independently arrived, and the other is like competitive intelligence is pretty good.
Yeah, I think both are probably true.
It's funny, because when I used to work on products at Google, we'd spend two years working on something.
And then the week of launch,
somebody else would come out with something, and we'd launch it, and then people would claim that we copied them.
And so I do think a lot of this stuff just happens to be, in some cases, just where the whole industry is heading.
And kind of, people are aware that multimodality is one of the really big areas.
And a lot of these things are years of work going into it.
So kind of interesting to watch that as an external observer.
Yeah.
I this is also not a training run that is a one-week copy effort, right?
Well, and then I think the last thing that is that I've been thinking a lot about is like
when are we going to get smarter models?
So, you know, we got multimodality capability.
That's exciting.
It's more of a lateral expansion of the models.
And the industry needs smarter models.
We need GP5 or we need Gemini 2 or whatever that those models are going to be.
And so,
to me,
it you know,
I was somewhat disappointed because I just want much smarter models that are going to enable kind of as we mentioned before,
you know, way more applications to be built on top of them.
The years long, end of year.
Okay, so quick fire and a lot of chime in if you have ones here.
Something you believe about AI that other people don't.
My biggest belief here is that the path to AGI is one that looks a lot more like curing cancer than developing a vaccine.
And what I mean by that is I think that the path to build AGI is going to be in,
you know,
you're going to have to solve a bunch of small problems that where you don't get that much positive leverage between solving one problem to solving the next problem.
And there's just sort of, you know, it's light.
which is you have to then zoom into each individual cancer and solve them independently.
And over a-decade time frame,
we're gonna look back and realize that we've built AGI,
we've cured cancer, but the path to get there will be this quite plotting road of solving individual capabilities and building individual data fly.
to support this end mission,
whereas I think a lot of people in industry paint the path to AGI is like,
you eventually we'll just, boop, we'll get there, we'll like, you swoop.
I think there's a lot of implications for how you actually think about the technology arc and how society is going to have deal with it.
I think it's actually a pretty bullish case for society adapting the technology because I think it's going to be consistent
slow progress for quite some time and society will have time to fully acclimate.
When you say solve like a problem at a time,
right, we just like pull away from the analogy a little bit and should I think of that as generality of multi-step reasoning is really
hard as, you know, Monte Carlo tree search is not the answer that people think it might be.
We're just going to run into scaling walls, like what sort of what are the dimensions of like solving multiple problems?
I think the main thing fundamentally is very limited generality that we get from these models.
And even for multimodality, for example, my understanding there's no positive transfer from learning in one modality to other modalities.
So training off of a bunch of video doesn't really help you that much with your text problems and vice versa.
And so I think what this means is like,
each sort of,
each niche capabilities or each area of capability is we're going to require separate flywheels,
data flywheels, to be able to push through and drive performance.
You don't yet believe in video as basis for a world model that helps.
I think that is great narrative.
I don't think there's strong scientific evidence of that yet.
Maybe there will be eventually,
but I think that this is I think the base case,
let's say,
is one where,
you know,
there's not that much generalization coming out of the models,
and so we actually just need to slowly solve lots and lots of little problems to ultimately result in AGI.
One question for you is like, you know, leader of scale, a scaling organization, like, what are you thinking about as a...
and this will almost sound cliche, but just how early we are in this in this technology.
I mean,
I think that there's You know,
it's strange because on the one hand,
it feels like we're so late because the tech giants are investing so much and there's a launches all the time and there's,
you know, there's, there's all sorts of investment into this space.
But look crowded in the obvious use cases.
Yeah, exactly.
Markets look super crowded.
But I think fundamentally we're still super early because the technology is, you know, one one hundredth or one one thousandth of its future capability.
And as we, as a community and as an industry and as a society ride that wave.
there's so many more chapters to the book.
And as a, you know, you about any organization, what we think about a lot is nimbleness.
Like do we ensure that as this technology can be developed that we're able to continue adapting alongside the developments of the technology.
Right, that's a great place to end.
Thanks so much for joining us today.
Yeah.
Thanks, Alex.
Thank you.
Find us on Twitter at NoPryersPod.
Subscribe our YouTube channel if you want to see our faces.
Follow the show on Apple Podcasts, Spotify, or wherever you listen.
That way you get a new episode every week.
And sign up for emails or find transcripts for every episode at No-Pryers.com.
Çeviri Dili
Seç
Daha Fazla Özelliği Açın
Trancy uzantısını yükleyerek, AI altyazılar, AI kelime tanımları, AI dilbilgisi analizi, AI konuşma vb. dahil olmak üzere daha fazla özelliği açabilirsiniz.
Büyük Video Platformlarıyla Uyumlu
Trancy, YouTube, Netflix, Udemy, Disney+, TED, edX, Kehan, Coursera gibi platformlar için çift dilli altyazı desteği sağlamakla kalmaz, aynı zamanda düzenli web sayfaları için AI kelime/cümle çevirisi, tam metin etkileşimli çeviri ve diğer özellikler sunar. Gerçek bir çok yönlü dil öğrenme asistanıdır.
Tüm Platform Tarayıcıları
Trancy, iOS Safari tarayıcı uzantısı dahil olmak üzere tüm platform tarayıcılarını destekler.
Çoklu Görüntüleme Modları
Tiyatro, okuma, karışık ve diğer görüntüleme modlarını destekleyerek kapsamlı bir çift dilli deneyim sunar.
Çoklu Uygulama Modları
Cümle dikte, sözlü değerlendirme, çoktan seçmeli, dikte ve diğer uygulama modlarını destekler.
AI Video Özeti
Videoları özetlemek ve hızlı bir şekilde temel içeriği kavramak için OpenAI kullanın.
AI Altyazılar
Sadece 3-5 dakikada doğru ve hızlı YouTube AI altyazıları oluşturun.
AI Kelime Tanımları
Altyazılardaki kelimelere dokunarak, AI destekli tanımlarıyla kelime anlamlarını öğrenin.
AI Dilbilgisi Analizi
Cümle dilbilgisini analiz ederek cümle anlamlarını hızlıca anlayın ve zor dilbilgisi konularını öğrenin.
Daha Fazla Web Özelliği
Çift dilli video altyazılarının yanı sıra, Trancy ayrıca web sayfaları için kelime çevirisi ve tam metin çevirisi sağlar.