Stable Diffusion TripoSR Transform Image To 3D Object - Is it Good?

Stable Diffusion TripoSR Transform Image To 3D Object - Is it Good? - 雙語字幕

Hello everyone,

so today I want to talk about the new 3D models from Stable Stability AI that is partnership with Tripo AI and create a new models called Tripo SR.

Now, Tripo SR is a 3D model that able to generate from a flat 2D image and transform it into 360 degrees angles objects like this.

So when you can have an image,

for example, we have a stable defusions image, we can using that and transform it into a 3D object.

And eventually we can,

maybe later we can use newer stable defusions,

three models or stable videos,

defusions, newer versions model, and create animations using these three 3D objects, and how can we do that, and we don't know when those new Stable Diffusions

3 models release.

yet.

But right now at this moment, this triple SR models are released and happening already on the internet.

So you can download these models in hugging face or you can use official website to run this if your hardware is kind of not powerful enough to handle this.

And you can use hugging face if you have powerful hardware computers.

Now in here so far,

I see is that they are training triples for five days on 22 GPUs notes, which is using to train these models.

So I'm not sure if we require something high end like that to run this, but then let's try in here because why not, right?

We can download these models.

So basically you go to this hugging phase and click to the files and versions here, you can download this CKPT files extension.

Now, this is the models for Tripo SR models files.

Now, we will talk about how to installation that in local machine later.

But first of all, let's see what is this all about.

So, Stability AI, our partnership with Tripo AI, and creating this model.

This is an individual company that is specialized.

in creating 3D animations.

They also created this service that allow you to use that online to just generate 3D objects in the website.

And there you have the web platform that means use their website on here to create 3D objects.

And you can use API call if you have back-end server program you want to integrate that with your software.

You do that as well.

And they have some pricing if you wanna take a look at that,

if you really need that for online processing, 3D objects, and that is up to you.

So let's see, they have some examples here.

As you can click the bubbles on here and they have showing these objects and then let's see like this one,

you see the explosions from the volcanoes and then you see hamburgers.

That is 3D.

So actually their 3D's are it's not just flat pictures and then create multiple angles without any layering and texture and actually you dot.

And hopefully what they claim in here is true that all these little dots creating the texture of 3D objects.

So here's the demos.

You guys can take a look when you have time and that's basically showing all the 3D's.

How can they use this AI 3D generators to generate such 3D objects and put that into and animations videos like this.

So that is pretty cool things.

Now let's go to how to install this now in github.com.

I have seen a new project called Sotripo senior.

This models obviously we can use in comfy UI already right now today.

All we did is install this custom node.

and install the model files from this hugging face link and then we can run this but before that we

have to do the installations so let's go through here we have to clone the project or you can download

the cp files on the top of this hugging github page and then you have to go to the subfolder of

this custom notes and go through this install dependency in command prompts.

So basically that is really easy.

You just go to each of this, just copy this one and go to command prompt and copy and paste.

So after that we have to download the models files.

Of course, we are seeing this one in HuggingFace, then we can restart our Comfy UI and it's done.

So this is a demo of their very basic models to generate 3D objects from an image.

So one image are able to generate 3D objects and you are able to view these objects 360 degrees and ups and downs.

view of this object.

So let's try and install this.

So the first thing we got to do is we have to download this one.

For me, I remember all the procedures in the installations already.

It's very simple.

So just click this download button and then we can locate this models file in our comfy UI models folder and checkpoint.

Now in here, I want to rename this models file because I just don't want this raw models CKPT, right?

And later on, we don't know what is that going on.

So let's put that into triple SR and then I give it an underscore and there we go.

Now we are done with downloading the models right now.

And then we have to install this.

So basically what you can do is go to the custom notes of the comfy UI.

So I have that in here.

And basically for Windows, you can go to here, just copy this line and go to Windows Type CMD.

And you got this command prompt open.

You got the GitHub.

your local machine and whatever, and you can do that.

And I just, yeah, just few seconds and I download the whole project already.

And it's a very small size actually, it's just few folders and few files here, very small size.

And then you go into the subfolder of that, which is this one, not in here in command prom.

So we got to do CD and then the subfolder's name.

And we have located in there, then we got to start the install dependency.

So all the requirements TXT files are list every modules and dependencies that we got to install.

So it's all automatically downloaded in here.

It's basically very easy simple.

You don't have to take care of anything.

Just copy and paste and press the enter button.

So we can try and restart our comfy UI.

Yes, so we got this installed and let's go play around with 3D models in comfy UI.

All right, so this is a very simple examples from the comfy UI custom notes project in GitHub.

And as you can see, there is a loader, a model loader, and then a sampler for this 3D modeling.

And then you have to load an image as a reference image to create that.

So let's try something like this and see.

So actually for remove backgrounds, I have already did something experiments in my comfy UI already.

So you guys can check it out using comfy UI essential.

This is another custom note that you can do remove backgrounds because I saw some of their examples in here do have using the background remove

So let's see if we need that or not

so after I refresh this button in comfy manager and then actually I have did the right

installations and I have that in here as you can right click the menu and you can see Trepo SR, this is the one.

We have the model loader, we have the sampler and viewer, that is the three models custom notes for these 3D models.

So we can start play around with this one.

And let's say we have sampler and let's say we have viewers.

Okay, so the viewer is pretty nice.

So let's try with the WAS notes.

This is because this one removed background and transparency.

I want to see if this can handle transparency of the triple SR viewers.

So right here, we have the sampler.

So actually, let's go to put that in the middle because sampling and then the beginning of here we have the model loader.

So model loader we have to select the one that we just download of course that is the triples ckpt files

So I got that one already downloaded in here

So I have renamed the files actually you guys have to rename this file as

well If you want to use that because you don't want just a raw models files name and list all over your model loader

So this is how I manage that.

So models connect to the sampler models.

That is very easy, simple.

And then the reference image.

Actually, I want to do this reference image from here.

The one that I'm totally removed the backgrounds.

So here's the preview image of removed backgrounds and I want to see how that will looks and things like that.

Oh, actually we cannot use this one because we need the reference mass.

So these custom nodes do not have that.

So let's go back to the comfy UI essential.

This one, we got the mass.

So this is a very complete solution for what we need to do and we can delete those because we don't need those anymore.

This is the preview of that.

So let's bring it up to here.

I want to preview just see what kind of thing it will be like.

So we have to use this RMBG section.

So using this one, you can remove backgrounds in this section.

Just a custom node to help you remove background, that's it.

And then we got an image, and then the output, the mass of the remove background area.

So put that in the reference mass, and then also the image.

Put that in reference image as well.

Then are good to go.

and this one we don't need that and right here we have to mesh so mesh is actually the

3d multiple angles data type in here so we have to connect this one so threshold 25 and then the resolution 256.

I would not set this one right now and try the default settings,

which is this one, and then we will try that later on, let's see.

So models once again, loader here, remove sections, and we are good to go.

Let's try it.

So we got something need to download because the first time I loaded using these features of removed backgrounds.

Okay, so we got this one actually, it's like a clay toys mixing with a character and it doesn't look really good on here.

So far I test this one is kind of not performing good for human characters,

especially you have a very detailed,

and well,

this is a real pictures and then If you see this one and after all found out that this model could use on games like

Minecraft They are going to use that for similar like Minecraft game to do 3d objects

So that's why their input image in here for the demo is not going to be a real human characters or very detailed 3d objects

objects.

And they are using some simple objects to do their demos here.

So yeah, I'm going to try with some simple objects like a rabbit or a dog or a robot like this one.

And we can try it.

So by trying this, I'm going to use a text to image workflow.

This is a very simple one.

Don't be surprised.

How can I create that, just click this default workflow button and you can have this very simple text to image workflow.

Then I'm going to paste that in here and let's see.

So I'm not going to use the safe image instead.

I will be connecting the image output to preview image for this one Then we don't need the load image in here.

So this will be a text to 3d objects Simple workflow.

So what we have by default income vui of course is the purple glass bottle That will be our by default that one and then we got to try something like a real vis or the

ref animate Let's try this one.

So we got the glass box have the glass bottle on here and hopefully this simple object shape is going to be worked.

So yeah, there you go.

We have at least a better quality than the human characters.

There you go, this is glass bottles in 3D objects, but some of that is kind of missing in here.

Maybe this model have not fully trained it in this first versions of their models.

Hopefully they have an update later, and let's try it.

Let's see a rabbit.

Yeah, there will be like an easier shape for this kind of text to 3ds or image to 3ds model to create a rabbit like that.

Yeah, that is going to be okay for this kind of models, 3D models.

So, as you can see, there's the back here actually is generated by AI, and then they're referencing the front of this.

And actually, we got the middle and the back of here.

But obviously, in this first sampling generation, it's not going to be very detailed.

You've got eyeballs and things like that in the other end of the image,

but actually seeing this model Start to understanding what is the object shape and the things on the opposite side of this rabbit.

Maybe let's try a robot Okay, let's try this robot.

Okay, we got a pretty complex shape for 3d models.

Let's see if this overcome this challenge.

Yeah, it's not going to be looking good for this one, but it is still able to generate

the front and then the sign here is still able to project the shoulders of this robot.

And yeah, it looks pretty okay.

It's not too detailed for the draft of 3D objects is okay.

And let's see like a plant, let's see plants, something like that.

As you know that in Minecraft, those 3D objects, like simple objects like that, is always appearing in the games.

And this model are working for those kind of objects.

Generations okay.

And if you zoom out, put that into one of the game backgrounds and it will be looking like that.

So yeah, there you go.

This is very cool features and new 3D models from Stable Diffusions and Tripo AI.

And if you cannot run Compuai locally, you can try that out in their website, it's tripo3d.ai.

take a look at that and I will see you guys in the next videos, have a nice day, see you

翻譯語言

選擇翻譯語言

視頻總結導出字幕口語評測

解鎖更多功能

安裝 Trancy 擴展，可以解鎖更多功能，包括AI字幕、AI單詞釋義、AI語法分析、AI口語等

兼容主流視頻平台

Trancy 不僅提供對 YouTube、Netflix、Udemy、Disney+、TED、edX、Kehan、Coursera 等平台的雙語字幕支持，還能實現對普通網頁的 AI 劃詞/劃句翻譯、全文沉浸翻譯等功能，真正的語言學習全能助手。

支持全平臺瀏覽器

Trancy 支持全平臺使用，包括iOS Safari瀏覽器擴展

多種觀影模式

支持劇場、閱讀、混合等多種觀影模式，全方位雙語體驗

多種練習模式

支持句子精聽、口語測評、選擇填空、默寫等多種練習方式

AI 視頻總結

使用 OpenAI 對視頻總結，快速視頻概要，掌握關鍵內容

AI 字幕

只需3-5分鐘，即可生成 YouTube AI 字幕，精準且快速

AI 單詞釋義

輕點字幕中的單詞，即可查詢釋義，並有AI釋義賦能

AI 語法分析

對句子進行語法分析，快速理解句子含義，掌握難點語法

更多網頁功能

Trancy 支持視頻雙語字幕同時，還可提供網頁的單詞翻譯和全文翻譯功能