AutoGEN + MemGPT + Local LLM (Complete Tutorial) 😍

AutoGEN + MemGPT + Local LLM (Complete Tutorial) 😍 - 雙語字幕

Alright guys, the much awaited video is here and I've worked like 4 hours to get the solution.

At one point of time I thought of moving to the next project, but hope kept me going.

In this video I'm going to show you how you can connect memgpt,

autogen and local large language models using run pods,

so you're going to connect your local large language models with autogen and memgpt, let's get started how this is done.

So I have this code here.

and I shall share this code in the description or I will attach a link to the GitHub repo and you can run the code.

Thanks for tuning in.

Bye bye.

Have a nice I'm just kidding.

In this video,

we are going to see each and every detail on how I got to the solution and thought process that I had in obtaining the solution.

This has been a great problem because if you watch my videos, you can see the first video which was the autogenoid.

local LLMs get rid of ABI keys.

This is an interesting video you can watch and if you head over to this video,

memgpt with local LLMs or this video, autogen will local LLMs and this video.

In this video I powered the memgpt and autogen separately using LLMs and I received quite a number of comments where people

wanted to see an integration of everything together.

So this is where you have local large language models, you have autogen, you have memgpt working together.

Actually you can see in the autogen we have different agents because this is a multi-agent framework and if you look at the examples,

so for example we have these agents and we wanted to work something like this.

User proxy agent is the normal autogen and for example, the assistant agent is a memgpt agent and That would be interesting.

That is what people wanted and all the API is that it is going to use is from the From the local models from the local large language models

For example, dolphin 2.0 mril.

You use Aeroboros,

but not all models are ready as of yet to be used in memgpt right now we I think only we have two models which can work for memgpt that is

the dolphin 2.0 and 2.1 and then we have arrow bars you can check up

the documentation for finding so you can go to memgpt then github

and you can go to this place where you can run memgbt with local LLMs.

So you can click here and this is where I went for finding in a solution.

So okay we have Araberers, we Zephyr, Zephyr, we have Dolphin as well.

So I will try with Zephyr later.

In this video we're going to look at Dolphin 2.0 because comparing Dolphin 2.1 and 2.0 I found the Dolphin 2.0 better than the Dolphin 2.1

but having said that and having the base understanding of what we're going to do

let me summarize again what we're going So auto gen has these different agents.

And now we want to replace one of these agents with memgbt agent, which has infinite memory.

So in order to get more details of what memgbt is and how does it function.

And if you want to look at the paper, I have, you can check out the videos that I have.

I have published and this is an extensive video on explaining what MEMGPD is and in the last

few days I have been publishing videos related to auto-gen and MEMGPD, it's like a crazy technology.

and today we have found the solution to do this, do the integration.

So if you're a beginner and you want to get everything from the start, just you need to install Python.

So I'll start from the very basics.

So you need to install Python and you need an editor, which is known as VS code.

This is my favorite.

So don't download the recent version.

I'll go for something like Python.

And after you did this, next we need to create an account of my runpots.

I have a link to this.

It please go and check in through my link.

I will do get an affiliate commission though.

So you go there and create an account there and add some credits.

You need to put some credits, but if your system is very strong, you don't need a cloud GPU.

You can do it on your own system.

using the Ooga Booga text generation web UI.

So my machine is a little bit weak, a week, so I have decided to try it on run pods.

And now let's get started.

So first thing you're going to need to do is go ahead.

and open up a new folder open up a new folder open folder so let me select a folder let me go to

my projects so this is 167th project and let's say mem gpt auto gen and LLM.

So this is the folder that I've selected.

Select the folder here.

Next, as always, we need to make a virtual environment.

So we go to View, Command-Palette, create VENV, use Python 3.11.

the environment has been created.

And then we are going to make our code here.

So first of all, I'm going to click here where it says add new file.

I'm going to say app dot pie.

This will create a new file.

I'm going to load up a terminal known as new terminal.

Okay.

So we have this and this environment is selected.

So we go to V and V, we go to scripts, then we go to activate dot bat.

And here we need to So after activation, you can see that this is dot v and v.

So if you know this, it's great.

But if you don't, don't worry.

So city dot dot city dot dot, we go back to the main folder now.

So that we can access the app dot pi here.

So this is the configuration has been done.

Now, let me go ahead and copy the code, because it's not that I'm going to code it now, because it it's not

feed a few hours for me to get the solution.

So you can see the wiggly lines that we have here.

So it means that we need to install some things.

Let me start from the bottom.

So let me make some space.

So here we have open AI.

So we need to install pip install open AI.

And let's wait for the installation to complete.

After this we would need to install the autogen here.

So for installing autogen, let me just make some clearance here.

CLS.

So would say pip install py auto gen by auto gen.

Okay, this will install the pi auto gen packages or auto gen packages.

So where are the packages installed?

These packages get installed in the V and V.

Then we have this libs library and the site packages.

So it gets installed somewhere here.

You can see this folder.

This is auto gen.

So this is successfully done.

Let me clear again.

And now we can install the pi mem GPT.

Okay, so click press enter.

And this is going to download the the mem GPT as well.

Okay.

All right, it is downloading.

Okay, this is done.

So let me clear this up again, CLS.

So we have installed everything and all the wiggly lines are gone now.

Let us look at the code.

And then we are going to have an a look at the integration of the API.

So first of all,

we have import the operating system,

import the auto gen, and then there are quite a few modules that you need to download from the mem GPT library.

So from the mem GPT,

we have auto gen mem as memgpt origin,

then we have the interface, agent, system, utils, presets, constants, personas, humans, persistent manager, and then we import the open AI.

Of course, we are not going to use open AI.

API, but this is just a decoy for the code.

And the code will think that it is using the open AI, but we are going to use a different endpoints.

And that is our local LLM endpoints.

So that is just a trickery that we are going to play with this code.

Okay.

So just like in order gen, we define the configuration here.

And this is API.

an AI API base, we're going to look at this, what is this, we're going to look at that.

And we are going to pull this from the run pods, where we are going to host our local elements.

And API key,

we don't need to put anything next we have this flag so use memgpt we can say true or false when we say it's

false then it would be a normal autogen kind of thing where you would have a

user proxy and then you are going to have an agent.

Okay, we'll look into that.

Don't worry.

But when we say use memgpt is equal to true, then it is going to use the memgpt.

Okay.

So what you can do now is another thing is I'm just going to paste it here.

So LLM config is it is going to use the configuration list here.

And this is the LLM config.

So this is used by Next, we are going to need the configuration, the API list, the API keys for the mem GPT to work.

So this is where the whole configuration is.

We'll have a deep dive into this, but let's look is the configuration part, we'll come back to this again.

But just like the, in the auto gen, we're going to define the user proxy here.

So this is a user proxy name is the user proxy.

It's a human admin,

it is going to look up the last two messages,

because if you put in,

for example,

five messages,

it is going to take a lot of tokens and the dolphin model which we are going to use has only 2000

tokens and it's very difficult with 2000 to take a very large context.

So human input mode is terminated.

We would tell you terminate at the end, we need to type in exit so as to exit.

And default user is you are going to figure it out on your own work by yourself,

the user won't reply until you output terminate to end the conversation.

So it will ask assistant agent to type terminate and then the user proxy agent,

which is a replacement of us, the human beings would do something about it.

This is just a standard thing.

Next, we have this new thing.

This is the interface that you need to make using the module known as AutoGen interface,

which is the module that we have created using the MemGPT library.

All right, so this is going to use the MemGPT agent here.

And what else?

We need the persistent manager.

We start this.

We have this persona.

This is I'm a 10x engineer trained in Python.

The human is, I'm a team manager at this company and MemGPT agent.

So this is the main agent of MemGPT that you're going to use.

So for the MemGPT, we have the presets and we're going to use the presets here.

The preset is the default preset model is GPT4, but we're not going to use the API of GPT4.

persona is persona human is human interfaces interface and persistent manager is the persistent

manager and agent config is the LLM config that we are we have already placed here.

This is the LLM config.

Okay, so at this step, I hope you're able to follow this step is a statement.

not use memgpt.

So if we have put use memgpt here as false, then this means that we are going to use the auto agent, the normal example.

So this is the auto agent.

That's a normal auto agent and assistant agent here.

That is a 10x engineer training Python.

But this is a step where we need, where we want to use memgpt.

So when this flag is true,

then this will be the code that it will run, then we're going to print memgpt agent at work, then this is the coder.

So memgpt auto agent and memgpt agent.

the name is memgbt-quarter and agent is this memgpt agent.

Okay, so as like the autogen, we are going to initiate the chat with asking the quota to write a function to print numbers 1 to 10.

I hope this is clear, but I would like to summarize it again.

So here we install all the libraries, import all the libraries.

First of all, you install all the libraries, then you import this, we set up the configuration list here.

We also can change the flag to true or false, so that auto agent or the memgbt.

This is the user proxy.

This will always be the same.

But here we have the auto agent or you can use the mem gpt using these configurations here and we initiate the chat here.

So what remains is how I was able to integrate this using the API keys and how to place the API keys how to use the API keys this is what

we're gonna do here.

So in order to get these API keys you can just use gpt4 and that is easy because integrates well.

But if you look at the cost that I've already shown in the previous video,

if you go to platform,

just hold on,

if you go to platform.openai.com,

then you go to the usage, you would find that, you know, these cost $5 and just running it.

twice.

I mean, this is so crazy.

So if you don't want to have,

you know, spend cost like this, this is what we have been looking for as the solution for this.

So this is the solution now.

This is starts at using the Run parts here.

So you create an account, add some credits and go to templates then you go to run the blocks LLM here, deploy this.

So this is selected, the template is selected.

Next we are going to use the...

GPUs.

My favorite is this one RTX 6000, keeping in mind the cost and the availability and the RAM.

So here 7860,

you can watch my other videos to get a deep explanation of this but 7860 is the Ooga Booga Text Generation Web UI and we have 5000 here HTTP ports.

But what you're going to do is add one more and that will be our port through which we are going

to use the memgbden or the gen together.

So we click on set of rights, click on continue, click on deploy.

This is going to load up the pods.

You click on my pods and you can click here.

You can see that the pod is getting ready.

Now let's wait for it.

Okay, so click on connect then and you can see that this port is not ready.

We will make it ready, but first go to 7860.

This is going to open up the ukha booga text generation web UI and tell me how are you feeling

about this technology go to model Then go to this,

we're going to use delphin 2.0,

delphin 2.0, mril 7b, click on copy here and you click on paste here, paste it here, then click on download.

This is going to load up the files that's going to download the files.

Okay, so the download is done and next we are going to refresh this.

So it's going to load up the downloaded models here.

Then press click on none and then click on the model back.

then click on load and this is going to start up the loading sequence and let's wait for it successfully loaded.

Now once this is loaded we need to open that 5001 port.

So for opening that port you need to and then you need to click on open AI and then apply flex extensions and restart this.

Once you start restart that it will restart the web UI interface and now if you go to model and Okay,

you go to my pods and here you see that this port has been opened And now this is the API key that you are going to be using for our project

So run pod has just downloaded the models the large language models local large language models And,

well, just a model here, so it downloaded a Dolphin 2.0 model here, and we have opened the port of 5001, which is the OpenAI's API key, sort of, which will behave like

the OpenAI API key, okay?

So we head over to text generation, and then we are going to copy this control C.

And we had, we're going to head over to the code, and we're going to paste it here.

So this is our API base.

So it here,

make it V1,

and make it 5001 okay we are also going to copy the same thing here and put it here okay so this is done so

this is done just copy the text definition web UI link to this location change it to 5001 and change it to V1.

So this is the entire configuration that we've been looking for so control C and now for the moment of truth I'm gonna make some space here so as to see

the results better and I'm going to change the flag to false.

So initially we are just running the normal autogen.

So we are first just running the normal autogen where we are going to have see,

and we are going to have a quarter, but this will be an auto gen quarter.

So let's see the output.

So how do you run this?

So say Python.

So let's run this Python app dot pi.

And let's see, we are using only auto gen.

So Python app dot pi.

What is this?

We have this code.

And okay, we start here from here.

We have the writer function to print numbers one to 10.

We have this code here and it executed the code to get this.

So this works here and now, okay, and now if we change this to true, and then run this by an app.py.

Let's see if it works now.

When you set this to true, the agent that we're going to use.

So we have this output mem gpt agent at work.

And now again, we have the user proxy saying that write a function to print numbers one to 10.

And we get this beautiful output.

It's pretty great.

But, and you can see this, and which means this is running and it's pretty great.

So now combined with this knowledge, combined with this power, what we can do is we can set up very smart problems here.

We can set up a very default reply or we can set a good human admin for the different functions

that you want this agent to be used.

Okay, so I think this should, I should end the video because this is quite a long video.

But having said that, I would like to summarize everything that we have done today.

Here, first lot

Here, first of all, we

we wanted to

to make use a memgpt agent here and for that we started up the web generation

the text generation web ui from using the myPods we have a pod running here and we

started up the text definition web ui we downloaded a model and then we used

URL and change this to 501 and change this to v1 here so as to get the so as to set the APIs here.

So this is the API of the auto agent, auto gen.

And this is the API of memgpt.

After using the API, we were able to change the flag here.

So where is the flag?

We are able to change the flag.

So when you say this false, this is just the auto gen running when we set this to true.

the both the autogen and memgpt running.

So I will attach the link to the codes and this is just a project.

Now that we are able to connect the autogen and memgpt,

we should be able to take on bigger projects, especially combined with the power of local elements, because the cost of gpt4 is huge.

So now you have this part.

please try this and mention in the comment section if you face any

difficulties in doing the same if you don't have my run parts if you don't

have run parts or if you have a very very strong GPU machine then you can use

that or you can use run parts through this link that will help me a lot also I

thank you for watching this video till the end and I will come up with new

videos just again on different technologies that I find is you know

shareable to you all and this is a very deep dive video of how to

get everything installed and please mention in the comment section if you find this

sort of videos interesting or you want to change the style of presentation

you want to change something I'm very open to seditions because this is a new channel and ready to change to anything that you want.

So please mention in the video section,

please mention in the description of what you want to see,

because I have so many videos lined up so many things that I want to share that I got that I get hang up on,

you know, what to present next.

So having said that, this is your host.

I think you enjoyed the video.

Therefore, please press the like button, subscribe to my channel, and yeah, please watch these other videos that are appearing.

Please watch the videos that I mentioned in the start of the video.

that will help you get a deep understanding of that of everything that is happening in this space.

Now having said that I will be back in the new video until then continue watching this video.

Thank you and have a nice day.

翻譯語言

選擇翻譯語言

視頻總結導出字幕口語評測

解鎖更多功能

安裝 Trancy 擴展，可以解鎖更多功能，包括AI字幕、AI單詞釋義、AI語法分析、AI口語等

兼容主流視頻平台

Trancy 不僅提供對 YouTube、Netflix、Udemy、Disney+、TED、edX、Kehan、Coursera 等平台的雙語字幕支持，還能實現對普通網頁的 AI 劃詞/劃句翻譯、全文沉浸翻譯等功能，真正的語言學習全能助手。

支持全平臺瀏覽器

Trancy 支持全平臺使用，包括iOS Safari瀏覽器擴展

多種觀影模式

支持劇場、閱讀、混合等多種觀影模式，全方位雙語體驗

多種練習模式

支持句子精聽、口語測評、選擇填空、默寫等多種練習方式

AI 視頻總結

使用 OpenAI 對視頻總結，快速視頻概要，掌握關鍵內容

AI 字幕

只需3-5分鐘，即可生成 YouTube AI 字幕，精準且快速

AI 單詞釋義

輕點字幕中的單詞，即可查詢釋義，並有AI釋義賦能

AI 語法分析

對句子進行語法分析，快速理解句子含義，掌握難點語法

更多網頁功能

Trancy 支持視頻雙語字幕同時，還可提供網頁的單詞翻譯和全文翻譯功能