MIT143102016-V034000 - 双语字幕
Now, let's think about harvesting data.
So two ways to harvest data.
First, you harvest data that's already there.
You go with your software.
One you need to find is like a combined tractor or whatever.
It's already there.
We don't need to plant it.
And then sometimes you actually need to plant it first.
That takes a little bit more time sometimes, but that's also useful.
So let me go over these two.
So what's grape scraping?
It could be full data from some data from some It could be calling an entire webpage, maybe from form to form.
It could be a set of forms running in the background,
so I'll give you an example where you keep a sort of querying,
for example, the of books, and any of the above in an ongoing fashion where you keep sending.
What does crawling mean?
Crawling means searching all over the website.
So in a page, you could just find one page and extract what you want from one page.
Or you could just keep going to the website and extract,
like, go through, for example, If you want to look for the, you want to construct the database of the price of
light, calling will involve going from, you know, keep querying.
This is, you know, Boston L.A.
And then Boston to Phoenix and Boston to San Diego.
So you're calling the site to find out what it is.
Like a color.
So to get data from the internet, that's not already in available form.
There are two big classes of methods to do that.
First of all, many of the big websites maintain what's called an API.
our application program interface, which basically help a particular program or a particular application to communicate with other applications.
So if you want data from Twitter,
or you want data from Facebook,
or you want data from Google Map,
you will not physically go on each of the website and then copy the information and put it in your database.
I mean, you could, and I'll show you a little bit how to do it later.
But it's not typically the way you would do it.
Typically something like Google Map,
I'll show you an example with Google Map,
but will have an API which is, they will directly give you the data that you need upon receiving a query from you.
Sometimes it is free,
sometimes it is, you have to pay, sometimes it is free for a little while, and then you have to pay for the rest.
Sometimes you first need authorization from the person, from the data you want, etc.
The Twitter,
a lot of the data is public anyway, because once you can follow someone and once you follow them, you can get their news.
So a lot of data from Twitter is perfectly fine.
And way you would do that is not by actually doing it.
scraping from the Twitter page the way you would do that by communicating with Twitter to an API and telling exactly what you need.
Say you want all of the tweet for
解锁更多功能
安装 Trancy 扩展，可以解锁更多功能，包括AI字幕、AI单词释义、AI语法分析、AI口语等
兼容主流视频平台
Trancy 不仅提供对 YouTube, Netflix, Udemy, Disney+, TED, edX, Kehan, Coursera 等平台的双语字幕支持，还能实现对普通网页的 AI 划词/划句翻译、全文沉浸翻译等功能，真正的语言学习全能助手。
支持全平台浏览器
Trancy 支持全平台使用，包括iOS Safari浏览器扩展
多种观影模式
支持剧场、阅读、混合等多种观影模式，全方位双语体验
多种练习模式
支持句子精听、口语测评、选择填空、默写等多种练习方式
AI 视频总结
使用 OpenAI 对视频总结，快速视频概要，掌握关键内容
AI 字幕
只需3-5分钟，即可生成 YouTube AI 字幕，精准且快速
AI 单词释义
轻点字幕中的单词，即可查询释义，并有AI释义赋能
AI 语法分析
对句子进行语法分析，快速理解句子含义，掌握难点语法
更多网页功能
Trancy 支持视频双语字幕同时，还可提供网页的单词翻译和全文翻译功能