Crawler 3T

Đây là thư viện dùng để bóc tách dữ liệu html

Installation

npm install crawler-3t

Usage

Class ModelMongoose

1. mod_sources

• name_index
• SourcesNews
• Articles

2. mod_baogom

• name_index
• mod_acticles
• mod_links
• mod_categories

Class HtmlParser

1. GetHtmlDoc

• body: html
• $: jquery

  GetHtmlDoc(url,function(error, body, $));

Class HtmlExtract

1. getTitle

 var title =  getTitle($);

2. getDesc

 var description =  getDesc($);

3. getImage

 var url_image =  getImage($);

Class ReadRss

1. getListFeed

getListFeed(url_rss,function(error,list_feed));

2. getListFeedByBodyXml

getListFeedByBodyXml(bodyXml,function(error,list_feed));

UploadImage

var UploadImage = require('crawler-3t').UploadImage;
var img_url = 'https://s.aolcdn.com/hss/storage/midas/8935b712fc16c493a66b57c8b5ec7f03/203531071/google-translate-ai-2016-03-11-01.jpg';

UploadImage.Upload_Postimage_Org(img_url, function(data) {
    console.log(data);
});