Welcome to OGeek Q&A Community for programmer and developer-Open, Learning and Share
Welcome To Ask or Share your Answers For Others

Categories

0 votes
3.6k views
in Technique[技术] by (71.8m points)

nodejs代理采集找不到资源

const http = require("http");
const cheerio = require("cheerio");
const iconv = require('iconv-lite');
const BufferHelper = require('bufferhelper');

var App = {

isDoing: 0,
concurrentCount: 0,
start: function () {
    var _this = this;
    var datas = [];
    var opt = {
        host: 'www.66ip.cn',
        port: '80',
        method: 'GET',//这里是发送的方法
        path: 'http://www.dyjqd.com',     //这里是访问的路径
    }
    http.get(opt, function (res) {
        var bufferHelper = new BufferHelper();
        res.on('data', function (chunk) {
            bufferHelper.concat(chunk);
        });
        res.on('end', function () {
            var html = iconv.decode(bufferHelper.toBuffer(), 'GBK');
            console.log(html);
        })
    });
}

}
App.start();

运行后
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN""http://www.w3.org/TR/html4/strict.dtd">
<HTML><HEAD><TITLE>Not Found</TITLE>
<META HTTP-EQUIV="Content-Type" Content="text/html; charset=us-ascii"></HEAD>
<BODY><h2>Not Found</h2>


<p>HTTP Error 404. The requested resource is not found.</p>
</BODY></HTML>

如果不采用代理
http.get('http://www.dyjqd.com', function (res)
直接是可以采集到内容,不知道哪里写错了,请求大神指导


与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
Welcome To Ask or Share your Answers For Others

1 Reply

0 votes
by (71.8m points)
等待大神解答

与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
OGeek|极客中国-欢迎来到极客的世界,一个免费开放的程序员编程交流平台!开放,进步,分享!让技术改变生活,让极客改变未来! Welcome to OGeek Q&A Community for programmer and developer-Open, Learning and Share
Click Here to Ask a Question

...