javascript - Node: Downloading a zip through Request, Zip being corrupted

Question

I'm using the excellent Request library for downloading files in Node for a small command line tool I'm working on. Request works perfectly for pulling in a single file, no problems at all, but it's not working for ZIPs.

For example, I'm trying to download the Twitter Bootstrap archive, which is at the URL:

http://twitter.github.com/bootstrap/assets/bootstrap.zip

The relevant part of the code is:

var fileUrl = "http://twitter.github.com/bootstrap/assets/bootstrap.zip";
var output = "bootstrap.zip";
request(fileUrl, function(err, resp, body) {
  if(err) throw err;
  fs.writeFile(output, body, function(err) {
    console.log("file written!");
  }
}

I've tried setting the encoding to "binary" too but no luck. The actual zip is ~74KB, but when downloaded through the above code it's ~134KB and on double clicking in Finder to extract it, I get the error:

Unable to extract "bootstrap" into "nodetest" (Error 21 - Is a directory)

I get the feeling this is an encoding issue but not sure where to go from here.

Answer 1

Yes, the problem is with encoding. When you wait for the whole transfer to finish body is coerced to a string by default. You can tell request to give you a Buffer instead by setting the encoding option to null:

var fileUrl = "http://twitter.github.com/bootstrap/assets/bootstrap.zip";
var output = "bootstrap.zip";
request({url: fileUrl, encoding: null}, function(err, resp, body) {
  if(err) throw err;
  fs.writeFile(output, body, function(err) {
    console.log("file written!");
  });
});

Another more elegant solution is to use pipe() to point the response to a file writable stream:

request('http://twitter.github.com/bootstrap/assets/bootstrap.zip')
  .pipe(fs.createWriteStream('bootstrap.zip'))
  .on('close', function () {
    console.log('File written!');
  });

A one liner always wins :)

pipe() returns the destination stream (the WriteStream in this case), so you can listen to its close event to get notified when the file was written.