google apps script - Search for a string using UrlFetchApp when output is truncated

Question

I have a long list of websites where I want to search for a specific string. In this case the string is "cast area". I have come up with this script that looks through the list (I have confined it to the row that I know has a valid output). In the log, it says Logging output too large. Truncating output. I have read online that this shouldn't matter - it's just too big for the log, not that it's given up on looking through the rest.

function getData() {
 var ss = SpreadsheetApp.getActiveSpreadsheet();
 var sheet = ss.getSheetByName('URLs');

 var firstURL = 24673
 var lastURL = 24678

 var i
 for (i=firstURL; i<lastURL; i++) {

   var data = sheet.getRange(i,1).getValue();
   var response = UrlFetchApp.fetch(data).getContentText();

   Logger.log(data)
   //Logger.log(response)

    if (response.toLowerCase().indexOf("cast area")>-1) {
      Logger.log(1)
    } 
  }
}

To test it out, here is the code with the url included that has the words "CAST AREA" in the Notes section of the page. I'm hoping the log should return the number 1 to show that it works.

function getData() {
 var ss = SpreadsheetApp.getActiveSpreadsheet();
 var sheet = ss.getSheetByName('URLs');

 var data = "https://elanthipedia.play.net/Faenella%27s_Grace";
 var response = UrlFetchApp.fetch(data).getContentText();

 Logger.log(data)
 //Logger.log(response)

 if (response.indexOf("CAST AREA")>-1) {
  Logger.log(1)
 }
}

question from:https://stackoverflow.com/questions/65840718/search-for-a-string-using-urlfetchapp-when-output-is-truncated

Answer 1

If you want to visualize a contentText that is too large for the Logging output - write this data somewhere

The simplest solution would be to write it into a spreadsheet.

Sample:

function getData() {
 var ss = SpreadsheetApp.getActiveSpreadsheet();
 var sheet = ss.getSheetByName('URLs');

 var data = "https://elanthipedia.play.net/Faenella%27s_Grace";
 var response = UrlFetchApp.fetch(data).getContentText();
 sheet.getRange(sheet.getLastRow()+1,1).setValue(response);
}

Obviously, to visualize the data in a nicer way it makes sense to parse the responce and write the different objects into different columns of the spreadsheet.