微信公众号搜"智元新知"关注
微信扫一扫可直接关注哦!

Google CSE“搜索结果数量”刮刀:请求限制|优化脚本 相关:

如何解决Google CSE“搜索结果数量”刮刀:请求限制|优化脚本 相关:

我想要达到的目标: 我使用带有 Google Apps 脚本的 Google 电子表格来收集某些域和主机的索引页面数量。因此,我将 Google 自定义搜索引擎 (CSE) 称为“site:domain”请求。从 CSE 结果中仅提取“总结果”部分并将其保存到 Google 电子表格中。

var result = JSON.parse(response.getContentText());
    Logger.log( "Obtained %s search results in %s seconds.",result.searchinformation.formattedTotalResults,result.searchinformation.formattedSearchTime);
     
      aresult = JSON.stringify(result).match(/"totalResults":"(.+?)"/)[1].trim();
      return aresult;

问题:我很快就达到了每日请求限制 - 因为我必须为每个请求调用太多数据。

我的问题:有什么方法可以让脚本更高效?我只需要域或主机的搜索结果数量

/**
 * Use Google's customsearch API to perform a search query.
 * See https://developers.google.com/custom-search/json-api/v1/using_rest.
 *
 * @param {string} query   Search query to perform,e.g. "test"
 *
 * returns {object}        See response data structure at
 *                         https://developers.google.com/custom-search/json-api/v1/reference/cse/list#response
 */

function mysearchFor( query ) {
  
  // Base URL to access customsearch
  var urlTemplate = "https://www.googleapis.com/customsearch/v1?key=%KEY%&cx=%CX%&q=%Q%";

  // initialize sheets: 1. Get the spreadsheet,2. Get the first and the second sheets in this spreadsheet
  var spreadSheet = SpreadsheetApp.getActiveSpreadsheet();
  var inputSheet = spreadSheet.getSheets()[0];
  var outputSheet = spreadSheet.getSheets()[1];

  // get the last non-empty row number in the input sheet 
  var inputLastRow = inputSheet.getLastRow();
  // get the first empty row number in the output sheet
  var outputLastRow = outputSheet.getLastRow() + 1;

  // Script-specific credentials & search engine
  var GS_Apikey = inputSheet.getRange('B2').getValue();
  var searchEngineID = inputSheet.getRange('C2').getValue();

  // Build custom url
  var url = urlTemplate
    .replace("%KEY%",encodeURIComponent(GS_Apikey))
    .replace("%CX%",encodeURIComponent(searchEngineID))
    .replace("%Q%",encodeURIComponent(query));
  
  var params = {
    muteHttpExceptions: true
  };

  // Perform search
  Logger.log( UrlFetchApp.getRequest(url,params) );  // Log query to be sent
  var response = UrlFetchApp.fetch(url,params);
  var respCode = response.getResponseCode();

  if (respCode !== 200) {
    throw new Error ("Error " +respCode + " " + response.getContentText());
  }
  else {
    // Successful search,log & return results

    var result = JSON.parse(response.getContentText());
    Logger.log( "Obtained %s search results in %s seconds.",result.searchinformation.formattedSearchTime);
      //return result;
      //return (JSON.stringify(myresult));
      aresult = JSON.stringify(result).match(/"totalResults":"(.+?)"/)[1].trim();
      //aresult.match(/"totalResults":"(.+?)"/)[1].trim();
      return aresult;
  }
}

相关:

版权声明:本文内容由互联网用户自发贡献,该文观点与技术仅代表作者本人。本站仅提供信息存储空间服务,不拥有所有权,不承担相关法律责任。如发现本站有涉嫌侵权/违法违规的内容, 请发送邮件至 dio@foxmail.com 举报,一经查实,本站将立刻删除。