微信公众号搜"智元新知"关注
微信扫一扫可直接关注哦!

如何使用javascript apify puppeteer从谷歌趋势下载Csv

如何解决如何使用javascript apify puppeteer从谷歌趋势下载Csv

我使用该代码使用 apify js 从谷歌趋势下载 csv,但它不起作用,你能帮我吗? 结果是一个内容错误的 csv。 我尝试从谷歌趋势中获取所有 csv。

const path = require('path');
const downloadpath1 = path.resolve(__dirname,'./downloads');
  
const fs = require('fs');
try{
const util = require('util');
await util.promisify(fs.mkdir)(downloadpath1);  
}
  catch( e){


  }
  
await page.setRequestInterception(true); 
//await page.click(csvSelector);
  await page.waitForSelector(csvSelector)
 const hrefElement = await page.$(csvSelector);
  await hrefElement.click();

const xRequest = await new Promise(resolve => {
  page.on('request',async interceptedRequest => {
            const type = interceptedRequest.resourceType();
            log.info(type)
            if (  type == "xhr" ) {interceptedRequest.abort();
             resolve(interceptedRequest);}
            else interceptedRequest.continue();
        });
});
log.info(xRequest._url);
const request = require('request-promise');
const options = {
    encoding: null,method: xRequest._method,uri: xRequest._url,body: xRequest._postData,headers: xRequest._headers
}

/* add the cookies */
const cookies = await page.cookies();
options.headers.Cookie = cookies.map(ck => ck.name + '=' + ck.value).join(';');
const response = await request(options); 

fs.writeFileSync(downloadpath1+'/binary.csv',response);
  

 
const fileObjs = fs.readdirsync(downloadpath1,{ withFileTypes: true });
  
console.log("\nCurrent directory files:");
fileObjs.forEach(file => {
  console.log(file);
});

 
// There won't be more files so let's pick the first
const fileData = fs.readFileSync(downloadpath1+`/${fileObjs[0].name}`);
log.info(fileData);
// Now we can use the data or save it into Key-value store. 

await Apify.setValue('MY-Csv.csv',fileData,{ contentType: 'application/csv'});

希望有人能为此提出解决方案,谢谢

版权声明:本文内容由互联网用户自发贡献,该文观点与技术仅代表作者本人。本站仅提供信息存储空间服务,不拥有所有权,不承担相关法律责任。如发现本站有涉嫌侵权/违法违规的内容, 请发送邮件至 dio@foxmail.com 举报,一经查实,本站将立刻删除。