如何解决Node.js web Scraper 多次运行功能我只希望它运行一次
我正在尝试将 ${car}、${link}、${price} 写入 csv 文件。我当前的代码是这样做的,但它多次运行我的函数,最终在我的 csv 文件中分别出现了 ${car}、${link} 和 ${price} 5 次,我只想要它一次。任何帮助将非常感激!提前致谢!
const request = require('request');
const cheerio = require('cheerio');
const fs = require('fs');
const writeStream = fs.createWriteStream('post.csv');
//Write Headers
writeStream.write(`Title,Link,Price \n`)
request('https://bringatrailer.com/bmw/e46/?q=e46',(error,response,html) => {
if(!error && response.statusCode == 200) {
const $ = cheerio.load(html);
//Grabs the title and link of the featured e46 listings off of bring a trailer
$('.featured-listing-title-link').each((a,title) => {
const car = $(title).text();
const link = $(title).attr('href');
//Grabs the prices of the featured e46 listings off of bring a trailer
$('.featured-listing-meta-value').each((i,value) => {
const price = $(value).text().replace(/,/,"");
writeStream.write(`${car},${link},${price} \n`);
});
});
//Write to CSV
console.log('Scraping Complete...')
}
});
仅供参考,这是我希望我的 csv 文件的样子
解决方法
根据这个特定网站的 DOM 结构,这对我来说是有效的。
const request = require("request");
const cheerio = require("cheerio");
const fs = require("fs");
const writeStream = fs.createWriteStream("post.csv");
writeStream.write(`Title,Link,Price \n`);
request("https://bringatrailer.com/bmw/e46/?q=e46",(error,response,html) => {
if (!error && response.statusCode == 200) {
const $ = cheerio.load(html);
var counter = 0;
$(".featured-listing-title-link").each((a,title) => {
const car = $(title).text();
const link = $(title).attr("href");
const pricetag = cheerio.load($(".featured-listing-meta-value")[counter]);
const price = pricetag.text();
//Grabs the prices of the featured e46 listings off of bring a trailer
writeStream.write(`${car},${link},${price} \n`);
counter += 2;
});
//Write to CSV
console.log("Scraping Complete...");
}
});
版权声明:本文内容由互联网用户自发贡献,该文观点与技术仅代表作者本人。本站仅提供信息存储空间服务,不拥有所有权,不承担相关法律责任。如发现本站有涉嫌侵权/违法违规的内容, 请发送邮件至 dio@foxmail.com 举报,一经查实,本站将立刻删除。