微信公众号搜"智元新知"关注
微信扫一扫可直接关注哦!

爬图片之线程池

 1 from concurrent.futures import ThreadPoolExecutor
 2 import requests
 3 from lxml import etree
 4 
 5 
 6 class PicApp:
 7     @classmethod
 8     def download_pic(cls, string):
 9         resp = requests.get(string)
10         html = etree.HTML(resp.text)
11         table = html.xpath("/html/body/div[5]/ul")[0]
12         alist = table.xpath("./li/a/@href")
13         for u in alist:
14             resp = requests.get(u)
15             html = etree.HTML(resp.text)
16             a = html.xpath("/html/body/div[3]/div[2]/a")[0]
17             src = a.xpath("./@href")[0]
18             # 下载图片
19             img = requests.get(src)
20             name = src.split("/")[-1]
21             with open("img/" + name, mode="wb") as f:
22                 print("图片下载中------------------")
23                 f.write(img.content)
24                 f.close()
25 
26 
27 if __name__ == '__main__':
28     app = PicApp
29     with ThreadPoolExecutor(50) as t:
30         for i in range(25, 44):
31             url = f"https://www.3gbizhi.com/wallDM/index_{i}.html"
32             t.submit(app.download_pic, url)
33     print("下载完毕!")

 

版权声明:本文内容由互联网用户自发贡献,该文观点与技术仅代表作者本人。本站仅提供信息存储空间服务,不拥有所有权,不承担相关法律责任。如发现本站有涉嫌侵权/违法违规的内容, 请发送邮件至 dio@foxmail.com 举报,一经查实,本站将立刻删除。

相关推荐