我正在使用pandas和read_csv方法加载一个非常庞大的csv文件,如1000万条记录,我想知道是否有办法显示该加载的进度,如:
100,000 lines read
150,000 lines read
谢谢.
最佳答案
要显示这样的进度:
Completed 1 %
Completed 2 %
...
Completed 99 %
Completed 100 %
你可以试试这个:
import os,pandas
filename = "VeryLong.csv"
lines_number = sum(1 for line in open(filename))
lines_in_chunk = 500 # I don't kNow what size is better
counter = 0
completed = 0
reader = pandas.read_csv(filename,chunksize=lines_in_chunk)
for chunk in reader:
# < ... reading the chunk somehow... >
# showing progress:
counter += lines_in_chunk
new_completed = int(round(float(counter)/lines_number * 100))
if (new_completed > completed):
completed = new_completed
print "Completed",completed,"%"
版权声明:本文内容由互联网用户自发贡献,该文观点与技术仅代表作者本人。本站仅提供信息存储空间服务,不拥有所有权,不承担相关法律责任。如发现本站有涉嫌侵权/违法违规的内容, 请发送邮件至 dio@foxmail.com 举报,一经查实,本站将立刻删除。