如何解决R 循环下载联合国商品贸易统计数据
我正在尝试下载 UN ComTrade 数据集。 当然,这是一个巨大的文件,因此我需要将其拆分为许多文件(我认为将每个记者国家和商品分开),但我仍然找不到合适的解决方案。 然而,我已经尝试了本主题 Writing a loop to extract Comtrade data and export them into multiple csv files 中的建议,这似乎适合我的情况,但没有结果,因为我不断收到以下错误消息:
Error in file(file,"rt") :
cannot open the connection to 'http://comTrade.un.org/api/get?max=50000&type=C&freq=M&px=HS&ps=all&r=American Samoa&p=0&rg=2&cc=842520&fmt=csv'
Inoltre: Warning message:
In file(file,"rt") :
Error in file(file,"rt") :
cannot open the connection to 'http://comTrade.un.org/api/get?max=50000&type=C&freq=M&px=HS&ps=all&r=American Samoa&p=0&rg=2&cc=842520&fmt=csv'
无论如何,以下是我从此刻起遵循的步骤。
## Stefan A's code
get.ComTrade <- function(url="http://comTrade.un.org/api/get?",maxrec=50000,type="C",freq="A",px="HS",ps="Now",r,p,rg="all",cc="TOTAL",fmt="json"
)
{
string<- paste(url,"max=",maxrec,"&" #maximum no. of records returned,"type=",type,"&" #type of Trade (c=commodities),"freq=",freq,"&" #frequency,"px=",px,"&" #classification,"ps=",ps,"&" #time period,"r=","&" #reporting area,"p=","&" #partner country,"rg=",rg,"&" #Trade flow,"cc=",cc,"&" #classification code,"fmt=",fmt #Format,sep = ""
)
if(fmt == "csv") {
raw.data<- read.csv(string,header=TRUE)
return(list(validation=NULL,data=raw.data))
} else {
if(fmt == "json" ) {
raw.data<- fromJSON(file=string)
data<- raw.data$dataset
validation<- unlist(raw.data$validation,recursive=TRUE)
ndata<- NULL
if(length(data)> 0) {
var.names<- names(data[[1]])
data<- as.data.frame(t( sapply(data,rbind)))
ndata<- NULL
for(i in 1:ncol(data)){
data[sapply(data[,i],is.null),i]<- NA
ndata<- cbind(ndata,unlist(data[,i]))
}
ndata<- as.data.frame(ndata)
colnames(ndata)<- var.names
}
return(list(validation=validation,data =ndata))
}
}
}
然后,我从以下位置获得记者名单:
library(rjson)
string <- "http://comTrade.un.org/data/cache/partnerAreas.json"
reporters <- fromJSON(file=string)
reporters <- as.data.frame(t(sapply(reporters$results,rbind)))
因此,我清理了报告者列表,以避免出现无数据行的问题。
ls_reporters <- reporters[ ! reporters$V2 %in% c("Africa CAMEU region,not elsewhere specified","Areas,"CACM,"Caribbean,"Eastern Europe,"Europe EFTA,"Europe EU,"LAIA,"north America,the Caribbean and Central America not elsewhere specified","northern Africa,"Oceania,"Other Africa,"Other Asia,"Other Europe,"Rest of America,"Western Asia,not elsewhere specified"),]
最后,我继续建议的循环。
for (i in 1:length(country_list)){
d <- get.ComTrade(r="i",p="all",ps="2006",rg="2",freq="A")
df <- cbind(d$data[2],d$data[10],d$data[13],d$data[32])
df$million <- as.numeric(as.character(df[,4]))/1000000
myfile <- file.path("~/working directory",paste0("_",i,".csv"))
write.table(df,file=myfile,na = "NA",row.names = FALSE,col.names = FALSE,quote=FALSE,append=FALSE,sep="")
}
版权声明:本文内容由互联网用户自发贡献,该文观点与技术仅代表作者本人。本站仅提供信息存储空间服务,不拥有所有权,不承担相关法律责任。如发现本站有涉嫌侵权/违法违规的内容, 请发送邮件至 dio@foxmail.com 举报,一经查实,本站将立刻删除。