如何解决如何使用Java API将Avro文件中的非可选值仅插入BigQuery表中
这是我的Avro模式:
{
"namespace": "example.gcp","type": "record","name": "Client","fields": [{
"name": "id","type": "long"
},{
"name": "name","type": "string"
},{
"name": "phone","type": ["string","null"]
},{
"name": "address","null"]
}
]
}
部分重做了GitHub上GCP BigQuery示例的代码,该代码将Avro文件(存储在Google Cloud Storage中)的所有字段加载到表(BigQuery)中。
public static ModelAndView loadAvroFromGCS(String datasetName,String tableName,String sourceUri) {
String message;
try {
BigQuery bigquery = BigQueryOptions.getDefaultInstance().getService();
TableId tableId = TableId.of(datasetName,tableName);
LoadJobConfiguration loadConfig =
LoadJobConfiguration.of(tableId,sourceUri,FormatOptions.avro());
Job job = bigquery.create(JobInfo.of(loadConfig));
job = job.waitFor();
message = "Successfully loaded data file to " + tableName;
} catch (BigQueryException | InterruptedException e) {
message = "Error: " + e.getMessage();
}
return new ModelAndView("index")
.addObject("datasetName",datasetName)
.addObject("message",message);
}
问题是如何更改代码以将("id","name")
中的非可选字段table_1
和所有字段("id","name","phone","address")
保存到一个数据集中的table_2
中?
版权声明:本文内容由互联网用户自发贡献,该文观点与技术仅代表作者本人。本站仅提供信息存储空间服务,不拥有所有权,不承担相关法律责任。如发现本站有涉嫌侵权/违法违规的内容, 请发送邮件至 dio@foxmail.com 举报,一经查实,本站将立刻删除。