微信公众号搜"智元新知"关注
微信扫一扫可直接关注哦!

AWS Glue:从 GET 请求传递 ID 值{1,2,3} 并将其传递给第二个 GET 请求,并将 ID 值附加到响应并转换为数据帧

如何解决AWS Glue:从 GET 请求传递 ID 值{1,2,3} 并将其传递给第二个 GET 请求,并将 ID 值附加到响应并转换为数据帧

First Json Response:
  "data": [
        {
            "id": 123,"first_name": "abc","surname": "efg","email": "abc@gmail.com","full_name": "abc efg"
          },{
            "id": 567,"first_name": "ghj","surname": "klm","email": "ghj@gmail.com","full_name": "ghj klm"
]

skill_response=requests.get("XXXXXX",headers=headersAPI),headers=headersAPI)
skill_json=skill_response.json()

df = spark.read.json(sc.parallelize([skill_json]))

从第一个请求传递 ID 值 (123,567) ->如何将 int 转换为字符串并将其传递给第二个请求(每个) per ={'123','567'}

for per in persons:
    per_response=requests.get("xxxxxxxxxxxxxxxxxxxx/"+per,headers=headersAPI)
    api_results.append(per_response.json())
    api_results.append(per) ## (OR)
    api_results.append({"person_id":per})
    second = spark.read.json(sc.parallelize([api_results]))
    dynamic_frame = DynamicFrame.fromDF(second,glue_ctx=glueContext,name="second")
    DataSink0 = glueContext.write_dynamic_frame.from_options(frame = dynamic_frame,connection_type = "s3",format = "csv",connection_options={"path": "s3://xxxxx/","partitionKeys": []},transformation_ctx= "DataSink0")

我们可以使用for循环传递第一个json值(skill_json["data"][i]["id"]吗?或者使用json.loads()?

版权声明:本文内容由互联网用户自发贡献,该文观点与技术仅代表作者本人。本站仅提供信息存储空间服务,不拥有所有权,不承担相关法律责任。如发现本站有涉嫌侵权/违法违规的内容, 请发送邮件至 dio@foxmail.com 举报,一经查实,本站将立刻删除。