TensorFlow训练好的模型以tensorflow原生方式保存成protobuf文件后可以用许多方式部署运行。
例如：通过 tensorflow-js 可以用javascrip脚本加载模型并在浏览器中运行模型。
通过 tensorflow-lite 可以在移动和嵌入式设备上加载并运行TensorFlow模型。
通过 tensorflow-serving 可以加载模型后提供网络接口API服务，通过任意编程语言发送网络请求都可以获取模型预测结果。
通过 tensorFlow for Java接口，可以在Java或者spark(scala)中调用tensorflow模型进行预测。
我们主要介绍tensorflow serving部署模型、使用spark(scala)调用tensorflow模型的方法
〇，tensorflow serving模型部署概述
使用 tensorflow serving 部署模型要完成以下步骤。
可通过以下colab链接测试效果《tf_serving》： https://colab.research.google.com/drive/1vS5LAYJTEn-H0GDb1irzIuyRB8E3eWc8
%tensorflow_version 2.x import tensorflow as tf print(tf.__version__) from tensorflow.keras import *
一，准备protobuf模型文件
我们使用tf.keras 训练一个简单的线性回归模型，并保存成protobuf文件。
import tensorflow as tf models,layers,optimizers ## 样本数量 n = 800 # 生成测试用数据集 X = tf.random.uniform([n,2],minval=-10,maxval=10) w0 = tf.constant([[2.0],[-1.0]]) b0 = tf.constant(3.0) Y = X@w0 + b0 + tf.random.normal([n,1],mean = 0.0,stddev= 2.0) @表示矩阵乘法,增加正态扰动 # 建立模型 tf.keras.backend.clear_session() inputs = layers.Input(shape = (2,),name ="inputs") 设置输入名字为inputs outputs = layers.Dense(1,name = outputs")(inputs) 设置输出名字为outputs linear = models.Model(inputs = inputs,outputs = outputs) linear.summary() # 使用fit方法进行训练 linear.compile(optimizer=rmsprop",loss=msemae"]) linear.fit(X,Y,batch_size = 8,epochs = 100) tf.print(w = ].kernel) tf.b = ].bias) # 将模型保存成pb格式文件 export_path = ./data/linear_model/ version = 1" 后续可以通过版本号进行模型版本迭代与管理 linear.save(export_path+version,save_format=tf")
Model: model" _________________________________________________________________ Layer (type) Output Shape Param # ================================================================= inputs (InputLayer) [(None,2)] 0 outputs (Dense) (None,1) 3 ================================================================= Total params: 3 Trainable params: 3 Non-trainable params: 0 Epoch 1/100 100/100 [==============================] - 0s 2ms/step - loss: 273.0472 - mae: 13.9096 Epoch 2/100 100/100 [==============================] - 0s 2ms/step - loss: 250.0846 - mae: 13.3155 Epoch 3/100 100/100 [==============================] - 0s 2ms/step - loss: 228.0106 - mae: 12.7211 Epoch 4/100 100/100 [==============================] - 0s 2ms/step - loss: 208.5060 - mae: 12.1514 Epoch 5/100 100/100 [==============================] - 0s 2ms/step - loss: 188.6825 - mae: 11.5647 Epoch 6/100 100/100 [==============================] - 0s 2ms/step - loss: 170.6377 - mae: 10.9862 Epoch 7/100 100/100 [==============================] - 0s 2ms/step - loss: 153.1913 - mae: 10.4133 Epoch 8/100 100/100 [==============================] - 0s 2ms/step - loss: 137.3440 - mae: 9.8525 Epoch 9/100 100/100 [==============================] - 0s 2ms/step - loss: 122.1956 - mae: 9.2907 Epoch 10/100 100/100 [==============================] - 0s 2ms/step - loss: 108.5923 - mae: 8.7439 Epoch 11/100 100/100 [==============================] - 0s 2ms/step - loss: 94.8144 - mae: 8.1773 Epoch 12/100 100/100 [==============================] - 0s 2ms/step - loss: 83.0037 - mae: 7.6339 Epoch 13/100 100/100 [==============================] - 0s 2ms/step - loss: 71.8595 - mae: 7.1003 Epoch 14/100 100/100 [==============================] - 0s 2ms/step - loss: 61.8016 - mae: 6.5690 Epoch 15/100 100/100 [==============================] - 0s 2ms/step - loss: 52.5519 - mae: 6.0456 Epoch 16/100 100/100 [==============================] - 0s 2ms/step - loss: 44.4070 - mae: 5.5431 Epoch 17/100 100/100 [==============================] - 0s 2ms/step - loss: 37.0890 - mae: 5.0457 Epoch 18/100 100/100 [==============================] - 0s 2ms/step - loss: 30.6758 - mae: 4.5701 Epoch 19/100 100/100 [==============================] - 0s 2ms/step - loss: 25.1626 - mae: 4.1214 Epoch 20/100 100/100 [==============================] - 0s 2ms/step - loss: 20.3433 - mae: 3.6880 Epoch 21/100 100/100 [==============================] - 0s 2ms/step - loss: 16.3199 - mae: 3.2814 Epoch 22/100 100/100 [==============================] - 0s 2ms/step - loss: 13.1249 - mae: 2.9330 Epoch 23/100 100/100 [==============================] - 0s 2ms/step - loss: 10.4714 - mae: 2.6117 Epoch 24/100 100/100 [==============================] - 0s 2ms/step - loss: 8.5397 - mae: 2.3433 Epoch 25/100 100/100 [==============================] - 0s 2ms/step - loss: 7.0484 - mae: 2.1351 Epoch 26/100 100/100 [==============================] - 0s 2ms/step - loss: 6.1226 - mae: 1.9872 Epoch 27/100 100/100 [==============================] - 0s 2ms/step - loss: 5.6070 - mae: 1.9047 Epoch 28/100 100/100 [==============================] - 0s 2ms/step - loss: 5.2954 - mae: 1.8510 Epoch 29/100 100/100 [==============================] - 0s 2ms/step - loss: 5.0835 - mae: 1.8137 Epoch 30/100 100/100 [==============================] - 0s 2ms/step - loss: 4.9148 - mae: 1.7841 Epoch 31/100 100/100 [==============================] - 0s 2ms/step - loss: 4.7715 - mae: 1.7581 Epoch 32/100 100/100 [==============================] - 0s 2ms/step - loss: 4.6395 - mae: 1.7303 Epoch 33/100 100/100 [==============================] - 0s 2ms/step - loss: 4.5205 - mae: 1.7106 Epoch 34/100 100/100 [==============================] - 0s 1ms/step - loss: 4.4232 - mae: 1.6903 Epoch 35/100 100/100 [==============================] - 0s 2ms/step - loss: 4.3417 - mae: 1.6738 Epoch 36/100 100/100 [==============================] - 0s 1ms/step - loss: 4.2691 - mae: 1.6579 Epoch 37/100 100/100 [==============================] - 0s 2ms/step - loss: 4.2078 - mae: 1.6470 Epoch 38/100 100/100 [==============================] - 0s 2ms/step - loss: 4.1606 - mae: 1.6381 Epoch 39/100 100/100 [==============================] - 0s 2ms/step - loss: 4.1203 - mae: 1.6292 Epoch 40/100 100/100 [==============================] - 0s 2ms/step - loss: 4.0847 - mae: 1.6230 Epoch 41/100 100/100 [==============================] - 0s 2ms/step - loss: 4.0589 - mae: 1.6182 Epoch 42/100 100/100 [==============================] - 0s 2ms/step - loss: 4.0382 - mae: 1.6141 Epoch 43/100 100/100 [==============================] - 0s 2ms/step - loss: 4.0188 - mae: 1.6109 Epoch 44/100 100/100 [==============================] - 0s 2ms/step - loss: 4.0089 - mae: 1.6098 Epoch 45/100 100/100 [==============================] - 0s 2ms/step - loss: 3.9979 - mae: 1.6075 Epoch 46/100 100/100 [==============================] - 0s 2ms/step - loss: 3.9891 - mae: 1.6055 Epoch 47/100 100/100 [==============================] - 0s 2ms/step - loss: 3.9848 - mae: 1.6053 Epoch 48/100 100/100 [==============================] - 0s 2ms/step - loss: 3.9806 - mae: 1.6044 Epoch 49/100 100/100 [==============================] - 0s 2ms/step - loss: 3.9752 - mae: 1.6037 Epoch 50/100 100/100 [==============================] - 0s 2ms/step - loss: 3.9739 - mae: 1.6038 Epoch 51/100 100/100 [==============================] - 0s 2ms/step - loss: 3.9712 - mae: 1.6024 Epoch 52/100 100/100 [==============================] - 0s 2ms/step - loss: 3.9690 - mae: 1.6024 Epoch 53/100 100/100 [==============================] - 0s 2ms/step - loss: 3.9685 - mae: 1.6021 Epoch 54/100 100/100 [==============================] - 0s 2ms/step - loss: 3.9667 - mae: 1.6021 Epoch 55/100 100/100 [==============================] - 0s 2ms/step - loss: 3.9651 - mae: 1.6009 Epoch 56/100 100/100 [==============================] - 0s 2ms/step - loss: 3.9656 - mae: 1.6019 Epoch 57/100 100/100 [==============================] - 0s 2ms/step - loss: 3.9635 - mae: 1.6016 Epoch 58/100 100/100 [==============================] - 0s 2ms/step - loss: 3.9640 - mae: 1.6012 Epoch 59/100 100/100 [==============================] - 0s 2ms/step - loss: 3.9655 - mae: 1.6018 Epoch 60/100 100/100 [==============================] - 0s 2ms/step - loss: 3.9639 - mae: 1.6016 Epoch 61/100 100/100 [==============================] - 0s 2ms/step - loss: 3.9650 - mae: 1.6010 Epoch 62/100 100/100 [==============================] - 0s 2ms/step - loss: 3.9651 - mae: 1.6017 Epoch 63/100 100/100 [==============================] - 0s 2ms/step - loss: 3.9646 - mae: 1.6021 Epoch 64/100 100/100 [==============================] - 0s 2ms/step - loss: 3.9638 - mae: 1.6019 Epoch 65/100 100/100 [==============================] - 0s 2ms/step - loss: 3.9639 - mae: 1.6027 Epoch 66/100 100/100 [==============================] - 0s 2ms/step - loss: 3.9656 - mae: 1.6013 Epoch 67/100 100/100 [==============================] - 0s 2ms/step - loss: 3.9645 - mae: 1.6019 Epoch 68/100 100/100 [==============================] - 0s 2ms/step - loss: 3.9635 - mae: 1.6024 Epoch 69/100 100/100 [==============================] - 0s 2ms/step - loss: 3.9637 - mae: 1.6015 Epoch 70/100 100/100 [==============================] - 0s 2ms/step - loss: 3.9643 - mae: 1.6022 Epoch 71/100 100/100 [==============================] - 0s 2ms/step - loss: 3.9626 - mae: 1.6022 Epoch 72/100 100/100 [==============================] - 0s 2ms/step - loss: 3.9632 - mae: 1.6015 Epoch 73/100 100/100 [==============================] - 0s 2ms/step - loss: 3.9638 - mae: 1.6023 Epoch 74/100 100/100 [==============================] - 0s 2ms/step - loss: 3.9643 - mae: 1.6017 Epoch 75/100 100/100 [==============================] - 0s 2ms/step - loss: 3.9638 - mae: 1.6003 Epoch 76/100 100/100 [==============================] - 0s 2ms/step - loss: 3.9648 - mae: 1.6022 Epoch 77/100 100/100 [==============================] - 0s 2ms/step - loss: 3.9624 - mae: 1.6023 Epoch 78/100 100/100 [==============================] - 0s 2ms/step - loss: 3.9637 - mae: 1.6019 Epoch 79/100 100/100 [==============================] - 0s 2ms/step - loss: 3.9644 - mae: 1.6019 Epoch 80/100 100/100 [==============================] - 0s 2ms/step - loss: 3.9648 - mae: 1.6018 Epoch 81/100 100/100 [==============================] - 0s 2ms/step - loss: 3.9649 - mae: 1.6025 Epoch 82/100 100/100 [==============================] - 0s 2ms/step - loss: 3.9631 - mae: 1.6021 Epoch 83/100 100/100 [==============================] - 0s 2ms/step - loss: 3.9650 - mae: 1.6020 Epoch 84/100 100/100 [==============================] - 0s 2ms/step - loss: 3.9640 - mae: 1.6020 Epoch 85/100 100/100 [==============================] - 0s 2ms/step - loss: 3.9644 - mae: 1.6014 Epoch 86/100 100/100 [==============================] - 0s 2ms/step - loss: 3.9638 - mae: 1.6017 Epoch 87/100 100/100 [==============================] - 0s 2ms/step - loss: 3.9638 - mae: 1.6024 Epoch 88/100 100/100 [==============================] - 0s 2ms/step - loss: 3.9646 - mae: 1.6016 Epoch 89/100 100/100 [==============================] - 0s 2ms/step - loss: 3.9643 - mae: 1.6016 Epoch 90/100 100/100 [==============================] - 0s 2ms/step - loss: 3.9636 - mae: 1.6019 Epoch 91/100 100/100 [==============================] - 0s 2ms/step - loss: 3.9637 - mae: 1.6029 Epoch 92/100 100/100 [==============================] - 0s 2ms/step - loss: 3.9644 - mae: 1.6026 Epoch 93/100 100/100 [==============================] - 0s 2ms/step - loss: 3.9637 - mae: 1.6014 Epoch 94/100 100/100 [==============================] - 0s 2ms/step - loss: 3.9623 - mae: 1.6019 Epoch 95/100 100/100 [==============================] - 0s 2ms/step - loss: 3.9637 - mae: 1.6015 Epoch 96/100 100/100 [==============================] - 0s 2ms/step - loss: 3.9641 - mae: 1.6017 Epoch 97/100 100/100 [==============================] - 0s 2ms/step - loss: 3.9635 - mae: 1.6027 Epoch 98/100 100/100 [==============================] - 0s 2ms/step - loss: 3.9644 - mae: 1.6024 Epoch 99/100 100/100 [==============================] - 0s 2ms/step - loss: 3.9640 - mae: 1.6021 Epoch 100/100 100/100 [==============================] - 0s 2ms/step - loss: 3.9638 - mae: 1.6024 w = [[1.99997306] [-1.01220131]] b = [2.88236618] WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/resource_variable_ops.py:1817: calling BaseResourceVariable.__init__ (from tensorflow.python.ops.resource_variable_ops) with constraint is deprecated and will be removed in a future version. Instructions for updating: If using Keras pass *_constraint arguments to layers. INFO:tensorflow:Assets written to: ./data/linear_model/1/assets
查看保存的模型文件
!ls {export_path+version}
assets saved_model.pb variables
查看模型文件相关信息
!saved_model_cli show --dir {export_path+str(version)} --all
MetaGraphDef with tag-set: 'serve' contains the following SignatureDefs: signature_def[__saved_model_init_op]: The given SavedModel SignatureDef contains the following input(s): The given SavedModel SignatureDef contains the following output(s): outputs[] tensor_info: dtype: DT_INVALID shape: unkNown_rank name: NoOp Method name is: signature_def[serving_default]: The given SavedModel SignatureDef contains the following input(s): inputs[] tensor_info: dtype: DT_FLOAT shape: (-1,2) name: serving_default_inputs:0 The given SavedModel SignatureDef contains the following output(s): outputs[) name: StatefulPartitionedCall:0 Method name is: tensorflow/serving/predict WARNING: Logging before flag parsing goes to stderr. W0413 05:10:30.262132 140384690243456 deprecation.py:506] From /usr/local/lib/python2.7/dist-packages/tensorflow_core/python/ops/resource_variable_ops.py:1786: calling _constraint arguments to layers. Defined Functions: Function Name: __call__ Option 1 Callable with: Argument 1 inputs: TensorSpec(shape=(None,2),dtype=tf.float32,name=u) Argument 2 DType: bool Value: False Argument 3 DType: nonetype Value: None Option DType: bool Value: True Argument DType: nonetype Value: None Function Name: _default_save_signature) Function Name: call_and_return_all_conditional_losses DType: nonetype Value: None
二，安装 tensorflow serving
安装 tensorflow serving 有2种主要方法：通过Docker镜像安装，通过apt安装。
通过Docker镜像安装是最简单，最直接的方法，推荐采用。
Docker可以理解成一种容器，其上面可以给各种不同的程序提供独立的运行环境。
一般业务中用到tensorflow的企业都会有运维同学通过Docker 搭建 tensorflow serving.
无需算法工程师同学动手安装，以下安装过程仅供参考。
Windows: https://www.runoob.com/docker/windows-docker-install.html
MacOs: https://www.runoob.com/docker/macos-docker-install.html
CentOS: https://www.runoob.com/docker/centos-docker-install.html
安装Docker成功后，使用如下命令加载 tensorflow/serving 镜像到Docker中
docker pull tensorflow/serving
三，启动 tensorflow serving 服务
!docker run -t --rm -p 8501:8501 \ -v /Users/.../data/linear_model/ \ -e MODEL_NAME=linear_model \ tensorflow/serving & >server.log 2>&1
四，向API服务发送请求
可以使用任何编程语言的http功能发送请求，下面示范linux的 curl 命令发送请求，以及Python的requests库发送请求。
!curl -d {"instances": [1.0,2.0,5.0]} \ -X POST http://localhost:8501/v1/models/linear_model:predict { predictions": [[3.06546211],[5.01313448] ] }
json,requests data = json.dumps({signature_name": instances": [[1.0,2.0],[5.0,7.0]]}) headers = {content-typeapplication/json} json_response = requests.post(http://localhost:8501/v1/models/linear_model:predict,data=data,headers=headers) predictions = json.loads(json_response.text)[] print(predictions)
参考：
开源电子书地址：https://lyhue1991.github.io/eat_tensorflow2_in_30_days/
GitHub 项目地址：https://github.com/lyhue1991/eat_tensorflow2_in_30_days
版权声明：本文内容由互联网用户自发贡献，该文观点与技术仅代表作者本人。本站仅提供信息存储空间服务，不拥有所有权，不承担相关法律责任。如发现本站有涉嫌侵权/违法违规的内容， 请发送邮件至 dio@foxmail.com 举报，一经查实，本站将立刻删除。
