使用 s2i 打包镜像

一、安装

Source-to-Image (S2I) is a toolkit and workflow for building reproducible container images from source code. S2I produces ready-to-run images by injecting source code into a container image and letting the container prepare that source code for execution. By creating self-assembling builder images, you can version and control your build environments exactly like you use container images to version your runtime environments.

s2i官网:https://github.com/openshift/source-to-image

1、下载对应的版本

wget https://github.com/openshift/source-to-image/releases/download/v1.3.1/source-to-image-v1.3.1-a5a77147-linux-amd64.tar.gz

2、解压缩

tar -xvf source-to-image-v1.3.1-a5a77147-linux-amd64.tar.gz

查看:

[root@ks-allinone s2i]# pwd
/root/s2i
[root@ks-allinone s2i]# ls -l
total 24068
-rwxr-xr-x 1 root root 17949824 Oct 10 02:40 s2i
-rw-r--r-- 1 root root  6692560 Oct 10 02:45 source-to-image-v1.3.1-a5a77147-linux-amd64.tar.gz
lrwxrwxrwx 1 root root        3 Oct 10 02:40 sti -> s2i
[root@ks-allinone s2i]# 

3、加入到PATH路径

cd /root/s2i
cp s2i /usr/local/bin

查看

[root@ks-allinone s2i]# cd /usr/local/bin
[root@ks-allinone bin]# ls -l
total 391620
-rwxr-xr-x 1 root root  35861536 Feb 24 01:32 calicoctl
-rwxr-xr-x 1 root root       220 Feb 24 01:40 calicoctl.sh
-rwxr-x--- 1 root root       344 Feb 24 01:39 etcd
-rwxr-xr-x 1 root root  15246720 Mar 30  2018 etcdctl
drwx------ 2 root root      4096 Feb 24 01:39 etcd-scripts
-rwxr-xr-x 1 root root  41819776 Feb 24 01:30 helm
-rwxr-xr-x 1 root root  95601713 Feb 24 01:29 istioctl
-rwxr-xr-x 1 root root  40480768 Feb 24 01:26 kubeadm
-rwxr-xr-x 1 root root  42893312 Feb 24 01:27 kubectl
-rwxr-xr-x 1 root root 111135384 Feb 24 01:31 kubelet
drwxr-xr-x 2 kube root      4096 Feb 24 01:23 kubernetes-scripts
-rwxr-xr-x 1 root root  17949824 Feb 28 11:56 s2i
[root@ks-allinone bin]# 

二、使用

打包方法:

Examples:

# Build a Docker image from a remote Git repository
$ s2i build https://github.com/openshift/ruby-hello-world centos/ruby-22-centos7 hello-world-app

# Build from a local directory.  If this directory is a git repo then the current commit will be built.
$ s2i build . centos/ruby-22-centos7 hello-world-app

示例

$ s2i build https://github.com/sclorg/django-ex centos/python-35-centos7 hello-python

在上述s2i build命令中,我们选定一个builder构建器,其基于镜像centos/python-35-centos7,指定了存放在github仓库中(地址为https://github.com/sclorg/dja...)的源码路径,命令最终为我们生成了可运行的镜像,其为我们隐藏了构建镜像的细节,这些细节与操作交由构建器来实施。

查看打包的镜像:

[root@ks-allinone images]# docker images | grep 'python'
hello-python                                                   latest              23ac6f9bb627        8 minutes ago       675MB
centos/python-35-centos7                                       latest              2db34dda8fd8        23 months ago       645MB

启动镜像:

$ docker run -p 8080:8080 hello-python

三、实战

将模型包上传到服务器,然后使用打包命令对对这个模型包进行镜像打包。

打包镜像

[root@ks-allinone model_packages]# pwd
/root/model_packages
[root@ks-allinone model_packages]# ls -l
total 4
drwxr-xr-x 4 root root 4096 Feb 28 12:33 model_1

# 本地构建
 [root@ks-allinone model_packages]# s2i build ./model_1 centos/python-35-centos7 hello-model
---> Installing application source ...
Build completed successfully
[root@ks-allinone model_packages]# ls -l

查看已经打包的镜像:

[root@ks-allinone model_packages]# docker images | grep 'model'
hello-model                                                    latest              7a91ec13de40        52 seconds ago      648MB
[root@ks-allinone model_packages]# 

You've just built and run a new container image from source code in a git repository, no Dockerfile necessary.

四、打包模型服务

1、镜像拉取

在上边安装好 s2i 后,再下载 seldon-core-s2i 基础镜像:

$ sudo docker pull seldonio/seldon-core-s2i-python3:0.18

2、上传模型包

模型包seldon-core-cll 上传到服务器/home/hemei/model/ 目录下,模型包文件列表:

[hemei@master seldon-core-cll]$ pwd
/home/hemei/model/seldon-core-cll
[hemei@master seldon-core-cll]$ ls -al
total 36
drwxrwxr-x. 3 hemei hemei   109 Feb 26 17:02 .
drwxrwxr-x. 9 hemei hemei   262 Feb 26 09:29 ..
-rw-rw-r--. 1 hemei hemei   873 Feb 25 15:54 finalized_model2.sav
-rw-rw-r--. 1 hemei hemei  1008 Feb 25 16:58 MyModel.py
-rw-rw-r--. 1 hemei hemei 24048 Feb 25 15:54 pima_data.csv
-rw-rw-r--. 1 hemei hemei    36 Feb 25 16:19 requirements.txt

2、模型包添加环境变量

在模型包中文件中添加环境变量文件 .s2i/environment

[hemei@master seldon-core-cll]$ ls -al
total 36
drwxrwxr-x. 3 hemei hemei   109 Feb 26 17:02 .
drwxrwxr-x. 9 hemei hemei   262 Feb 26 09:29 ..
-rw-rw-r--. 1 hemei hemei   873 Feb 25 15:54 finalized_model2.sav
-rw-rw-r--. 1 hemei hemei  1008 Feb 25 16:58 MyModel.py
-rw-rw-r--. 1 hemei hemei 24048 Feb 25 15:54 pima_data.csv
-rw-rw-r--. 1 hemei hemei    36 Feb 25 16:19 requirements.txt
drwxrwxr-x. 2 hemei hemei    25 Feb 26 19:37 .s2i
[hemei@master seldon-core-cll]$ cd .s2i/
[hemei@master .s2i]$ ls -l
total 4
-rw-rw-r--. 1 hemei hemei 66 Feb 25 16:04 environment
[hemei@master .s2i]$ cat environment 
MODEL_NAME=MyModel
API_TYPE=REST
SERVICE_TYPE=MODEL
PERSISTENCE=0
[hemei@master .s2i]$ 

模型包文件展示MyModel.py

from pandas import read_csv
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LogisticRegression
from pandas import read_csv
from pickle import load
class MyModel:
    def __init__(self):
        filename = 'pima_data.csv'
        names = ['preg', 'plas', 'pres', 'skin', 'test', 'mass', 'pedi', 'age', 'class']
        data = read_csv(filename, names=names)
        # 将数据分为输入数据和输出结果
        array = data.values
        X = array[:, 0:8]
        Y = array[:, 8]
        test_size = 0.33
        seed = 4
        X_train, X_test2, Y_traing, Y_test2 = train_test_split(X, Y, test_size=test_size, random_state=seed)
        model_file = 'finalized_model2.sav'
        with open(model_file, 'rb') as model_f:
           self._model = load(model_f)
        self._X_test2 = X_test2
        self._Y_test2 = Y_test2

    def predict(self,X,y,sample_weight=None):
        result = self._model.score(self._X_test2, self._Y_test2)
        return [result]

依赖文件:requirements.txt

scikit-learn==0.24.1
pandas==0.23.4

3、通过s2i打包镜像

sudo /usr/local/bin/s2i build /home/hemei/model/seldon-core-cll seldonio/seldon-core-s2i-python3:0.18  model-app-demo

查看打包好的镜像:

[hemei@master seldon-core-cll]$ sudo docker images | grep 'model'
[sudo] password for hemei: 
model-app-demo                                                         latest                     127a16149b34   52 minutes ago   1.07GB

此时的镜像可通过docker运行,也可做为seldonDepartment的镜像通过k8s部署

4、运行镜像

sudo docker run -p 5001:5000 model-app-demo

5、测试


[hemei@master ~]$ curl -d 'json={"data":{"ndarray":[[1.0,2.0,3.0,4.0]]}}' http://0.0.0.0:5001/predict
{"data":{"names":[],"ndarray":[0.8070866141732284]},"meta":{}}

6、重定向pip源

由于seldon-core-s2i-python3:0.18 中默认的pip源地址是国外的地址,下载缓慢。
这边使用自己重写的s2i 脚本,达到重新定向pip源的目的。
首先拿到基础镜像里面的assemble 脚本,添加以下配置pip源的脚本

file

其次在模型包环境配置的文件中添加源配置
file

最后s2i 创建镜像时指定s2i脚本目录
file

为者常成,行者常至