# Tensorflow model server

## Setup server
Prepare:
```
apt install gnupg2
```
Add repo:
```
echo "deb [arch=amd64] http://storage.googleapis.com/tensorflow-serving-apt stable tensorflow-model-server tensorflow-model-server-universal" | tee /etc/apt/sources.list.d/tensorflow-serving.list && \
curl https://storage.googleapis.com/tensorflow-serving-apt/tensorflow-serving.release.pub.gpg | apt-key add -
```
Install package:
```
apt update
apt install tensorflow-model-server
```

## Setup models
```
mkdir -p /srv/tensorflow/workdir
mkdir -p /srv/tensorflow/models
```
Choose models from [tfhub.dev](https://tfhub.dev/) and for each do:
```
# example: https://tfhub.dev/google/universal-sentence-encoder-multilingual/3
mkdir -p /srv/tensorflow/models/universal-sentence-encoder-multilingual/3
cd /srv/tensorflow/models/universal-sentence-encoder-multilingual/3
wget https://tfhub.dev/google/universal-sentence-encoder-multilingual/3?tf-hub-format=compressed
tar xvfz universal-sentence-encoder-multilingual_3.tar.gz
rm universal-sentence-encoder-multilingual_3.tar.gz
```

Check:
```
tensorflow_model_server --rest_api_port=9000 --model_base_path="/srv/tensorflow/models/universal-sentence-encoder-multilingual/" --model_name=sentences
```

Config file `/srv/tensorflow/config`:
```
model_config_list: {
  config: {
    name: "sentences",
    base_path: "/srv/tensorflow/models/universal-sentence-encoder-multilingual",
    model_platform: "tensorflow"
    model_version_policy: {latest{}},
  },
  config: {
    ... (next model)
  },
}
```

## Systemd integration
Edit /etc/systemd/system/tensorflow.service
```
[Unit]
Description=tensorflow model server
After=network.target auditd.service

[Service]
Type=simple
WorkingDirectory=/srv/tensorflow/workdir
ExecStart=/usr/bin/tensorflow_model_server --rest_api_port=9000 --model_config_file=/srv/tensorflow/config
KillMode=process
Restart=on-failure
RestartSec=30s

[Install]
WantedBy=multi-user.target
```
and
```
systemctl daemon-reload
systemctl enable tensorflow
systemctl start tensorflow
```

Check:
```
http -j GET http://localhost:9000/v1/models/sentences
```

## Usage
Show model details:
```
http -j GET http://localhost:9000/v1/models/sentences/metadata
```

## Docs

* `/usr/bin/tensorflow_model_server --help`
* https://github.com/tensorflow/serving/
* [REST API](https://www.tensorflow.org/tfx/serving/api_rest)
* https://github.com/hey-car/tensorflow-model-server

Datasets:
* https://www.tensorflow.org/datasets/catalog/overview