The document summarizes MLOps using Protobuf in Unity for a 3D game called FunAI. It discusses using Unity and MLAgents to build a learning environment, training models in Python and playing them in a Unity docker container. The key steps are: 1. Building a Unity environment with MLAgents to get observations from sensors and take actions through behaviors. 2. Recording data from the Unity environment and using it to train models in Python. 3. Serializing the data with Protobuf for efficient communication between Python and Unity via gRPC. 4. Dockerizing the training process and playing trained models to deploy the MLOps pipeline.

1. MLOps using Protobuf in Unity
Ivan Chiou
2022.12.03

2. Outline
• What is MLOps
• What is Protobuf
• PAIA 3D game – FunAI
• Training model in Unity
• Play AI+Model in docker container
• Architecture of PAIA MLOps
• Deploy to cloud service
• Q&A

3. MLOps
圖片取自CD.Fundation
From https://www.twblogs.net/a/5c016050bd9eee7aed33b95d
Machine Learning(ML)
Cloud
Computing

4. Protobuf(protocol buffers)
• Protocol buffers are Google‘s language-neutral, platform-
neutral, extensible mechanism for serializing structured data –
think XML, but smaller, faster, and simpler. You define how you
want your data to be structured once, then you can use special
generated source code to easily write and read your structured
data to and from a variety of data streams and using a variety of
languages. (from Google https://developers.google.com)
message Person {
optional string name = 1;
optional int32 id = 2;
optional string email = 3;
}
message Action {
optional string api_version = 1;
optional string id = 2;
optional Command command = 3;
optional bool acceleration = 4;
optional bool brake = 5;
optional float steering = 6;
}

5. https://www.paia-arena.com/
程式積木 ML積木 Python
機器學習 AI 線上遊戲平台
What is PAIA
• Machine learning AI online game platform
• Learn AI, ML without any background knowledge
• Learn for fun in game
• Learning AI anywhere

6. 3D Game - FunAI https://funai.paia-arena.com/

7. Unity Environment
• Unity is a cross-platform 3D game engine developed by Unity Technologies
• Source code

8. FunAI - Learning Environment
● ML-Agents

9. FunAI – Process Diagram

10. Unity Environment – MLAgents (v2.0.0)
• What is Agent?
• Agent is a main instance to interact with the environment which
contains sensors, behaviors, and the functionality to receive rewards
Agent Class

11. Unity Environment - Sensors
• Use SensorComponents through Isensor interface

12. Unity Environment - Behaviors
• Behavior is like the brain of agent.
• It is the object we will train
• Observation → Sensor
• Action → Behavior

13. Unity Environment
• Observation is the obtained data through
the agent’s sensors, and Action is
continuous steps via behavior(brain) and it
will command agent execute these steps.

14. Unity Environment
• Demonstration Recorder saves the ML-Agents’ raw data
including observations, actions, and rewards into the .demo
file. Python code will read these data to train model.

15. Python → Unity
• requirements.txt
• mlagents==0.26.0
• torch
• numpy
• Pillow
• opencv-python
• paramiko
• ffmpeg-python
• python-dotenv
• from mlagents_envs.environment import UnityEnvironment
• UnityEnvironment is the communication channel between Python and
Unity ML-Agents

16. Python → Unity
• Training: python3 ml.py
• Contest
• set “RUNNING_MODE=GAME”, or execute “python ml.py game”
• Launch Unity
• from mlagents_envs import env_utils
• import unity
• unity_app = unity.get_unity_app()
• env_utils.launch_executable(unity_app, args=[]).wait()
• env = UnityEnvironment(file_name=unity_app)
• behavior_names = list(env.behavior_specs)
• ObservationSpec
• ActionSpec

17. MLAgents to Protobuf
• Unity ML-Agents Protobuf Definitions
• PAIA.proto
enum Event { // 事件
EVENT_NONE; // 一般狀態
EVENT_FINISH; // 結束（其他狀況）
EVENT_RESTART; // 重新開始回合
EVENT_WIN; // 結束（有在時限內完成）
EVENT_TIMEOUT; // 超時
EVENT_UNDRIVABLE; // 不能動了（用完油料或輪胎）
}
message State {
message Observation {
message Ray {
optional bool hit = 1;
optional float distance = 2;
}
message Image {
optional bytes data = 1;
optional int32 height = 2;
optional int32 width = 3;
optional int32 channels = 4;
}
struct Action { // 動作資訊
string api_version; // API 版本
string id; // 使用者名稱
Command command; // 動作指令
bool acceleration; // 是否加速
bool brake; // 是否減速
float steering; // 轉彎
}
'Image' : _reflection.GeneratedProtocolMessageType('Image',
(_message.Message,), {
'DESCRIPTOR' : _STATE_OBSERVATION_IMAGE,
'__module__' : 'PAIA_pb2'
# @@protoc_insertion_point(class_scope:State.Observation.Image)
})

18. Send Protobuf message by gRPC
• Python → Unity (ML-Agents) communicate via gRPC
• gRPC is a cross-platform open source high performance Remote
Procedure Call framework. gRPC was initially created by Google,
which has used a single general-purpose RPC infrastructure called
Stubby to connect the large number of microservices running within
and across its data centers for over a decade
server = grpc.server(futures.ThreadPoolExecutor(max_workers=10))
PAIA_pb2_grpc.add_PAIAServicer_to_server(PAIAServicer(), server)
port = int_ENV('PAIA_ID', PAIA_ID)
server.add_insecure_port(f'[::]:{port}')
server.start()
server.wait_for_termination()
def add_PAIAServicer_to_server(servicer, server):
rpc_method_handlers = {
'hook': grpc.unary_unary_rpc_method_handler(
servicer.hook,
request_deserializer=PAIA__pb2.Action.FromString,
response_serializer=PAIA__pb2.State.SerializeToString,
),
}
generic_handler = grpc.method_handlers_generic_handler(
'PAIA', rpc_method_handlers)
server.add_generic_rpc_handlers((generic_handler,))

19. Training Model
• python3 /home/PAIA/define_epoch.py $repeat ; ./training.sh
• training.sh - DISPLAY=:0 python3 ml.py (headless server)

20. Trained Model
• Pickle file
Best mean reward updated 0.044 -> 0.083, model saved
Removed player: kart
Game Finished
Epispde: 2, Epsilon: 0.9758, Progress: 0.122
Best mean reward updated 0.083 -> 0.096, model saved
Finishing ...
brain = ml_play[0]
action = brain.decision(state)
if action.command == PAIA.Command.COMMAND_RESTART:
autosave(brain, pickle_path, True)

21. Dockerize
• docker build -f Dockerfile.base -t base .
• echo "Base Image Created!!"
• docker build -f Dockerfile.hub -t unity-funai .
• echo "Target Image Created!!“
• docker run -d --name $NAME -v /local_path:/container_path -it
unity-funai:latest /bin/bash -c "python3
/home/PAIA/define_epoch.py $MAX_EPOCH ; ./training.sh"

22. Play AI + pre-trained model
• docker run -d --name funai -v
/home/$username/autosave:/home/PAIA/model_storage -it
unity-funai:latest /bin/bash -c “./play.sh“
• play.sh - python3 ml.py game

23. PAIA MLOps
Architecture

24. MLOps Deployment Architecture
kubectl set image -f manifests/deploy/funai-deploy.yaml
${IMAGE_NAME}=${REGISTRY}/${IMAGE_NAME}:latest --local -o yaml | kubectl apply -f -

25. Unity Deployment Architecture
• GPU driver in docker
docker run -it
-e DISPLAY=:0
--volume="/tmp/.X11-unix:/tmp/.X11-unix:rw"
--runtime=nvidia
--entrypoint /bin/bash
ml-agents-visobs-gpu

26. Unity Deployment Architecture
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
name: unity-thing
name: unity-deployment
spec:
replicas: 1
selector:
matchLabels:
name: unity-thing
template:
metadata:
labels:
name: unity-thing
spec:
containers:
- image: $(REGISTRY)/unity-funai:latest
command: ["/bin/bash", "-c"]
args: [“python3 /home/PAIA/define_epoch.py 2 ; ./training.sh"]
imagePullPolicy: Always
name: unity-funai
env:
- name: POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
envFrom:
- configMapRef:
name: thing-config
- secretRef:
name: configure-secret
volumeMounts:
- mountPath: "/iot-things"
name: thing-pv
subPathExpr: $(POD_NAME)
volumes:
- name: thing-pv
emptyDir: {}

27. Continuous Deployment
stages {
stage('Deploy Image') {
steps{
script {
docker.withRegistry("https://${registry}", registryCredential ) {
sh("docker pull ${registry}/${image_name}:${version}")
sh("docker tag ${registry}/${image_name}:${version} ${image_name}:${version}")
}
}
}
}

28. Continuous Deployment
stage('Apply k8s image in cluster') {
steps {
sh "rm -r -f ${repo_name}"
sh "git clone -b ${BRANCH_NAME} https://${GITLAB_API_TOKEN}@gitlab.com/paia-digital-twin/${repo_name}.git"
sh "cd ${repo_name} && git pull origin ${BRANCH_NAME}"
sh 'kubectl get nodes'
sh 'kubectl config set-context --current --namespace=default'
sh 'kubectl get pods'
sh "cd ${repo_name} && sed -i 's/$DEPLOY_NAME/'${DEPLOY_NAME}'/g' ./manifests/deploy/${image_name}-deploy.yaml"
sh "cd ${repo_name} && kubectl set image -f manifests/deploy/${image_name}-deploy.yaml
${image_name}=${registry}/${image_name}:${version} --local -o yaml | kubectl apply -f -"
}
}
}
}

29. Continuous Deployment

30. Trained Model Upload
• Upload PAIA code + trained model to database
• curl -H "Content-Type: application/json"
• -H "Authorization: Bearer ${TOKEN}"
• -X POST -d
• {"name":"$datatime","desc":"","type":"PY","files":["$
FILES_AI","$FILES_MODEL"]}' https://backend.paia-
arena.com/api/v1/game/${GAME_ID}/code

31. Create recorded video

32. FunAI – Process Diagram

33. Conclusion
• PAIA isolates each microservice including Unity, Python,
protobuf, and its database to containerizes their processes of
AI simulation, training and competition. It makes developers
maintain and find bugs of PAIA system easily.
• Explain how PAIA implement MLOps architecture.
• PAIA continue to cultivate ML and AI programming
competition on children. It will let every students in rural area
have equal possibility to learn computers and programming.

34. Program the world
• Prof. 蘇文鈺 https://programtheworld.tw/

35. PAIA Position Opening
• Frontend Engineer
• Backend Engineer
service@paia-tech.com
JOIN US!
公司成立時間
團隊組成
合作夥伴
帕亞科技成立於 2022年 01 月。
團隊來自國立成功大學與中
華民國愛自造者學習協會,
本團隊主要研發人員有5年
以上軟體開發經驗,擅長機器
學習AI程式開發與程式教育。
合作夥伴包含國立成功大學
資訊工程系、中華民國愛自
造者學習協會、忘言科技、
好想工作室等。

36. Q & A