2. Outline
• What is MLOps
• What is Protobuf
• PAIA 3D game – FunAI
• Training model in Unity
• Play AI+Model in docker container
• Architecture of PAIA MLOps
• Deploy to cloud service
• Q&A
4. Protobuf(protocol buffers)
• Protocol buffers are Google‘s language-neutral, platform-
neutral, extensible mechanism for serializing structured data –
think XML, but smaller, faster, and simpler. You define how you
want your data to be structured once, then you can use special
generated source code to easily write and read your structured
data to and from a variety of data streams and using a variety of
languages. (from Google https://developers.google.com)
message Person {
optional string name = 1;
optional int32 id = 2;
optional string email = 3;
}
message Action {
optional string api_version = 1;
optional string id = 2;
optional Command command = 3;
optional bool acceleration = 4;
optional bool brake = 5;
optional float steering = 6;
}
5. https://www.paia-arena.com/
程式積木 ML積木 Python
機器學習 AI 線上遊戲平台
What is PAIA
• Machine learning AI online game platform
• Learn AI, ML without any background knowledge
• Learn for fun in game
• Learning AI anywhere
10. Unity Environment – MLAgents (v2.0.0)
• What is Agent?
• Agent is a main instance to interact with the environment which
contains sensors, behaviors, and the functionality to receive rewards
Agent Class
11. Unity Environment - Sensors
• Use SensorComponents through Isensor interface
12. Unity Environment - Behaviors
• Behavior is like the brain of agent.
• It is the object we will train
• Observation → Sensor
• Action → Behavior
13. Unity Environment
• Observation is the obtained data through
the agent’s sensors, and Action is
continuous steps via behavior(brain) and it
will command agent execute these steps.
14. Unity Environment
• Demonstration Recorder saves the ML-Agents’ raw data
including observations, actions, and rewards into the .demo
file. Python code will read these data to train model.
15. Python → Unity
• requirements.txt
• mlagents==0.26.0
• torch
• numpy
• Pillow
• opencv-python
• paramiko
• ffmpeg-python
• python-dotenv
• from mlagents_envs.environment import UnityEnvironment
• UnityEnvironment is the communication channel between Python and
Unity ML-Agents
18. Send Protobuf message by gRPC
• Python → Unity (ML-Agents) communicate via gRPC
• gRPC is a cross-platform open source high performance Remote
Procedure Call framework. gRPC was initially created by Google,
which has used a single general-purpose RPC infrastructure called
Stubby to connect the large number of microservices running within
and across its data centers for over a decade
server = grpc.server(futures.ThreadPoolExecutor(max_workers=10))
PAIA_pb2_grpc.add_PAIAServicer_to_server(PAIAServicer(), server)
port = int_ENV('PAIA_ID', PAIA_ID)
server.add_insecure_port(f'[::]:{port}')
server.start()
server.wait_for_termination()
def add_PAIAServicer_to_server(servicer, server):
rpc_method_handlers = {
'hook': grpc.unary_unary_rpc_method_handler(
servicer.hook,
request_deserializer=PAIA__pb2.Action.FromString,
response_serializer=PAIA__pb2.State.SerializeToString,
),
}
generic_handler = grpc.method_handlers_generic_handler(
'PAIA', rpc_method_handlers)
server.add_generic_rpc_handlers((generic_handler,))
19. Training Model
• python3 /home/PAIA/define_epoch.py $repeat ; ./training.sh
• training.sh - DISPLAY=:0 python3 ml.py (headless server)
20. Trained Model
• Pickle file
Best mean reward updated 0.044 -> 0.083, model saved
Removed player: kart
Game Finished
Epispde: 2, Epsilon: 0.9758, Progress: 0.122
Best mean reward updated 0.083 -> 0.096, model saved
Finishing ...
brain = ml_play[0]
action = brain.decision(state)
if action.command == PAIA.Command.COMMAND_RESTART:
autosave(brain, pickle_path, True)
22. Play AI + pre-trained model
• docker run -d --name funai -v
/home/$username/autosave:/home/PAIA/model_storage -it
unity-funai:latest /bin/bash -c “./play.sh“
• play.sh - python3 ml.py game
33. Conclusion
• PAIA isolates each microservice including Unity, Python,
protobuf, and its database to containerizes their processes of
AI simulation, training and competition. It makes developers
maintain and find bugs of PAIA system easily.
• Explain how PAIA implement MLOps architecture.
• PAIA continue to cultivate ML and AI programming
competition on children. It will let every students in rural area
have equal possibility to learn computers and programming.