Stackless Python In Eve

Stackless Python in EVE Kristján Valur Jónsson [email_address] CCP Games inc.

EVE
MMORPG Space game
Client / server
Single shard massive server
130.000 active players, >24.000 concurrent users
World concurrency record on a shard
425,537 lines of Python code and growing.
client: 158,656
server: 218,645
common: 48,236
Possible only because of Stackless Python

The Tranquility cluster
400 GHz CPU / 200 Gb RAM
2 Routers (CISCO Alteon)
14 Proxy servers (IBM Blade)
55 Sol servers (IBM x335)
2 DB servers (clustered, IBM Brick x445)
FastT600 Fiber, 56 x FC 15k disks, DS4300 + 3*EXP700
Windows 2000, MS SQL Server
Currently being upgraded
AMD x64 IBM Blade
RAMSAN gizmo
64 bit binaries

EVE Architecture
COM-like basic architecture
Python tightly integrated at an early stage
Home-grown wrapping of BLUE objects

Stackless in EVE
BLUE foundation: robust, but cumbersome
RAD
Stackless Python: Python and so much more
EVE is inconceivable without Stackless
Everyone is a programmer
Interactive development.
Files reloaded automatically
Classes updated with new methods
Demo:

Stackless Python
Tasklets
Threads of execution. Not OS threads
Lightweight
No pre-emption
No memory fragmentation due to stacks
Channels
Tasklet rendezvous point
Data passing
Scheduling
Synchronization

Stackless?
No C stack
Python stack in linked frame objects
Tasklet switching by swapping frame chain
Compromise
stackless where possible.
C stack whisked away if necessary

Channels

Channel semantics
Send on a channel with no receiver blocks tasklet.
Send on a channel with a (blocked) receiver, suspends tasklet and runs receiver immediately. Sender runs again in due course.
Symmetric behaviour.
“ Balance”, can have a queue of readers or writers.
Conceptually similar to Unix pipes

Channel semantics, cont.
Scheduling semantics are precise:
A blocked tasklet is run immediately
Usable as a building block:
semaphores
mutex
critical section
condition variables

The main loop
Establish stackless context
int WinMain(...) {
PyObject *myApp = new EveApp();
PyObject *r = PyStackless_CallMethod_Main(myApp, “WinMain”, 0);
return PyInt_AsLong( r );

Regular Windows message loop
Runs in Stackless context
The “Main Tasklet”
The main loop cont.
PyObject* EveApp::WinMain(PyObject *self, PyObject *args) {
while(!mExit) {
PyOS->ExecFile("script:/sys/autoexec.py");
MSG msg;
while(PeekMessage(&msg, 0, 0, 0, PM_REMOVE)){
TranslateMessage(&msg);
DispatchMessage(&msg);
}
for (TickIt i = mTickers.begin(; i != mTickers.end(); i++)
i->mCb-> OnTick (mTime, (void*)taskname);
}
}

Autoexec.py import blue def Startup(): import service srvMng = service.ServiceManager() run = ["dataconfig", "godma", “ui", …] srvMng.Run(run) #Start up the client in a tasklet! if CheckDXVersion(): import blue blue.pyos. CreateTasklet (Startup, (), {})

Tickers
Tickers are BLUE modules:
Trinity (the renderer)
Netclient
DB (on the server)
Audio
PyOS (special python services)
…

The PyOS tick:
Runs fresh tasklets
(sleepers awoken elsewhere)
Tick() { … mSynchro->Tick() PyObject *watchdogResult; do { watchdogResult = PyStackless_RunWatchdog(20000000); if (!watchdogResult) PyFlushError("PumpPython::Watchdog"); Py_XDECREF(watchdogresult); } while (!watchdogResult);

blue.pyos.synchro
Synchro:
Provides Thread-like tasklet utilities:
Sleep(ms)
Yield()
BeNice()

Sleep: A python script makes the call blue.pyos.Sleep(200)
C++ code runs:
Main tasklet check
sleeper = New Sleeper(); mSleepers.insert(sleeper); PyObject *r = PyChannel_Receive(sleeper->mChannel);
Another tasklet runs
blue.pyos.synchro cont.

blue.pyos.synchro, ctd.
Main tasklet in windows loop enters PyOS::Tick()
mSleepers are examined for all that are due we do:
mSleepers.remove(sleeper); PyChannel_Send(sleepers.mChannel, Py_NONE);
Main tasklet is suspended (but runnable), sleeper runs.

Points to note:
A tasklet goes to sleep by calling PyChannel_Receive() on a channel which has no pending sender.
It will sleep there (block) until someone sends
Typically the main tasklet does this, doing PyChannel_Send() on a channel with a reader
Ergo: The main tasklet may not block

Socket Receive
Use Windows asynchronous file API
Provide a synchronous python API. A python script calls Read().
Tasklet may be blocked for a long time, (many frames) other tasklets continue running.
Do this using channels.

Receive, cont.
Python script runs: data = socket.Read()
C code executes the request:
Request *r = new Request(this); WSAReceive(mSocket, …); mRequests->insert( r ); PyChannel_Receive(r->mChannel);
Tasklet is blocked

Receive, cont.
Socket server is ticked from main loop
For all requests that are marked completed, it transfers the data to the sleeping tasklets:
PyObject *r = PyString_FromStringAndSize(req->mData, req->mDataLen); PyChannel_Send(req->mChannel, r); Py_DECREF(data); delete req;
The sleeping tasklet wakes up, main tasklet is suspended (but runnable)

Receive completed

Main Tasklet
The one running the windows loop
Can be suspended, allowing other tasklets to run
Can be blocked, as long as there is another tasklet to unblock it (dangerous)
Is responsible for waking up Sleepers, Yielders, IO tasklets, etc. therefore cannot be one of them
Is flagged as non-blockable ( stackless.get_current().block_trap = True )

Channel magic
Channels perform the stackless context switch.
If there is a C stack in the call chain, it will magically swap the stacks.
Your entire C stack (with C and python invocations) is whisked away and stored, to be replaced with a new one.
This allows stackless to simulate cooperative multi-threading

Co-operative multitasking
Context is switched only at known points.
In Stackless, this is channel.send() and channel.receive()
Also synchro.Yield(), synchro.Sleep(), BeNice(), socket and DB ops, etc.
(all boil down to send() and receive() )
No unexpected context switches
Almost no race conditions
Program like you are single-threaded
Very few exceptions.
This extends to C state too!

Tasklets
Tasklets are cheap
70,000 on a server node
Used liberally to reduce perceived lag
UI events forked out to tasklets
A click can have heavy consequences.
Heavy logic
DB Access
Networks access
special rendering tasks forked out to tasklets.
controlling an audio track
“ tasklet it out”
Use blue.pyos.synchro.BeNice() in large loops

Example: UI Event:
Main tasklet receives window messages such as WM_LBUTTONUP
Trinity invokes handler on UI elements or global handler
Handler “tasklets out” any action to allow main thread to continue immediately.
def OnGlobalUp(self, *args): mo = eve.triapp.uilib.mouseOver if mo in self.children: uthread.new(mo._OnClick) class Action(xtriui.QuickDeco): def _OnClick(self, *args): pass

What we don’t understand:
Why isn’t everyone using Stackless Python?

That’s all
For more info:
http://www.ccpgames.com
http://www.eve-online.com
http://www.stackless.com
[email_address]

Stackless Python In Eve

by guest91855c on Aug 14, 2008

Accessibility

Categories

Tags

Upload Details

Usage Rights

3 Embeds 8

Statistics

Stackless Python In Eve — Presentation Transcript

http://www.slideshare.net	6
http://www.techgig.com	1
http://paper.li	1