SlideShare a Scribd company logo
1 of 45
Download to read offline
Software Architecture of an MMO
David Salz
david@sandbox-interactive.com
Who am I?
 David Salz
 15 years in the industry
 CTO, Co-Founder of Sandbox Interactive
 35 people
 based in Berlin
 we only do Albion Online!
In this talk:
 What is Albion Online?
 Middleware + Design Decisions
 Server Architecture
 How we use Unity (and how not)
Albion Online
 Sandbox MMORPG
 Cross-Platform (Windows/OSX/Linux/Android/iOS)
 One World (no „Shards“ or „Servers“, not even for different platforms)
 Player-Driven Economy (everything is player-crafted)
 No Character Classes („you are what you wear“)
 Strong focus on PvP + Guilds
 Hardcore („Full Loot“ in PVP Areas)
 Pay-to-play model
 4 years in the making
 Currently in Closed Beta w/ 80.000+ „founding“ players
 „Release“ in Q4/2016
Albion Online
 The initial pitch (2012)
 Play-crafted equipment, buildings
 One World (like EVE Online)
 Guild vs. Guild Territotorial conquest
 Top-Down perspective + Combat (like League of Legends)
 Simple graphics, „can even be 2D“
 PVP focus, no PVE, no Mobs, no Dungeons
 Release in 2013 
Middleware Selection
 Engine Selection
 Unity made sense – inexpensive, accessible
 Cross-Platform was a „target of opportunity“
 Database Selection
 One world  need a very scalable database, NoSQL
 Cassandra sounded suitable
 still use SQL for query-heavy problems
 Networking Middleware
 Photon!
 can use C# on server (like in Unity!)
 works with all Unity platforms
Apache Cassandra
 NoSQL Database
 Originally developed by Facebook
 Open Source (Apache-License)
 Java
 Concept: everything is a hash table
 in-memory (as much as possible)
 optimized for high throughput
 scales horizontally (just add servers)
 redundant (no single point of failure)
 CQL: SQL-like language (w/ massive restrictions – it‘s NoSQL)
Apache Cassandra
 funny story from public Alpha Test…
 all buildings disappeared during lunch break
 found this in Cassandra Changelog:
 W(
 lesson learned: careful with cutting edge technology
* Fix possible loss of 2ndary index entries during
compaction (CASSANDRA-6517)
Apache Cassandra
 Cassandra uses timestamps to resolve
conflicts between database nodes
 also requires exact time sync beween servers
 i.e. ntp service
Hosting
 Hosted with Softlayer
 good, but quite expensive
 bare-metal machines only
 currently: 8x 10-Core Intel Xeon, 32 GB RAM
 Should be able to handle ~15k CCUs
 Originally:
 Hosted across 2 Datacenters (better ping for players)
 Worked, but bad idea! (inter-server disconnects, latency)
Networking
 Photon
 UDP-based protocol (reliable/unreliable as needed)
 TCP for chat, inter-server communication
 use only basic message functionality (no Photon frameworks)
 mostly simple messages, in rare cases JSON content
 Had to implement secure TCP-reconnect abilities
 Chat
 Tried IRC…
 … was only trouble (difficult to secure, customize)
 Implemented own system in a couple of days
Consequences
 Server needs to work without Unity
 Ideally, client works without Unity, too!
 think: tools, stress-test-bots!
 Use Unity only as rendering front-end
 cannot rely on Unity features for basic functions!
 levels / game objects
 collision
 pathfinding
Separation
ObjectFactory ObjectViewFactory
Object
+Position
+…
+RequestAction()
ObjectView
+Renderer
+AnimationCtrl
+HandleInput()
destroyed
changed
…etc..
created
Object
+Position
+…
+RequestAction()
Server Client Unity-Client
Interest-Management
changed
…etc..
obj-enter
obj-leave
Server
Server Farm
Game Server
Game Server
Game Server
Login Server
World Server
Marketplace Server
GoldMarket Server
Statistics Server
Ranking Server
BackOffice Server
Chat Server
Client
Connect to different game
server depending on
location in game world
Game DB
(Cassandra)
Accounts DB
(Postgres)
Market DB
(Postgres)
GoldMarket DB
(Postgres)
 Todo: screenshot of gold market
 Todo: screenshot of marketplace
Game Servers / World Server
 game world split into unique „clusters“ of ~ 1 km²
 current world: ~600, next world: x2
 distributed among game servers
 player changes servers when travelling
 handoff done through database (even on same server)
 Instances are dynamically created
 world server coordinates this
 World Server
 responsible for everything „global“
 guilds, parties, coordination of guild battles etc.
Threading Model (Game Server)
IO ThreadIO ThreadNetwork IO
Thread
Event
Queue
Logging IO
Thread
Event
Scheduler
Cluster Thread
Event
Queue
Event
Scheduler
Cluster Thread
Pathfinding
Thread
Database IO
Thread
Cluster Thread makes
request
Events / Results get
put into Event Queue
Cluster polls Events
from Queue and
Scheduler
Threading Model
 Game logic for one game area („cluster“) is single-threaded
 Interactions between game objects too complicated otherwise; would
require lots of locking
 Objects „sleep“ whenever possible (especially mobs)
 All IO (Network, Database, Logging) and other „heavy lifting“
(pathfinding) offloaded to worker threads
 Instances (Player islands, instanced dungeons) are combined in thread
pools
 Other servers (chat, rankings) are completely multi-threaded
There is no spoon!
 There is no game loop!
 i.e. no Update() or Tick() functions for game objects
 Gameplay thread processes events in order of arrival
 Events are:
 Network input, i.e. player commands
 Timers (Event Scheduler)
 If an object needs a tick (Mobs do), it creates a recurring timer
 Results from DB queries (asynchronous)
 Results from pathfinding
 …
Interest Management
 Players should only „see“ (network-wise) what‘s on their screen
 important to reduce traffic…
 … and prevent cheating!
 Find objects a player can „see“ (and update when things move)
 Objects send messages to all players that „see“ them
 Needs to be efficient!
 ~500 mobs / cluster
 > 10.000 interactive objects (mostly trees )
 up to 300 players
Interest Management
grid based hash (10x10m cells)
● cells contain list of objects inside
● list is updated by object itself when moving
● cells fire events:
● ObjectEnteredCell()
● ObjectLeftCell()player
● objects also fire events
● ObjectMoved()
● ObjectDestroyed()
● EventFired()
Interest Management
player
Interest area
Moves with player
Subscribe to objects when they
enter inner area
Unsubscribe when they leave
outer area
Tell player when:
● an object enters / leaves interest area
● an object in the area fires an event
Logging infrastructure
Game Server
Game Server
Game Server
Log File
Log File
Log File
Logstash
Agent Redis
Elasic Search
Kibana
 track errors, generate statistics
 classic ELK stack (Elastic Search, Logstash, Kibana)
 kind of unreliable for us
 Also use Cassandra Counters a lot!
Logstash
Agent
Logstash
Agent
Own tools
Kibana
Kibana
Client
Level Design
 Zones built in Unity (Unity scenes)
 Consist only of Unity prefabs („tiles“)
 ~ 30.000 / level
 Collision is painted for each tile
 Level gets exported to XML format
 level loading independed from Unity (for server, client)
 in Unity: instantiate prefabs
 actually faster and more controllable!
Level consists of
„tiles“ (Unity prefabs)
Tile has 3d collider
(for mouse picking +
ground height
definition)
2d collision is painted
per tile
(blocks movement /
blocks shots / blocks
placement etc.)
Ground tile
Object (non-ground) tile
Collision
 TODO: pictures, demo
Collision of all tiles
is blended together
Unity Features we (don‘t) use…
 Do:
 Editor: cool and customizable (but often too slow)
 Sound: yes, but want to switch to Wwise or FMOD
 Mechanim: now, after a long struggle (memory usage!)
 Not or very limited:
 Scenes – only to load GUIs (but not levels)
 Collision/Physics – only raycasts @ ground
 GUI – use modified NGUI instead
 Network – use Photon instead
 Baked light – scenes way too large
Unity troubles
 funny story…
 64 bit editor came just in time for us
 32bit editor was no longer able to build our project
 used > 2.7 GB RAM during build with is the max for
win32 processes
 headless was still working!
Character rendering
 Character + Equipment gets baked into one mesh
 want 1 draw call / character (well, + special effects)
 real-time
 parts of character mesh hidden depending on equipment
 Only one material for character + all equipment items
 Only one texture; also use vertex colors, specular
 Limit number of characters drawn (esp. on mobile)
 Todo: characters picture
Debugging
 Can run and debug complete environment locally on developers
machine
 Human-readable network log file
 Server and client errors caught in log files
 Searchable, with context
 Public staging system
 Community can preview patches, updates
 QA freelancers from the community
 Frequently mirror live database to public staging system
 Can download live database to developer machines
 TODO: screenshot of network log file
Cheaters
 people started using memory debuggers and packet sniffers
pretty much on day 1
 Absolute server authority is king!
 Includes what information you reveal to client!
 only one serious cheat ever found
 mistake on our side –“believed“ data from client without proper
check
 Gold Sellers / Account Stealing
 adding email verification greatly improved this
Cheaters
 .NET code very accessible for analysis
 camera hacks (minior problem because of interest management)
 found internal tools , cheats in code (not working on live)
 extracted data from client
 maps, player rankings
 cool community projects!
 Users built bots directly into client 
 Difficult to prevent
 Obfuscation – helps, but not much
 We are doing more integrity checks now!
 Future: Unity IL2CPP ?
Future Challenges
 Growing player base…
 Want a more diverse world
 exploring more modular level design right now
 Keeping iOS / Android version in sync with server
 ability to download data files without patching
 clients support old server / server supports old clients
Thank you!
Questions / Comments?
david@sandbox-interactive.com
We are hiring!
https://albiononline.com/en/jobs

More Related Content

What's hot

NDC12_Lockless게임서버설계와구현
NDC12_Lockless게임서버설계와구현NDC12_Lockless게임서버설계와구현
NDC12_Lockless게임서버설계와구현noerror
 
게임 분산 서버 구조
게임 분산 서버 구조게임 분산 서버 구조
게임 분산 서버 구조Hyunjik Bae
 
[데브루키/141206 박민근] 유니티 최적화 테크닉 총정리
[데브루키/141206 박민근] 유니티 최적화 테크닉 총정리[데브루키/141206 박민근] 유니티 최적화 테크닉 총정리
[데브루키/141206 박민근] 유니티 최적화 테크닉 총정리MinGeun Park
 
MMOG Server-Side 충돌 및 이동처리 설계와 구현
MMOG Server-Side 충돌 및 이동처리 설계와 구현MMOG Server-Side 충돌 및 이동처리 설계와 구현
MMOG Server-Side 충돌 및 이동처리 설계와 구현YEONG-CHEON YOU
 
중앙 서버 없는 게임 로직
중앙 서버 없는 게임 로직중앙 서버 없는 게임 로직
중앙 서버 없는 게임 로직Hoyoung Choi
 
홍성우, 게임 서버의 목차 - 시작부터 출시까지, NDC2019
홍성우, 게임 서버의 목차 - 시작부터 출시까지, NDC2019홍성우, 게임 서버의 목차 - 시작부터 출시까지, NDC2019
홍성우, 게임 서버의 목차 - 시작부터 출시까지, NDC2019devCAT Studio, NEXON
 
임태현, 게임 서버 디자인 가이드, NDC2013
임태현, 게임 서버 디자인 가이드, NDC2013임태현, 게임 서버 디자인 가이드, NDC2013
임태현, 게임 서버 디자인 가이드, NDC2013devCAT Studio, NEXON
 
05_Reliable UDP 구현
05_Reliable UDP 구현05_Reliable UDP 구현
05_Reliable UDP 구현noerror
 
사설 서버를 막는 방법들 (프리섭, 더이상은 Naver)
사설 서버를 막는 방법들 (프리섭, 더이상은 Naver)사설 서버를 막는 방법들 (프리섭, 더이상은 Naver)
사설 서버를 막는 방법들 (프리섭, 더이상은 Naver)Seungmo Koo
 
[KGC 2012] Online Game Server Architecture Case Study Performance and Security
[KGC 2012] Online Game Server Architecture Case Study Performance and Security[KGC 2012] Online Game Server Architecture Case Study Performance and Security
[KGC 2012] Online Game Server Architecture Case Study Performance and SecuritySeungmin Shin
 
실시간 게임 서버 최적화 전략
실시간 게임 서버 최적화 전략실시간 게임 서버 최적화 전략
실시간 게임 서버 최적화 전략YEONG-CHEON YOU
 
김동건, 할머니가 들려주신 마비노기 개발 전설, NDC2019
김동건, 할머니가 들려주신 마비노기 개발 전설, NDC2019김동건, 할머니가 들려주신 마비노기 개발 전설, NDC2019
김동건, 할머니가 들려주신 마비노기 개발 전설, NDC2019devCAT Studio, NEXON
 
Next-generation MMORPG service architecture
Next-generation MMORPG service architectureNext-generation MMORPG service architecture
Next-generation MMORPG service architectureJongwon Kim
 
[IGC 2017] 아마존 구승모 - 게임 엔진으로 서버 제작 및 운영까지
[IGC 2017] 아마존 구승모 - 게임 엔진으로 서버 제작 및 운영까지[IGC 2017] 아마존 구승모 - 게임 엔진으로 서버 제작 및 운영까지
[IGC 2017] 아마존 구승모 - 게임 엔진으로 서버 제작 및 운영까지강 민우
 
라이브 서비스를 위한 게임 서버 구성
라이브 서비스를 위한 게임 서버 구성라이브 서비스를 위한 게임 서버 구성
라이브 서비스를 위한 게임 서버 구성Hyunjik Bae
 
모바일 게임 최적화
모바일 게임 최적화 모바일 게임 최적화
모바일 게임 최적화 tartist
 
테라로 살펴본 MMORPG의 논타겟팅 시스템
테라로 살펴본 MMORPG의 논타겟팅 시스템테라로 살펴본 MMORPG의 논타겟팅 시스템
테라로 살펴본 MMORPG의 논타겟팅 시스템QooJuice
 
Ndc14 분산 서버 구축의 ABC
Ndc14 분산 서버 구축의 ABCNdc14 분산 서버 구축의 ABC
Ndc14 분산 서버 구축의 ABCHo Gyu Lee
 
A simple and powerful property system for C++ (talk at GCDC 2008, Leipzig)
A simple and powerful property system for C++ (talk at GCDC 2008, Leipzig)   A simple and powerful property system for C++ (talk at GCDC 2008, Leipzig)
A simple and powerful property system for C++ (talk at GCDC 2008, Leipzig) David Salz
 
AWS 클라우드 기반 게임 아키텍처 사례 - AWS Summit Seoul 2017
AWS 클라우드 기반 게임 아키텍처 사례 - AWS Summit Seoul 2017AWS 클라우드 기반 게임 아키텍처 사례 - AWS Summit Seoul 2017
AWS 클라우드 기반 게임 아키텍처 사례 - AWS Summit Seoul 2017Amazon Web Services Korea
 

What's hot (20)

NDC12_Lockless게임서버설계와구현
NDC12_Lockless게임서버설계와구현NDC12_Lockless게임서버설계와구현
NDC12_Lockless게임서버설계와구현
 
게임 분산 서버 구조
게임 분산 서버 구조게임 분산 서버 구조
게임 분산 서버 구조
 
[데브루키/141206 박민근] 유니티 최적화 테크닉 총정리
[데브루키/141206 박민근] 유니티 최적화 테크닉 총정리[데브루키/141206 박민근] 유니티 최적화 테크닉 총정리
[데브루키/141206 박민근] 유니티 최적화 테크닉 총정리
 
MMOG Server-Side 충돌 및 이동처리 설계와 구현
MMOG Server-Side 충돌 및 이동처리 설계와 구현MMOG Server-Side 충돌 및 이동처리 설계와 구현
MMOG Server-Side 충돌 및 이동처리 설계와 구현
 
중앙 서버 없는 게임 로직
중앙 서버 없는 게임 로직중앙 서버 없는 게임 로직
중앙 서버 없는 게임 로직
 
홍성우, 게임 서버의 목차 - 시작부터 출시까지, NDC2019
홍성우, 게임 서버의 목차 - 시작부터 출시까지, NDC2019홍성우, 게임 서버의 목차 - 시작부터 출시까지, NDC2019
홍성우, 게임 서버의 목차 - 시작부터 출시까지, NDC2019
 
임태현, 게임 서버 디자인 가이드, NDC2013
임태현, 게임 서버 디자인 가이드, NDC2013임태현, 게임 서버 디자인 가이드, NDC2013
임태현, 게임 서버 디자인 가이드, NDC2013
 
05_Reliable UDP 구현
05_Reliable UDP 구현05_Reliable UDP 구현
05_Reliable UDP 구현
 
사설 서버를 막는 방법들 (프리섭, 더이상은 Naver)
사설 서버를 막는 방법들 (프리섭, 더이상은 Naver)사설 서버를 막는 방법들 (프리섭, 더이상은 Naver)
사설 서버를 막는 방법들 (프리섭, 더이상은 Naver)
 
[KGC 2012] Online Game Server Architecture Case Study Performance and Security
[KGC 2012] Online Game Server Architecture Case Study Performance and Security[KGC 2012] Online Game Server Architecture Case Study Performance and Security
[KGC 2012] Online Game Server Architecture Case Study Performance and Security
 
실시간 게임 서버 최적화 전략
실시간 게임 서버 최적화 전략실시간 게임 서버 최적화 전략
실시간 게임 서버 최적화 전략
 
김동건, 할머니가 들려주신 마비노기 개발 전설, NDC2019
김동건, 할머니가 들려주신 마비노기 개발 전설, NDC2019김동건, 할머니가 들려주신 마비노기 개발 전설, NDC2019
김동건, 할머니가 들려주신 마비노기 개발 전설, NDC2019
 
Next-generation MMORPG service architecture
Next-generation MMORPG service architectureNext-generation MMORPG service architecture
Next-generation MMORPG service architecture
 
[IGC 2017] 아마존 구승모 - 게임 엔진으로 서버 제작 및 운영까지
[IGC 2017] 아마존 구승모 - 게임 엔진으로 서버 제작 및 운영까지[IGC 2017] 아마존 구승모 - 게임 엔진으로 서버 제작 및 운영까지
[IGC 2017] 아마존 구승모 - 게임 엔진으로 서버 제작 및 운영까지
 
라이브 서비스를 위한 게임 서버 구성
라이브 서비스를 위한 게임 서버 구성라이브 서비스를 위한 게임 서버 구성
라이브 서비스를 위한 게임 서버 구성
 
모바일 게임 최적화
모바일 게임 최적화 모바일 게임 최적화
모바일 게임 최적화
 
테라로 살펴본 MMORPG의 논타겟팅 시스템
테라로 살펴본 MMORPG의 논타겟팅 시스템테라로 살펴본 MMORPG의 논타겟팅 시스템
테라로 살펴본 MMORPG의 논타겟팅 시스템
 
Ndc14 분산 서버 구축의 ABC
Ndc14 분산 서버 구축의 ABCNdc14 분산 서버 구축의 ABC
Ndc14 분산 서버 구축의 ABC
 
A simple and powerful property system for C++ (talk at GCDC 2008, Leipzig)
A simple and powerful property system for C++ (talk at GCDC 2008, Leipzig)   A simple and powerful property system for C++ (talk at GCDC 2008, Leipzig)
A simple and powerful property system for C++ (talk at GCDC 2008, Leipzig)
 
AWS 클라우드 기반 게임 아키텍처 사례 - AWS Summit Seoul 2017
AWS 클라우드 기반 게임 아키텍처 사례 - AWS Summit Seoul 2017AWS 클라우드 기반 게임 아키텍처 사례 - AWS Summit Seoul 2017
AWS 클라우드 기반 게임 아키텍처 사례 - AWS Summit Seoul 2017
 

Viewers also liked

Supercell Company report
Supercell Company reportSupercell Company report
Supercell Company reportBrandon Kittle
 
The Multiplayer Classroom:
 Designing Coursework as a Game
The Multiplayer Classroom:
 Designing Coursework as a GameThe Multiplayer Classroom:
 Designing Coursework as a Game
The Multiplayer Classroom:
 Designing Coursework as a Gamegamifyforthewin
 
A Virtual World Distributed Server Developed In Erlang As A Tool For Analysin...
A Virtual World Distributed Server Developed In Erlang As A Tool For Analysin...A Virtual World Distributed Server Developed In Erlang As A Tool For Analysin...
A Virtual World Distributed Server Developed In Erlang As A Tool For Analysin...l xf
 
SRECon EU 2016: Riot Games Vs the Internet
SRECon EU 2016: Riot Games Vs the InternetSRECon EU 2016: Riot Games Vs the Internet
SRECon EU 2016: Riot Games Vs the InternetAdam Comerford
 
Game Server by Teguh
Game Server by TeguhGame Server by Teguh
Game Server by TeguhAgate Studio
 
Make your-game-multiplayer
Make your-game-multiplayerMake your-game-multiplayer
Make your-game-multiplayerAndrew Lee
 
Online mobile game server use Firebase realtime aatabase
Online mobile game server use Firebase realtime aatabaseOnline mobile game server use Firebase realtime aatabase
Online mobile game server use Firebase realtime aatabaseNguyễn Bá Thành
 
Clash of Clans: Secrets of Success
Clash of Clans: Secrets of SuccessClash of Clans: Secrets of Success
Clash of Clans: Secrets of Successanastasiaalikova
 
Online game server on Akka.NET (NDC2016)
Online game server on Akka.NET (NDC2016)Online game server on Akka.NET (NDC2016)
Online game server on Akka.NET (NDC2016)Esun Kim
 
Clash Royale vs Clash of Clans
Clash Royale vs Clash of ClansClash Royale vs Clash of Clans
Clash Royale vs Clash of ClansLaura Irazoqui
 
클라우드 기반 Unity 게임 서버 구축, 60분이면 충분하다
클라우드 기반 Unity 게임 서버 구축, 60분이면 충분하다클라우드 기반 Unity 게임 서버 구축, 60분이면 충분하다
클라우드 기반 Unity 게임 서버 구축, 60분이면 충분하다Dae Kim
 
Riot Games Scalable Data Warehouse Lecture at UCSB / UCLA
Riot Games Scalable Data Warehouse Lecture at UCSB / UCLARiot Games Scalable Data Warehouse Lecture at UCSB / UCLA
Riot Games Scalable Data Warehouse Lecture at UCSB / UCLAsean_seannery
 
Innovative Uses of Technology in International Education
Innovative Uses of Technology in International Education Innovative Uses of Technology in International Education
Innovative Uses of Technology in International Education Marty Bennett
 
Gem 1
Gem 1Gem 1
Gem 1bfnd
 

Viewers also liked (20)

Clash royale hack
Clash royale hackClash royale hack
Clash royale hack
 
Supercell Company report
Supercell Company reportSupercell Company report
Supercell Company report
 
brgames_games
brgames_gamesbrgames_games
brgames_games
 
The Multiplayer Classroom:
 Designing Coursework as a Game
The Multiplayer Classroom:
 Designing Coursework as a GameThe Multiplayer Classroom:
 Designing Coursework as a Game
The Multiplayer Classroom:
 Designing Coursework as a Game
 
A Virtual World Distributed Server Developed In Erlang As A Tool For Analysin...
A Virtual World Distributed Server Developed In Erlang As A Tool For Analysin...A Virtual World Distributed Server Developed In Erlang As A Tool For Analysin...
A Virtual World Distributed Server Developed In Erlang As A Tool For Analysin...
 
SRECon EU 2016: Riot Games Vs the Internet
SRECon EU 2016: Riot Games Vs the InternetSRECon EU 2016: Riot Games Vs the Internet
SRECon EU 2016: Riot Games Vs the Internet
 
Game Server by Teguh
Game Server by TeguhGame Server by Teguh
Game Server by Teguh
 
Make your-game-multiplayer
Make your-game-multiplayerMake your-game-multiplayer
Make your-game-multiplayer
 
Online mobile game server use Firebase realtime aatabase
Online mobile game server use Firebase realtime aatabaseOnline mobile game server use Firebase realtime aatabase
Online mobile game server use Firebase realtime aatabase
 
Cedec2013 photon network engine
Cedec2013 photon network engineCedec2013 photon network engine
Cedec2013 photon network engine
 
Supercell
SupercellSupercell
Supercell
 
Clash of Clans: Secrets of Success
Clash of Clans: Secrets of SuccessClash of Clans: Secrets of Success
Clash of Clans: Secrets of Success
 
Clash royale
Clash royaleClash royale
Clash royale
 
Online game server on Akka.NET (NDC2016)
Online game server on Akka.NET (NDC2016)Online game server on Akka.NET (NDC2016)
Online game server on Akka.NET (NDC2016)
 
Clash Royale vs Clash of Clans
Clash Royale vs Clash of ClansClash Royale vs Clash of Clans
Clash Royale vs Clash of Clans
 
Supercell Report
Supercell ReportSupercell Report
Supercell Report
 
클라우드 기반 Unity 게임 서버 구축, 60분이면 충분하다
클라우드 기반 Unity 게임 서버 구축, 60분이면 충분하다클라우드 기반 Unity 게임 서버 구축, 60분이면 충분하다
클라우드 기반 Unity 게임 서버 구축, 60분이면 충분하다
 
Riot Games Scalable Data Warehouse Lecture at UCSB / UCLA
Riot Games Scalable Data Warehouse Lecture at UCSB / UCLARiot Games Scalable Data Warehouse Lecture at UCSB / UCLA
Riot Games Scalable Data Warehouse Lecture at UCSB / UCLA
 
Innovative Uses of Technology in International Education
Innovative Uses of Technology in International Education Innovative Uses of Technology in International Education
Innovative Uses of Technology in International Education
 
Gem 1
Gem 1Gem 1
Gem 1
 

Similar to Albion Online - Software Architecture of an MMO (talk at Quo Vadis 2016, Berlin)

Catan world and Churchill
Catan world and ChurchillCatan world and Churchill
Catan world and ChurchillGrant Goodale
 
Casual Engines 2009
Casual Engines 2009Casual Engines 2009
Casual Engines 2009David Fox
 
The tech. behind RoboBlastPlanet
The tech. behind RoboBlastPlanetThe tech. behind RoboBlastPlanet
The tech. behind RoboBlastPlanetJavier Abud
 
Game development with Cocos2d-x Engine
Game development with Cocos2d-x EngineGame development with Cocos2d-x Engine
Game development with Cocos2d-x EngineDuy Tan Geek
 
Building Multiplayer Games (w/ Unity)
Building Multiplayer Games (w/ Unity)Building Multiplayer Games (w/ Unity)
Building Multiplayer Games (w/ Unity)Noam Gat
 
Photon Session / Unite12 Conference
Photon Session / Unite12 ConferencePhoton Session / Unite12 Conference
Photon Session / Unite12 ConferenceChristof Wegmann
 
AAA 3D GRAPHICS ON THE WEB WITH REACTJS + BABYLONJS + UNITY3D by Denis Radin ...
AAA 3D GRAPHICS ON THE WEB WITH REACTJS + BABYLONJS + UNITY3D by Denis Radin ...AAA 3D GRAPHICS ON THE WEB WITH REACTJS + BABYLONJS + UNITY3D by Denis Radin ...
AAA 3D GRAPHICS ON THE WEB WITH REACTJS + BABYLONJS + UNITY3D by Denis Radin ...DevClub_lv
 
JS Fest 2019. Денис Радин. AAA 3D графика в Web с ReactJS, BabylonJS и Unity3D
JS Fest 2019. Денис Радин. AAA 3D графика в Web с ReactJS, BabylonJS и Unity3DJS Fest 2019. Денис Радин. AAA 3D графика в Web с ReactJS, BabylonJS и Unity3D
JS Fest 2019. Денис Радин. AAA 3D графика в Web с ReactJS, BabylonJS и Unity3DJSFestUA
 
Harlan Beverly Lag The Barrier to innovation gdc austin 2009
Harlan Beverly Lag The Barrier to innovation gdc austin 2009Harlan Beverly Lag The Barrier to innovation gdc austin 2009
Harlan Beverly Lag The Barrier to innovation gdc austin 2009Harlan Beverly
 
BitSquid Tech: Benefits of a data-driven renderer
BitSquid Tech: Benefits of a data-driven rendererBitSquid Tech: Benefits of a data-driven renderer
BitSquid Tech: Benefits of a data-driven renderertobias_persson
 
UGC In Game : A Brief History and How We Bring It To Mobile | Zhuo Yue
UGC In Game : A Brief History and How We Bring It To Mobile | Zhuo YueUGC In Game : A Brief History and How We Bring It To Mobile | Zhuo Yue
UGC In Game : A Brief History and How We Bring It To Mobile | Zhuo YueJessica Tams
 
Frostbite Rendering Architecture and Real-time Procedural Shading & Texturing...
Frostbite Rendering Architecture and Real-time Procedural Shading & Texturing...Frostbite Rendering Architecture and Real-time Procedural Shading & Texturing...
Frostbite Rendering Architecture and Real-time Procedural Shading & Texturing...Johan Andersson
 
"it's time to kick ass and chew bubble gum...", a unity 3D live coding demo
"it's time to kick ass and chew bubble gum...", a unity 3D live coding demo"it's time to kick ass and chew bubble gum...", a unity 3D live coding demo
"it's time to kick ass and chew bubble gum...", a unity 3D live coding demoDavid Hodgetts
 
Game Development with Unity
Game Development with UnityGame Development with Unity
Game Development with Unitydavidluzgouveia
 
Supersize your production pipe enjmin 2013 v1.1 hd
Supersize your production pipe    enjmin 2013 v1.1 hdSupersize your production pipe    enjmin 2013 v1.1 hd
Supersize your production pipe enjmin 2013 v1.1 hdslantsixgames
 
Massively Social != Massively Multiplayer
Massively Social != Massively MultiplayerMassively Social != Massively Multiplayer
Massively Social != Massively MultiplayerPaul Furio
 
Introduction to html5 game programming with impact js
Introduction to html5 game programming with impact jsIntroduction to html5 game programming with impact js
Introduction to html5 game programming with impact jsLuca Galli
 
Windows Embedded in the Real World
Windows Embedded in the Real WorldWindows Embedded in the Real World
Windows Embedded in the Real Worldukdpe
 
Confrontation Pipeline and SCons
Confrontation Pipeline and SConsConfrontation Pipeline and SCons
Confrontation Pipeline and SConsslantsixgames
 

Similar to Albion Online - Software Architecture of an MMO (talk at Quo Vadis 2016, Berlin) (20)

Catan world and Churchill
Catan world and ChurchillCatan world and Churchill
Catan world and Churchill
 
Casual Engines 2009
Casual Engines 2009Casual Engines 2009
Casual Engines 2009
 
The tech. behind RoboBlastPlanet
The tech. behind RoboBlastPlanetThe tech. behind RoboBlastPlanet
The tech. behind RoboBlastPlanet
 
Game development with Cocos2d-x Engine
Game development with Cocos2d-x EngineGame development with Cocos2d-x Engine
Game development with Cocos2d-x Engine
 
Xna
XnaXna
Xna
 
Building Multiplayer Games (w/ Unity)
Building Multiplayer Games (w/ Unity)Building Multiplayer Games (w/ Unity)
Building Multiplayer Games (w/ Unity)
 
Photon Session / Unite12 Conference
Photon Session / Unite12 ConferencePhoton Session / Unite12 Conference
Photon Session / Unite12 Conference
 
AAA 3D GRAPHICS ON THE WEB WITH REACTJS + BABYLONJS + UNITY3D by Denis Radin ...
AAA 3D GRAPHICS ON THE WEB WITH REACTJS + BABYLONJS + UNITY3D by Denis Radin ...AAA 3D GRAPHICS ON THE WEB WITH REACTJS + BABYLONJS + UNITY3D by Denis Radin ...
AAA 3D GRAPHICS ON THE WEB WITH REACTJS + BABYLONJS + UNITY3D by Denis Radin ...
 
JS Fest 2019. Денис Радин. AAA 3D графика в Web с ReactJS, BabylonJS и Unity3D
JS Fest 2019. Денис Радин. AAA 3D графика в Web с ReactJS, BabylonJS и Unity3DJS Fest 2019. Денис Радин. AAA 3D графика в Web с ReactJS, BabylonJS и Unity3D
JS Fest 2019. Денис Радин. AAA 3D графика в Web с ReactJS, BabylonJS и Unity3D
 
Harlan Beverly Lag The Barrier to innovation gdc austin 2009
Harlan Beverly Lag The Barrier to innovation gdc austin 2009Harlan Beverly Lag The Barrier to innovation gdc austin 2009
Harlan Beverly Lag The Barrier to innovation gdc austin 2009
 
BitSquid Tech: Benefits of a data-driven renderer
BitSquid Tech: Benefits of a data-driven rendererBitSquid Tech: Benefits of a data-driven renderer
BitSquid Tech: Benefits of a data-driven renderer
 
UGC In Game : A Brief History and How We Bring It To Mobile | Zhuo Yue
UGC In Game : A Brief History and How We Bring It To Mobile | Zhuo YueUGC In Game : A Brief History and How We Bring It To Mobile | Zhuo Yue
UGC In Game : A Brief History and How We Bring It To Mobile | Zhuo Yue
 
Frostbite Rendering Architecture and Real-time Procedural Shading & Texturing...
Frostbite Rendering Architecture and Real-time Procedural Shading & Texturing...Frostbite Rendering Architecture and Real-time Procedural Shading & Texturing...
Frostbite Rendering Architecture and Real-time Procedural Shading & Texturing...
 
"it's time to kick ass and chew bubble gum...", a unity 3D live coding demo
"it's time to kick ass and chew bubble gum...", a unity 3D live coding demo"it's time to kick ass and chew bubble gum...", a unity 3D live coding demo
"it's time to kick ass and chew bubble gum...", a unity 3D live coding demo
 
Game Development with Unity
Game Development with UnityGame Development with Unity
Game Development with Unity
 
Supersize your production pipe enjmin 2013 v1.1 hd
Supersize your production pipe    enjmin 2013 v1.1 hdSupersize your production pipe    enjmin 2013 v1.1 hd
Supersize your production pipe enjmin 2013 v1.1 hd
 
Massively Social != Massively Multiplayer
Massively Social != Massively MultiplayerMassively Social != Massively Multiplayer
Massively Social != Massively Multiplayer
 
Introduction to html5 game programming with impact js
Introduction to html5 game programming with impact jsIntroduction to html5 game programming with impact js
Introduction to html5 game programming with impact js
 
Windows Embedded in the Real World
Windows Embedded in the Real WorldWindows Embedded in the Real World
Windows Embedded in the Real World
 
Confrontation Pipeline and SCons
Confrontation Pipeline and SConsConfrontation Pipeline and SCons
Confrontation Pipeline and SCons
 

Recently uploaded

UNI DI NAPOLI FEDERICO II - Il ruolo dei grafi nell'AI Conversazionale Ibrida
UNI DI NAPOLI FEDERICO II - Il ruolo dei grafi nell'AI Conversazionale IbridaUNI DI NAPOLI FEDERICO II - Il ruolo dei grafi nell'AI Conversazionale Ibrida
UNI DI NAPOLI FEDERICO II - Il ruolo dei grafi nell'AI Conversazionale IbridaNeo4j
 
Microsoft365_Dev_Security_2024_05_16.pdf
Microsoft365_Dev_Security_2024_05_16.pdfMicrosoft365_Dev_Security_2024_05_16.pdf
Microsoft365_Dev_Security_2024_05_16.pdfMarkus Moeller
 
architecting-ai-in-the-enterprise-apis-and-applications.pdf
architecting-ai-in-the-enterprise-apis-and-applications.pdfarchitecting-ai-in-the-enterprise-apis-and-applications.pdf
architecting-ai-in-the-enterprise-apis-and-applications.pdfWSO2
 
Modern binary build systems - PyCon 2024
Modern binary build systems - PyCon 2024Modern binary build systems - PyCon 2024
Modern binary build systems - PyCon 2024Henry Schreiner
 
Anypoint Code Builder - Munich MuleSoft Meetup - 16th May 2024
Anypoint Code Builder - Munich MuleSoft Meetup - 16th May 2024Anypoint Code Builder - Munich MuleSoft Meetup - 16th May 2024
Anypoint Code Builder - Munich MuleSoft Meetup - 16th May 2024MulesoftMunichMeetup
 
From Theory to Practice: Utilizing SpiraPlan's REST API
From Theory to Practice: Utilizing SpiraPlan's REST APIFrom Theory to Practice: Utilizing SpiraPlan's REST API
From Theory to Practice: Utilizing SpiraPlan's REST APIInflectra
 
The mythical technical debt. (Brooke, please, forgive me)
The mythical technical debt. (Brooke, please, forgive me)The mythical technical debt. (Brooke, please, forgive me)
The mythical technical debt. (Brooke, please, forgive me)Roberto Bettazzoni
 
A Deep Dive into Secure Product Development Frameworks.pdf
A Deep Dive into Secure Product Development Frameworks.pdfA Deep Dive into Secure Product Development Frameworks.pdf
A Deep Dive into Secure Product Development Frameworks.pdfICS
 
Navigation in flutter – how to add stack, tab, and drawer navigators to your ...
Navigation in flutter – how to add stack, tab, and drawer navigators to your ...Navigation in flutter – how to add stack, tab, and drawer navigators to your ...
Navigation in flutter – how to add stack, tab, and drawer navigators to your ...Flutter Agency
 
From Knowledge Graphs via Lego Bricks to scientific conversations.pptx
From Knowledge Graphs via Lego Bricks to scientific conversations.pptxFrom Knowledge Graphs via Lego Bricks to scientific conversations.pptx
From Knowledge Graphs via Lego Bricks to scientific conversations.pptxNeo4j
 
The Strategic Impact of Buying vs Building in Test Automation
The Strategic Impact of Buying vs Building in Test AutomationThe Strategic Impact of Buying vs Building in Test Automation
The Strategic Impact of Buying vs Building in Test AutomationElement34
 
The Evolution of Web App Testing_ An Ultimate Guide to Future Trends.pdf
The Evolution of Web App Testing_ An Ultimate Guide to Future Trends.pdfThe Evolution of Web App Testing_ An Ultimate Guide to Future Trends.pdf
The Evolution of Web App Testing_ An Ultimate Guide to Future Trends.pdfkalichargn70th171
 
Community is Just as Important as Code by Andrea Goulet
Community is Just as Important as Code by Andrea GouletCommunity is Just as Important as Code by Andrea Goulet
Community is Just as Important as Code by Andrea GouletAndrea Goulet
 
Jax, FL Admin Community Group 05.14.2024 Combined Deck
Jax, FL Admin Community Group 05.14.2024 Combined DeckJax, FL Admin Community Group 05.14.2024 Combined Deck
Jax, FL Admin Community Group 05.14.2024 Combined DeckMarc Lester
 
Rapidoform for Modern Form Building and Insights
Rapidoform for Modern Form Building and InsightsRapidoform for Modern Form Building and Insights
Rapidoform for Modern Form Building and Insightsrapidoform
 
Spring into AI presented by Dan Vega 5/14
Spring into AI presented by Dan Vega 5/14Spring into AI presented by Dan Vega 5/14
Spring into AI presented by Dan Vega 5/14VMware Tanzu
 
Team Transformation Tactics for Holistic Testing and Quality (NewCrafts Paris...
Team Transformation Tactics for Holistic Testing and Quality (NewCrafts Paris...Team Transformation Tactics for Holistic Testing and Quality (NewCrafts Paris...
Team Transformation Tactics for Holistic Testing and Quality (NewCrafts Paris...Lisi Hocke
 

Recently uploaded (20)

UNI DI NAPOLI FEDERICO II - Il ruolo dei grafi nell'AI Conversazionale Ibrida
UNI DI NAPOLI FEDERICO II - Il ruolo dei grafi nell'AI Conversazionale IbridaUNI DI NAPOLI FEDERICO II - Il ruolo dei grafi nell'AI Conversazionale Ibrida
UNI DI NAPOLI FEDERICO II - Il ruolo dei grafi nell'AI Conversazionale Ibrida
 
Microsoft365_Dev_Security_2024_05_16.pdf
Microsoft365_Dev_Security_2024_05_16.pdfMicrosoft365_Dev_Security_2024_05_16.pdf
Microsoft365_Dev_Security_2024_05_16.pdf
 
architecting-ai-in-the-enterprise-apis-and-applications.pdf
architecting-ai-in-the-enterprise-apis-and-applications.pdfarchitecting-ai-in-the-enterprise-apis-and-applications.pdf
architecting-ai-in-the-enterprise-apis-and-applications.pdf
 
Modern binary build systems - PyCon 2024
Modern binary build systems - PyCon 2024Modern binary build systems - PyCon 2024
Modern binary build systems - PyCon 2024
 
Anypoint Code Builder - Munich MuleSoft Meetup - 16th May 2024
Anypoint Code Builder - Munich MuleSoft Meetup - 16th May 2024Anypoint Code Builder - Munich MuleSoft Meetup - 16th May 2024
Anypoint Code Builder - Munich MuleSoft Meetup - 16th May 2024
 
Abortion Clinic In Pretoria ](+27832195400*)[ 🏥 Safe Abortion Pills in Pretor...
Abortion Clinic In Pretoria ](+27832195400*)[ 🏥 Safe Abortion Pills in Pretor...Abortion Clinic In Pretoria ](+27832195400*)[ 🏥 Safe Abortion Pills in Pretor...
Abortion Clinic In Pretoria ](+27832195400*)[ 🏥 Safe Abortion Pills in Pretor...
 
From Theory to Practice: Utilizing SpiraPlan's REST API
From Theory to Practice: Utilizing SpiraPlan's REST APIFrom Theory to Practice: Utilizing SpiraPlan's REST API
From Theory to Practice: Utilizing SpiraPlan's REST API
 
The mythical technical debt. (Brooke, please, forgive me)
The mythical technical debt. (Brooke, please, forgive me)The mythical technical debt. (Brooke, please, forgive me)
The mythical technical debt. (Brooke, please, forgive me)
 
A Deep Dive into Secure Product Development Frameworks.pdf
A Deep Dive into Secure Product Development Frameworks.pdfA Deep Dive into Secure Product Development Frameworks.pdf
A Deep Dive into Secure Product Development Frameworks.pdf
 
Abortion Clinic In Johannesburg ](+27832195400*)[ 🏥 Safe Abortion Pills in Jo...
Abortion Clinic In Johannesburg ](+27832195400*)[ 🏥 Safe Abortion Pills in Jo...Abortion Clinic In Johannesburg ](+27832195400*)[ 🏥 Safe Abortion Pills in Jo...
Abortion Clinic In Johannesburg ](+27832195400*)[ 🏥 Safe Abortion Pills in Jo...
 
Navigation in flutter – how to add stack, tab, and drawer navigators to your ...
Navigation in flutter – how to add stack, tab, and drawer navigators to your ...Navigation in flutter – how to add stack, tab, and drawer navigators to your ...
Navigation in flutter – how to add stack, tab, and drawer navigators to your ...
 
From Knowledge Graphs via Lego Bricks to scientific conversations.pptx
From Knowledge Graphs via Lego Bricks to scientific conversations.pptxFrom Knowledge Graphs via Lego Bricks to scientific conversations.pptx
From Knowledge Graphs via Lego Bricks to scientific conversations.pptx
 
Abortion Pill Prices Mthatha (@](+27832195400*)[ 🏥 Women's Abortion Clinic In...
Abortion Pill Prices Mthatha (@](+27832195400*)[ 🏥 Women's Abortion Clinic In...Abortion Pill Prices Mthatha (@](+27832195400*)[ 🏥 Women's Abortion Clinic In...
Abortion Pill Prices Mthatha (@](+27832195400*)[ 🏥 Women's Abortion Clinic In...
 
The Strategic Impact of Buying vs Building in Test Automation
The Strategic Impact of Buying vs Building in Test AutomationThe Strategic Impact of Buying vs Building in Test Automation
The Strategic Impact of Buying vs Building in Test Automation
 
The Evolution of Web App Testing_ An Ultimate Guide to Future Trends.pdf
The Evolution of Web App Testing_ An Ultimate Guide to Future Trends.pdfThe Evolution of Web App Testing_ An Ultimate Guide to Future Trends.pdf
The Evolution of Web App Testing_ An Ultimate Guide to Future Trends.pdf
 
Community is Just as Important as Code by Andrea Goulet
Community is Just as Important as Code by Andrea GouletCommunity is Just as Important as Code by Andrea Goulet
Community is Just as Important as Code by Andrea Goulet
 
Jax, FL Admin Community Group 05.14.2024 Combined Deck
Jax, FL Admin Community Group 05.14.2024 Combined DeckJax, FL Admin Community Group 05.14.2024 Combined Deck
Jax, FL Admin Community Group 05.14.2024 Combined Deck
 
Rapidoform for Modern Form Building and Insights
Rapidoform for Modern Form Building and InsightsRapidoform for Modern Form Building and Insights
Rapidoform for Modern Form Building and Insights
 
Spring into AI presented by Dan Vega 5/14
Spring into AI presented by Dan Vega 5/14Spring into AI presented by Dan Vega 5/14
Spring into AI presented by Dan Vega 5/14
 
Team Transformation Tactics for Holistic Testing and Quality (NewCrafts Paris...
Team Transformation Tactics for Holistic Testing and Quality (NewCrafts Paris...Team Transformation Tactics for Holistic Testing and Quality (NewCrafts Paris...
Team Transformation Tactics for Holistic Testing and Quality (NewCrafts Paris...
 

Albion Online - Software Architecture of an MMO (talk at Quo Vadis 2016, Berlin)

  • 1. Software Architecture of an MMO David Salz david@sandbox-interactive.com
  • 2. Who am I?  David Salz  15 years in the industry  CTO, Co-Founder of Sandbox Interactive  35 people  based in Berlin  we only do Albion Online!
  • 3. In this talk:  What is Albion Online?  Middleware + Design Decisions  Server Architecture  How we use Unity (and how not)
  • 4.
  • 5. Albion Online  Sandbox MMORPG  Cross-Platform (Windows/OSX/Linux/Android/iOS)  One World (no „Shards“ or „Servers“, not even for different platforms)  Player-Driven Economy (everything is player-crafted)  No Character Classes („you are what you wear“)  Strong focus on PvP + Guilds  Hardcore („Full Loot“ in PVP Areas)  Pay-to-play model  4 years in the making  Currently in Closed Beta w/ 80.000+ „founding“ players  „Release“ in Q4/2016
  • 6. Albion Online  The initial pitch (2012)  Play-crafted equipment, buildings  One World (like EVE Online)  Guild vs. Guild Territotorial conquest  Top-Down perspective + Combat (like League of Legends)  Simple graphics, „can even be 2D“  PVP focus, no PVE, no Mobs, no Dungeons  Release in 2013 
  • 7. Middleware Selection  Engine Selection  Unity made sense – inexpensive, accessible  Cross-Platform was a „target of opportunity“  Database Selection  One world  need a very scalable database, NoSQL  Cassandra sounded suitable  still use SQL for query-heavy problems  Networking Middleware  Photon!  can use C# on server (like in Unity!)  works with all Unity platforms
  • 8. Apache Cassandra  NoSQL Database  Originally developed by Facebook  Open Source (Apache-License)  Java  Concept: everything is a hash table  in-memory (as much as possible)  optimized for high throughput  scales horizontally (just add servers)  redundant (no single point of failure)  CQL: SQL-like language (w/ massive restrictions – it‘s NoSQL)
  • 9. Apache Cassandra  funny story from public Alpha Test…  all buildings disappeared during lunch break  found this in Cassandra Changelog:  W(  lesson learned: careful with cutting edge technology * Fix possible loss of 2ndary index entries during compaction (CASSANDRA-6517)
  • 10. Apache Cassandra  Cassandra uses timestamps to resolve conflicts between database nodes  also requires exact time sync beween servers  i.e. ntp service
  • 11. Hosting  Hosted with Softlayer  good, but quite expensive  bare-metal machines only  currently: 8x 10-Core Intel Xeon, 32 GB RAM  Should be able to handle ~15k CCUs  Originally:  Hosted across 2 Datacenters (better ping for players)  Worked, but bad idea! (inter-server disconnects, latency)
  • 12. Networking  Photon  UDP-based protocol (reliable/unreliable as needed)  TCP for chat, inter-server communication  use only basic message functionality (no Photon frameworks)  mostly simple messages, in rare cases JSON content  Had to implement secure TCP-reconnect abilities  Chat  Tried IRC…  … was only trouble (difficult to secure, customize)  Implemented own system in a couple of days
  • 13. Consequences  Server needs to work without Unity  Ideally, client works without Unity, too!  think: tools, stress-test-bots!  Use Unity only as rendering front-end  cannot rely on Unity features for basic functions!  levels / game objects  collision  pathfinding
  • 16. Server Farm Game Server Game Server Game Server Login Server World Server Marketplace Server GoldMarket Server Statistics Server Ranking Server BackOffice Server Chat Server Client Connect to different game server depending on location in game world Game DB (Cassandra) Accounts DB (Postgres) Market DB (Postgres) GoldMarket DB (Postgres)
  • 17.  Todo: screenshot of gold market
  • 18.  Todo: screenshot of marketplace
  • 19. Game Servers / World Server  game world split into unique „clusters“ of ~ 1 km²  current world: ~600, next world: x2  distributed among game servers  player changes servers when travelling  handoff done through database (even on same server)  Instances are dynamically created  world server coordinates this  World Server  responsible for everything „global“  guilds, parties, coordination of guild battles etc.
  • 20.
  • 21. Threading Model (Game Server) IO ThreadIO ThreadNetwork IO Thread Event Queue Logging IO Thread Event Scheduler Cluster Thread Event Queue Event Scheduler Cluster Thread Pathfinding Thread Database IO Thread Cluster Thread makes request Events / Results get put into Event Queue Cluster polls Events from Queue and Scheduler
  • 22. Threading Model  Game logic for one game area („cluster“) is single-threaded  Interactions between game objects too complicated otherwise; would require lots of locking  Objects „sleep“ whenever possible (especially mobs)  All IO (Network, Database, Logging) and other „heavy lifting“ (pathfinding) offloaded to worker threads  Instances (Player islands, instanced dungeons) are combined in thread pools  Other servers (chat, rankings) are completely multi-threaded
  • 23. There is no spoon!  There is no game loop!  i.e. no Update() or Tick() functions for game objects  Gameplay thread processes events in order of arrival  Events are:  Network input, i.e. player commands  Timers (Event Scheduler)  If an object needs a tick (Mobs do), it creates a recurring timer  Results from DB queries (asynchronous)  Results from pathfinding  …
  • 24. Interest Management  Players should only „see“ (network-wise) what‘s on their screen  important to reduce traffic…  … and prevent cheating!  Find objects a player can „see“ (and update when things move)  Objects send messages to all players that „see“ them  Needs to be efficient!  ~500 mobs / cluster  > 10.000 interactive objects (mostly trees )  up to 300 players
  • 25. Interest Management grid based hash (10x10m cells) ● cells contain list of objects inside ● list is updated by object itself when moving ● cells fire events: ● ObjectEnteredCell() ● ObjectLeftCell()player ● objects also fire events ● ObjectMoved() ● ObjectDestroyed() ● EventFired()
  • 26. Interest Management player Interest area Moves with player Subscribe to objects when they enter inner area Unsubscribe when they leave outer area Tell player when: ● an object enters / leaves interest area ● an object in the area fires an event
  • 27. Logging infrastructure Game Server Game Server Game Server Log File Log File Log File Logstash Agent Redis Elasic Search Kibana  track errors, generate statistics  classic ELK stack (Elastic Search, Logstash, Kibana)  kind of unreliable for us  Also use Cassandra Counters a lot! Logstash Agent Logstash Agent Own tools
  • 31. Level Design  Zones built in Unity (Unity scenes)  Consist only of Unity prefabs („tiles“)  ~ 30.000 / level  Collision is painted for each tile  Level gets exported to XML format  level loading independed from Unity (for server, client)  in Unity: instantiate prefabs  actually faster and more controllable!
  • 32. Level consists of „tiles“ (Unity prefabs) Tile has 3d collider (for mouse picking + ground height definition) 2d collision is painted per tile (blocks movement / blocks shots / blocks placement etc.) Ground tile Object (non-ground) tile
  • 33. Collision  TODO: pictures, demo Collision of all tiles is blended together
  • 34. Unity Features we (don‘t) use…  Do:  Editor: cool and customizable (but often too slow)  Sound: yes, but want to switch to Wwise or FMOD  Mechanim: now, after a long struggle (memory usage!)  Not or very limited:  Scenes – only to load GUIs (but not levels)  Collision/Physics – only raycasts @ ground  GUI – use modified NGUI instead  Network – use Photon instead  Baked light – scenes way too large
  • 35. Unity troubles  funny story…  64 bit editor came just in time for us  32bit editor was no longer able to build our project  used > 2.7 GB RAM during build with is the max for win32 processes  headless was still working!
  • 36. Character rendering  Character + Equipment gets baked into one mesh  want 1 draw call / character (well, + special effects)  real-time  parts of character mesh hidden depending on equipment  Only one material for character + all equipment items  Only one texture; also use vertex colors, specular  Limit number of characters drawn (esp. on mobile)
  • 37.
  • 39. Debugging  Can run and debug complete environment locally on developers machine  Human-readable network log file  Server and client errors caught in log files  Searchable, with context  Public staging system  Community can preview patches, updates  QA freelancers from the community  Frequently mirror live database to public staging system  Can download live database to developer machines
  • 40.  TODO: screenshot of network log file
  • 41. Cheaters  people started using memory debuggers and packet sniffers pretty much on day 1  Absolute server authority is king!  Includes what information you reveal to client!  only one serious cheat ever found  mistake on our side –“believed“ data from client without proper check  Gold Sellers / Account Stealing  adding email verification greatly improved this
  • 42. Cheaters  .NET code very accessible for analysis  camera hacks (minior problem because of interest management)  found internal tools , cheats in code (not working on live)  extracted data from client  maps, player rankings  cool community projects!  Users built bots directly into client   Difficult to prevent  Obfuscation – helps, but not much  We are doing more integrity checks now!  Future: Unity IL2CPP ?
  • 43.
  • 44. Future Challenges  Growing player base…  Want a more diverse world  exploring more modular level design right now  Keeping iOS / Android version in sync with server  ability to download data files without patching  clients support old server / server supports old clients
  • 45. Thank you! Questions / Comments? david@sandbox-interactive.com We are hiring! https://albiononline.com/en/jobs