Your systems. Working as one.May 15, 2013Reinier Torenbeekreinier@rti.comLearn How to Develop aDistributed Game of Life wi...
Agenda• Problem definition: Life Distributed• A solution: RTI Connext DDS• Applying DDS to Life Distributed: concepts• App...
Conways Game of Life• Devised by John Conway in 1970• Zero-player game– evolution determined by initial state– no further ...
Conways Game of LifeAt each step in time, the following transitions occur:1. Any live cell with fewer than two live neighb...
Conways Game of LifeGlider gunPulsarBlock
Conways Game of Life – DistributedProblem description: how can Life be properlyimplemented in a distributed fashion?• have...
Conways Game of Life – Distributed
Conways Game of Life – DistributedProblem description: how can Life be properlyimplemented in a distributed fashion?• have...
Conways Game of Life – Distributed
Conways Game of Life – DistributedProblem description: how can Life be properlyimplemented in a distributed fashion?• have...
Conways Game of Life – DistributedProperly here means:• with minimal impact on the application logics– let distribution ar...
Agenda• Problem definition: Life Distributed• A solution: RTI Connext DDS• Applying DDS to Life Distributed: concepts• App...
RTI Connext DDSA few words describing RTI Connext DDS:• an implementation of the Object ManagementGroup (OMG) Data Distrib...
RTI Connext DDSDDS revolves around the concept of a typed data-space that• consists of a collection of structured, observa...
RTI Connext DDSDDS revolves around the concept of a typed data-space that• allows for extensive fine-tuning– to adjust dis...
Agenda• Problem definition: Life Distributed• A solution: RTI Connext DDS• Applying DDS to Life Distributed: concepts• App...
Applying DDS to Life DistributedFirst step is to define the data-model in IDL• cells are observable items, or "instances"–...
module life {struct CellType {long row; //@keylong col; //@keyunsigned long generation;boolean alive;};};
Applying DDS to Life DistributedFirst step is to define the data-model in IDL• cells are observable items, or "instances"–...
row: 16col: 4generation: 25alive: false
Applying DDS to Life DistributedEach process is responsible for publishing thestate of a certain subset of cells of the Un...
(1,1) – (10,10)
Applying DDS to Life DistributedEach process is responsible for publishing thestate of a certain subset of cells of the Un...
Applying DDS to Life DistributedEach process subscribes to the required subsetof cells in order to determine its current s...
Applying DDS to Life DistributedEach process subscribes to the required subsetof cells in order to determine its current s...
"((row >= 1 AND row <= 11) OR row = 20) AND((col >= 1 AND col <= 11) OR col = 20)"
Applying DDS to Life DistributedEach process subscribes to the required subsetof cells in order to determine its current s...
Applying DDS to Life DistributedAdditional processes can be added to peek atthe evolution of Life:• subscribing to (a subs...
"row >= 8 AND row <= 13 AND col >= 8 AND col <= 13"
Applying DDS to Life DistributedAdditional processes can be added to peek atthe evolution of Life:• subscribing to (a subs...
Agenda• Problem definition: Life Distributed• A solution: RTI Connext DDS• Applying DDS to Life Distributed: concepts• App...
Life Distributed (pseudo-)codeLife Distributed prototype applications weredeveloped on Mac OS X• Life evolution applicatio...
Life Distributed (pseudo-)codeLife evolution application written in C:• application is responsible for– knowing about the ...
initialize DDScurrent generation = 0write sub-universe Life seed to DDSrepeatrepeatwait for DDS cell update forcurrent gen...
Life Distributed (pseudo-)codeWorth to note about the Life application:• loss of one cell-update will eventually stall the...
<dds><qos_library name="GameOfLifeQosLibrary"><qos_profile name="CellProfile"><topic_qos><reliability><kind>RELIABLE_RELIA...
Life Distributed (pseudo-)codeWorth to note about the Life application:• loss of one cell-update will eventually stall the...
<dds><qos_library name="GameOfLifeQosLibrary"><qos_profile name="CellProfile"><topic_qos><reliability><kind>RELIABLE_RELIA...
Life Distributed (pseudo-)codeWorth to note about the Life application:• DDS cell updates come from different places– most...
create DDS DomainParticipantwith DomainParticipant, create DDS Topic "CellTopic"with CellTopic and filterexpression, creat...
Life Distributed (pseudo-)codeWorth to note about the Life application:• DDS cell updates come from different places– most...
create DDS DomainParticipantwith DomainParticipant, create DDS Topic "CellTopic"with CellTopic and filterexpression, creat...
Life Distributed (pseudo-)codeLife observer application written in Python:• application is responsible for– subscribing to...
import clifedds as life#omitted option parsingfilterString = row>={} and col>={} and row<={} andcol<={}.format(options.min...
Life Distributed (pseudo-)codeLife observer application written in Python:• application is responsible for– subscribing to...
<dds><qos_library name="GameOfLifeQosLibrary"><qos_profile name="CellProfile"><topic_qos><history><kind>KEEP_LAST_HISTORY_...
Life Distributed (pseudo-)codedomainId# of generationsuniverse dimensionssub-universe dimensionsExample of running a singl...
Life Distributed (pseudo-)codeExample of running a life scenario:
Agenda• Problem definition: Life Distributed• A solution: RTI Connext DDS• Applying DDS to Life Distributed: concepts• App...
Fault toleranceNot all is lost if Life application crashes• if using TRANSIENT durabilityQoS, infrastructure will keep sta...
persistenceservice
Fault toleranceNot all is lost if Life application crashes• if using TRANSIENT durability QoS,infrastructure will keep sta...
Reliability and flow controlRunning the Python app with a larger grid:• with current QoS, faster writer with slowerreader ...
Reliability and flow controlRunning the Python app with a larger grid:• whenever at least one cell update is missing,the g...
<dds><qos_library name="GameOfLifeQosLibrary"><qos_profile name="CellProfile"><topic_qos><reliability><kind>RELIABLE_RELIA...
Reliability and flow controlRunning the Python app with a larger grid:• whenever at least one cell update is missing,the g...
More advanced problem solvingOther ways to improve the Life implementation:• for centralized grid configuration, distribut...
More advanced problem solvingOther ways to improve the Life implementation:• in addition to separate cells, distribute com...
Agenda• Problem definition: Life Distributed• A solution: RTI Connext DDS• Applying DDS to Life Distributed: concepts• App...
SummaryDistributed Life can be properly implementedleveraging DDS• communication complexity is off-loaded tomiddleware, de...
Agenda• Problem definition: Life Distributed• A solution: RTI Connext DDS• Applying DDS to Life Distributed: concepts• App...
Questions?Thanks!
Learn How to Develop a Distributed Game of Life with DDS
Learn How to Develop a Distributed Game of Life with DDS
Learn How to Develop a Distributed Game of Life with DDS
Learn How to Develop a Distributed Game of Life with DDS
Upcoming SlideShare
Loading in …5
×

Learn How to Develop a Distributed Game of Life with DDS

2,038 views

Published on

The Game of Life is a famous zero-player game that has been devised by John Conway more than forty years ago. It is a cellular automaton that runs on a rectangular grid of cells, each of which is either alive or dead. A set of four simple transition rules prescribe how the cells in the grid evolve from one generation to the other.

Life, as it is called for short, has been able to capture the fascination of many programmers ever since it was published – mostly because of the surprising behavior it can result in. Many programmers have written implementations of Life at some point, probably in an educational context or maybe just for fun.

This webinar is intended as a combination of the two: education and fun. It will show how a distributed version of Life can be implemented using the Data Distribution Service (DDS) standard from the Object Management Group (OMG). The presented Distributed Life system will be able to deal with real-life system requirements like fault tolerance, scalability and deployment flexibility. It will be shown that by leveraging advanced DDS data-management features, system developers can off-load most of the complexity associated with the distribution and fault-tolerance aspects onto the infrastructure and focus on the application logics.

Published in: Technology
0 Comments
3 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
2,038
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
20
Comments
0
Likes
3
Embeds 0
No embeds

No notes for slide

Learn How to Develop a Distributed Game of Life with DDS

  1. 1. Your systems. Working as one.May 15, 2013Reinier Torenbeekreinier@rti.comLearn How to Develop aDistributed Game of Life with DDS
  2. 2. Agenda• Problem definition: Life Distributed• A solution: RTI Connext DDS• Applying DDS to Life Distributed: concepts• Applying DDS to Life Distributed: (pseudo)-code• Advanced Life Distributed: leveraging DDS• Summary• Questions and Answers
  3. 3. Conways Game of Life• Devised by John Conway in 1970• Zero-player game– evolution determined by initial state– no further input required• Plays in two-dimensional, orthogonal grid ofsquare cells– originally of infinite size– for this webinar, toroidal array is used• At any moment in time, each cell is either dead oralive• Neighboring cells interact with each other– horizontally, vertically, or diagonally adjacent.
  4. 4. Conways Game of LifeAt each step in time, the following transitions occur:1. Any live cell with fewer than two live neighborsdies, as if caused by under-population.2. Any live cell with two or three live neighbors lives onto the next generation.3. Any live cell with more than three live neighborsdies, as if by overcrowding.4. Any dead cell with exactly three live neighborsbecomes a live cell, as if by reproduction.These rules continue to be applied repeatedly to createfurther generations.
  5. 5. Conways Game of LifeGlider gunPulsarBlock
  6. 6. Conways Game of Life – DistributedProblem description: how can Life be properlyimplemented in a distributed fashion?• have multiple processes work on parts of theUniverse in parallel
  7. 7. Conways Game of Life – Distributed
  8. 8. Conways Game of Life – DistributedProblem description: how can Life be properlyimplemented in a distributed fashion?• have multiple processes work on parts of theUniverse in parallel• have these processes exchange the requiredinformation for the evolutionary steps
  9. 9. Conways Game of Life – Distributed
  10. 10. Conways Game of Life – DistributedProblem description: how can Life be properlyimplemented in a distributed fashion?• have multiple processes work on parts of theUniverse in parallel• have these processes exchange the requiredinformation for the evolutionary stepsThis problem and its solution serve as anexample for developing distributed applicationsin general
  11. 11. Conways Game of Life – DistributedProperly here means:• with minimal impact on the application logics– let distribution artifacts be dealt with transparently– let the developer focus on Life and its algorithms• allowing for mixed environments– multiple programming languages, OS-es and hardware– asymmetric processing power• supporting scalability– for very large Life Universes on many machines– for load balancing of CPU intensive calculations• in a fault-tolerant fashion
  12. 12. Agenda• Problem definition: Life Distributed• A solution: RTI Connext DDS• Applying DDS to Life Distributed: concepts• Applying DDS to Life Distributed: (pseudo)-code• Advanced Life Distributed: leveraging DDS• Summary• Questions and Answers
  13. 13. RTI Connext DDSA few words describing RTI Connext DDS:• an implementation of the Object ManagementGroup (OMG) Data Distribution Service (DDS)– standardized, multi-language API– standardized wire-protocol– see www.rti.com/elearning for tutorials (some free)• a high performance, scalable, anonymouspublish/subscribe infrastructure• an advanced distributed data managementtechnology– supporting many features know from DBMS-es
  14. 14. RTI Connext DDSDDS revolves around the concept of a typed data-space that• consists of a collection of structured, observableitems which– go through their individual lifecycle ofcreation, updating and deletion (CRUD)– are updated by Publishers– are observed by Subscribers• is managed in a distributed fashion– by Connext libraries and (optionally) services– transparent to applications
  15. 15. RTI Connext DDSDDS revolves around the concept of a typed data-space that• allows for extensive fine-tuning– to adjust distribution behavior according toapplication needs– using standard Quality of Service (QoS) mechanisms• can evolve dynamically– allowing Publishers and Subscribers to join and leaveat any time– automatically discovering communication pathsbetween Publishers and Subscribers
  16. 16. Agenda• Problem definition: Life Distributed• A solution: RTI Connext DDS• Applying DDS to Life Distributed: concepts• Applying DDS to Life Distributed: (pseudo)-code• Advanced Life Distributed: leveraging DDS• Summary• Questions and Answers
  17. 17. Applying DDS to Life DistributedFirst step is to define the data-model in IDL• cells are observable items, or "instances"– row and col identify their location in the grid– generation identifies the "tick nr" in evolution– alive identifies the state of the cell
  18. 18. module life {struct CellType {long row; //@keylong col; //@keyunsigned long generation;boolean alive;};};
  19. 19. Applying DDS to Life DistributedFirst step is to define the data-model in IDL• cells are observable items, or "instances"– row and col identify their location in the grid– generation identifies the "tick nr" in evolution– alive identifies the state of the cell• the collection of all cells is the CellTopic Topic– cells exist side-by-side and for the Universe– conceptually stored "in the data-space"– in reality, local copies where needed
  20. 20. row: 16col: 4generation: 25alive: false
  21. 21. Applying DDS to Life DistributedEach process is responsible for publishing thestate of a certain subset of cells of the Universe:• a rectangle or square area with corners(rowmin,colmin)i and (rowmax,colmax)i for process i
  22. 22. (1,1) – (10,10)
  23. 23. Applying DDS to Life DistributedEach process is responsible for publishing thestate of a certain subset of cells of the Universe:• a rectangle or square area with corners(rowmin,colmin)i and (rowmax,colmax)i for process i• each cell is individually updated using thewrite() call on a CellTopic DataWriter– middleware analyzes the key values (row,col) andmaintains the individual states of all cells• updating happens generation by generation
  24. 24. Applying DDS to Life DistributedEach process subscribes to the required subsetof cells in order to determine its current state:• all neighboring cells, as well as its "own" cells
  25. 25. Applying DDS to Life DistributedEach process subscribes to the required subsetof cells in order to determine its current state:• all neighboring cells, as well as its "own" cells• using a SQL-expression to identify the cellssubscribed to (content-based filtering)– complexity is "Life-specific", not "DDS-specific"
  26. 26. "((row >= 1 AND row <= 11) OR row = 20) AND((col >= 1 AND col <= 11) OR col = 20)"
  27. 27. Applying DDS to Life DistributedEach process subscribes to the required subsetof cells in order to determine its current state:• all neighboring cells, as well as its "own" cells• using a SQL-expression to identify the cellssubscribed to (content-based filtering)– complexity is "Life-specific", not "DDS-specific"• middleware will deliver cell updates to thoseDataReaders that are interested in it
  28. 28. Applying DDS to Life DistributedAdditional processes can be added to peek atthe evolution of Life:• subscribing to (a subset of) the CellTopic
  29. 29. "row >= 8 AND row <= 13 AND col >= 8 AND col <= 13"
  30. 30. Applying DDS to Life DistributedAdditional processes can be added to peek atthe evolution of Life:• subscribing to (a subset of) the CellTopic• using any supported language, OS, platform– C, C++, Java, C#, Ada– Windows, Linux, AIX, Mac OS X, Solaris,INTEGRITY, LynxOS, VxWorks, QNX…• without changes to the existing applications– middleware discovers new topology anddistributes updates accordingly
  31. 31. Agenda• Problem definition: Life Distributed• A solution: RTI Connext DDS• Applying DDS to Life Distributed: concepts• Applying DDS to Life Distributed: (pseudo)-code• Advanced Life Distributed: leveraging DDS• Summary• Questions and Answers
  32. 32. Life Distributed (pseudo-)codeLife Distributed prototype applications weredeveloped on Mac OS X• Life evolution application written in C• Life observer application written in Python– using Pythons extension-API• (Pseudo-)code covers basic scenario only– more advanced apects are covered in next section
  33. 33. Life Distributed (pseudo-)codeLife evolution application written in C:• application is responsible for– knowing about the Life seed (initial state of cells)– executing the Life rules based on cell updatescoming from DDS– updating cell states after a full generation tick hasbeen processed• evolution of Life takes place one generation ata time– consequently, Life applications run in "lock-step"
  34. 34. initialize DDScurrent generation = 0write sub-universe Life seed to DDSrepeatrepeatwait for DDS cell update forcurrent generationupdate sub-universe with celluntil 8 neighbors seen for all cellsexecute Life rules on sub-universeincrease current generationwrite all new cell states to DDSuntil last generation reached
  35. 35. Life Distributed (pseudo-)codeWorth to note about the Life application:• loss of one cell-update will eventually stall thecomplete evolution– this is by nature of the Life algorithm– implies RELIABLE reliability QoS for DDS– history of 2 generations need to be stored to avoidoverwriting
  36. 36. <dds><qos_library name="GameOfLifeQosLibrary"><qos_profile name="CellProfile"><topic_qos><reliability><kind>RELIABLE_RELIABILITY_QOS</kind></reliability><history><kind>KEEP_LAST_HISTORY_QOS</kind><depth>2</depth></history><durability><kind>TRANSIENT_LOCAL_DURABILITY_QOS</kind></durability></topic_qos></qos_profile></qos_library></dds>
  37. 37. Life Distributed (pseudo-)codeWorth to note about the Life application:• loss of one cell-update will eventually stall thecomplete evolution– this is by nature of the Life algorithm– implies RELIABLE reliability QoS for DDS– history of 2 generations needs to be stored to avoidoverwriting of state of a single cell• startup-order issues resolved by DDS durability QoS– newly joined applications will be delivered current state– delivery of historical data transparent to applications– applications not waiting for other applications, but for cellupdates
  38. 38. <dds><qos_library name="GameOfLifeQosLibrary"><qos_profile name="CellProfile"><topic_qos><reliability><kind>RELIABLE_RELIABILITY_QOS</kind></reliability><history><kind>KEEP_LAST_HISTORY_QOS</kind><depth>2</depth></history><durability><kind>TRANSIENT_LOCAL_DURABILITY_QOS</kind></durability></topic_qos></qos_profile></qos_library></dds>
  39. 39. Life Distributed (pseudo-)codeWorth to note about the Life application:• DDS cell updates come from different places– mostly from the applications own DataWriter– also from neighboring sub-Universes DataWriters– all transparently arranged based on the filter
  40. 40. create DDS DomainParticipantwith DomainParticipant, create DDS Topic "CellTopic"with CellTopic and filterexpression, create DDSContentFilteredTopic "FilteredCellTopic"create DDS Subscribercreate DDS DataReader for FilteredCellTopic
  41. 41. Life Distributed (pseudo-)codeWorth to note about the Life application:• DDS cell updates come from different places– mostly from the applications own DataWriter– also from neighboring sub-Universes DataWriters– all transparently arranged based on the filter• algorithm relies on reading cell-updates for asingle generation– evolving one tick at a time– leverages DDS QueryCondition– "generation = %0" with %0 value changing
  42. 42. create DDS DomainParticipantwith DomainParticipant, create DDS Topic "CellTopic"with CellTopic and filterexpression, create DDSContentFilteredTopic "FilteredCellTopic"with DomainParticiapnt, create DDS Subscriberwith Subscriber and FilteredCellTopic, create DDSCellTopicDataReaderwith CellTopicDataReader, query expression andparameterlist, create QueryConditionwith DomainParticipant, create WaitSetattach QueryCondition to WaitSetin main loop:in generation loop:block thread in WaitSet, wait for data from DDSread with QueryCondition from CellTopicDataReaderincrease generationupdate query parameterlist with new generation
  43. 43. Life Distributed (pseudo-)codeLife observer application written in Python:• application is responsible for– subscribing to cell updates– printing cell states to display evolution– ignoring any generations that have missing cellupdates
  44. 44. import clifedds as life#omitted option parsingfilterString = row>={} and col>={} and row<={} andcol<={}.format(options.minRow, options.minCol,options.maxRow, options.maxCol)life.open(options.domainId, filterString)generation = 0while generation is not None:# read from DDS, block if nothing availble,# returns Nones in case of time-out after 10 secondsrow, col, generation, isAlive = life.read(10)# omitted administration for building and printing stringslife.close()
  45. 45. Life Distributed (pseudo-)codeLife observer application written in Python:• application is responsible for– subscribing to cell updates– printing cell states to display evolution– ignoring any generations that have missing cellupdates• for minimal impact, DataReader uses default QoSsettings– BEST_EFFORT reliability, so updates might be lost– VOLATILE durability, so no delivery of historicalupdates– still history depth of 2
  46. 46. <dds><qos_library name="GameOfLifeQosLibrary"><qos_profile name="CellProfile"><topic_qos><history><kind>KEEP_LAST_HISTORY_QOS</kind><depth>2</depth></history></topic_qos></qos_profile></qos_library></dds>
  47. 47. Life Distributed (pseudo-)codedomainId# of generationsuniverse dimensionssub-universe dimensionsExample of running a single life application:
  48. 48. Life Distributed (pseudo-)codeExample of running a life scenario:
  49. 49. Agenda• Problem definition: Life Distributed• A solution: RTI Connext DDS• Applying DDS to Life Distributed: concepts• Applying DDS to Life Distributed: (pseudo)-code• Advanced Life Distributed: leveraging DDS• Summary• Questions and Answers
  50. 50. Fault toleranceNot all is lost if Life application crashes• if using TRANSIENT durabilityQoS, infrastructure will keep status roaming– requires extra service to run (redundantly)• after restart, current status is availableautomatically– new incarnation can continue seamlessly
  51. 51. persistenceservice
  52. 52. Fault toleranceNot all is lost if Life application crashes• if using TRANSIENT durability QoS,infrastructure will keep status roaming– requires extra service to run (redundantly)• after restart, current status is availableautomatically– new incarnation can continue seamlessly• results in high robustness• even more advanced QoS-es are possible
  53. 53. Reliability and flow controlRunning the Python app with a larger grid:• with current QoS, faster writer with slowerreader will overwrite samples in reader• whenever at least one cell update ismissing, the generation is not printed (bydesign)
  54. 54. Reliability and flow controlRunning the Python app with a larger grid:• whenever at least one cell update is missing,the generation is not printed (by design)• with current QoS, faster writer with slowerreader will overwrite samples in reader• this is often desired result, to avoid system-wide impact of asymmetric processing power• if not desired, KEEP_ALL QoS can be leveraged
  55. 55. <dds><qos_library name="GameOfLifeQosLibrary"><qos_profile name="CellProfile"><topic_qos><reliability><kind>RELIABLE_RELIABILITY_QOS</kind></reliability><history><kind>KEEP_ALL_HISTORY_QOS</kind></history><resource_limits><max_samples_per_instance>2</max_samples_per_instance></resource_limits><durability><kind>TRANSIENT_LOCAL_DURABILITY_QOS</kind></durability></topic_qos></qos_profile></qos_library></dds>
  56. 56. Reliability and flow controlRunning the Python app with a larger grid:• whenever at least one cell update is missing,the generation is not printed (by design)• with current QoS, faster writer with slowerreader will overwrite samples in reader• this is often desired result, to avoid system-wide impact of asymmetric processing power• if not desired, KEEP_ALL QoS can be leveraged– flow control will slow down writer to avoid loss
  57. 57. More advanced problem solvingOther ways to improve the Life implementation:• for centralized grid configuration, distribute grid-sizes with DDS– with TRANSIENT or PERSISTENT QoS– this isolates configuration-features to one single app– dynamic grid-reconfiguration can be done by re-publishing grid-sizes• for centralized seed (generation 0)management, distribute seed with DDS– with TRANSIENT or PERSISTENT QoS– this isolates seeding to one single app
  58. 58. More advanced problem solvingOther ways to improve the Life implementation:• in addition to separate cells, distribute complete sub-Universe state using a more compact data-type– DDS supports a very rich set of data-types• bitmap-like type would work well– especially useful for very large scale Universes– can be used for seeding as well– with TRANSIENT QoS• multiple Universes can exist and evolve side-by-side usingPartitions– only readers and writers that have a Partition in common willinteract– Partitions can be added and removed on the fly– Partitions are string names, allowing good flexibility
  59. 59. Agenda• Problem definition: Life Distributed• A solution: RTI Connext DDS• Applying DDS to Life Distributed: concepts• Applying DDS to Life Distributed: (pseudo)-code• Advanced Life Distributed: leveraging DDS• Summary• Questions and Answers
  60. 60. SummaryDistributed Life can be properly implementedleveraging DDS• communication complexity is off-loaded tomiddleware, developer can focus on application• advanced QoS settings allow for adjustment torequirements and deployment characteristics• DDS features simplify extending Distributed Lifebeyond its basic implementation• all of this in a standardized, multi-language, multi-platform environment with an infrastructure builtto scale and perform
  61. 61. Agenda• Problem definition: Life Distributed• A solution: RTI Connext DDS• Applying DDS to Life Distributed: concepts• Applying DDS to Life Distributed: (pseudo)-code• Advanced Life Distributed: leveraging DDS• Summary• Questions and Answers
  62. 62. Questions?Thanks!

×