!Chaos Control System


Published on

The !CHAOS project is aiming at the development of a new concept of control system and data acquisition framework providing, with a high level of abstraction, all the services needed for controlling and managing a large scientific, or non-scientific, infrastructure.

Published in: Technology
1 Like
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

!Chaos Control System

  1. 1. Control  system  based  on  a Highly Abstracted  and Open Structure school  2012 http://chaos.infn.it C.  Bisegnivenerdì 14 dicembre 12 1
  2. 2. the  scene... imagine  to  have  this...venerdì 14 dicembre 12 2
  3. 3. the  scene... with  many   distributed controls  roomsvenerdì 14 dicembre 12 3
  4. 4. the  scene... with  many   distributed data  centervenerdì 14 dicembre 12 4
  5. 5. the  scene... with  only  ONE Control  System  with   complex  automated  taskvenerdì 14 dicembre 12 5
  6. 6. the  scene... somewhere  in  the  future   perhaps  will  have  thisvenerdì 14 dicembre 12 6
  7. 7. the  scene... with  only  ONE  Control  System  with   even  more  complex  automated  taskvenerdì 14 dicembre 12 7
  8. 8. the  scene... securely,  there   will  be  a  lot  of   chaosvenerdì 14 dicembre 12 8
  9. 9. the  scene... but  only  ONE !CHOAS instancevenerdì 14 dicembre 12 9
  10. 10. Objec>ve What  is  needed  to  control  the  CHAOTIC   complexity  of  big  apparatus?venerdì 14 dicembre 12 10
  11. 11. Objec>ve What  is  needed  to  control  the  CHAOTIC   complexity  of  big  apparatus? •  a  design  that  permits  to  scale  to  achieve  high  speed  in  the  data   acquisi@on  and  elabora@on  and  for  fault  tolerant; •  possibility  to  aBach  and  detach  an  en@re  subsystem; •  possibility  to  “configure”  a  specified  instrument  without  halt  the   full  system; •  possibility  to  hot  insert  an  instrument  or  a  subsystem;venerdì 14 dicembre 12 11
  12. 12. Objec>ve What  is  needed  to  control  the  CHAOTIC   complexity  of  big  apparatus? •  give  the  flexibility  to  implements  control  algorithm  on  abstracted   instrument  class; •  work  on  data  abstrac@on,  without  fixed  schema; •  give  a  data  system  that  permit  to  monitor  the  instrument   channels  and  query  the  history    instruments  data; •  the  data  can  be  managed  into  domains •  every  domain  can  be  implemented  with  different  technologyvenerdì 14 dicembre 12 12
  13. 13. Objec>vevenerdì 14 dicembre 12 13
  14. 14. What  is !CHAOS  (Control  system  based  on  Highly  Abstracted  and  Open  Structure),   is   an  experiment  of   CSN  5(technological  research  experiments)  at  the  LNF  and  Roma-­‐TV  sec@on. The  project  has  been  appreciated   by   educa@onal   world  and  the  University  of   Tor  Vergata   and   Cagliari  have  joined  the  development  team. Università TorVergata LNF - Roma2 Università di Cagliarivenerdì 14 dicembre 12 14
  15. 15. Objec>ve !CHAOS   has   been   designed   to   widely   scale   and   for   manage   large   apparatus  with  different  needs  in  term  of  storage  technology  and  speed   for  data  acquisi@onvenerdì 14 dicembre 12 15
  16. 16. Objec>ve •  new  topology  in  the  CS  that  permit  to  integrate  the  triggered  DAQ; •  define  a  new  way  to  access  real@me  data; •  data  acquisi@on,  storage  and  indexing  fully  scalable; •  high  scalability  in  every  cri@cal  point.venerdì 14 dicembre 12 16
  17. 17. Objec>ve A  CPP  Framework  and  services  that  permit  to  have: •  99%  of  adapta@on  @me  of  instrument  to  !CHOAS  done  by  framework •  simplify  the  crea@on  of  the  user  interfaces   •  simplify  the  crea@on  of  the  control  algorithms  basing  it  on  an  abstrac@on   of  the  I/O  channels  or  full  dataset  not  on  physical  device •  All  service  for  data  management  and  monitoring  already  done  and  fully   customizablevenerdì 14 dicembre 12 17
  18. 18. !CHAOS  Simplify  Vision 1venerdì 14 dicembre 12 18
  19. 19. Simplify  Vision User  specialized   Boundary Implementation CHAOS  Boundary Controls Algorithm Abstract RPC Driver Implementation Implementation Instruments Abstract network User Interface Driver Event Driver Abstract Direct I/O Live Data Live Data Live Data Directory Data Cloud Data Cloud Server Data Cloud Cloud History History Data Data History Data Implementation Implementation Implementationvenerdì 14 dicembre 12 19
  20. 20. !CHAOS  Real  Model 2venerdì 14 dicembre 12 20
  21. 21. User  CustomizaIon   User Interface for monitoring and controls Boundary AbstracIon   Boundary User Interface Toolkit Common Toolkit RPC - Event - Direct I/O Data  Boundary InformaIon   CQL Proxy Nodes Execution Unit Toolkit Boundary RPC - Event - Direct I/O Indexer Nodes Node Common Toolkit Management Nodes Information HTTP - API Live / History Security domainControls NodeAlgorithm Activity Chaos Live / History Directory domain Nodes RPC - Event - Direct I/O Common Toolkit Control Unit Toolkit Driver for sensor and instrumentsvenerdì 14 dicembre 12 21
  22. 22. Toolkit  and  Services !CHAOS  is  a  set  of  toolkit  and  services  that  will  permit  the   realizaAon  of  an  control  systemvenerdì 14 dicembre 12 22
  23. 23. !CHAOS  Toolkit  and  Services Common  Toolkit Directory Services CU  Toolkit Live  and  History  Data UI  Toolkit CQL  Proxy EU  Toolkit Indexer Managmentvenerdì 14 dicembre 12 23
  24. 24. !CHAOS  Toolkit implements   the   fundamental   soZwares   layer   to   Common  Toolkit RPC,   I/O,   Event   and   other’s   u@lity   code   and   for   logic abstracts   the   !CHAOS   resources   to   the   device   CU  Toolkit drivers  developers. abstracts  the  CHAOS  resources  to   the  developers   UI  Toolkit that   need  to  develop  control  and  monitoring  user   interface abstracts  the  CHAOS  resources  to   the  developers   EU  Toolkit that   need   to   develop   control   and   compu@ng   algorithmsvenerdì 14 dicembre 12 24
  25. 25. !CHAOS  Services Directory gives   services   via   RPC   (or   with   other   higher   transport   layers)   for   get   device   structured   Services informa@on,  system  topology  etc... give  the   access   to  insert   and  query   opera@on  on   CQL  Proxy live  and  history  data  to   other   CHOAS   node  using   CHOAS  Query  Language  (CQL) Indexer performs  the  indexing  of  the  new  inserted  data performs   the   management   of   the   datas   chunk   Management saved  on  the  storage  (erase,  move,  etc.)venerdì 14 dicembre 12 25
  26. 26. !CHOAS  Scalability  and  Fault-­‐Tolerant Some  node  needs  to  be  scaled  for  performance  reason  or  for  fault-­‐tolerant: •  Directory  Nodes •  Live  and  History  nodesvenerdì 14 dicembre 12 26
  27. 27. !CHOAS  Scalability  and  Fault-­‐Tolerant Server P1 Server P2 Framework Device 1 Driver X X Server P1 Server PN X Server P1 Server P2 Client Reader Server P2 Driver Server PN Framework Clients  p  address  has  a  priority,  the  ervices,  has  more  than  one  ip  address  ith   Every  i that  need  to  talk  to  scaled  s client,  as  first  address,  uses  the  one  wfor   when  a  server  goes  down,  another  ip  will  be  used highest  priority every  servicevenerdì 14 dicembre 12 27
  28. 28. !CHOAS  Scalability  and  Fault-­‐Tolerant Server P1 Server P2 Framework use at ok Device 1 Server P1 Driver TS1 Server PN use at Server P1 TS1 Server P2 Client Reader Server P2 Driver Server PN Framework the  trhe  operaIons  and  then  alculaIon  of  Ta  highest  priority,  previously  gtoes   all   e-­‐balance  occurs  w he  c a  server  with   S1  are  coordinated  by  one  of   he   the  new  server  will  be  used  starIng  from  a  pre-­‐determinated  Ime  stamp  (TS1) down  or  stopped  for  maintenance,  becomes  up  and  running directory  nodesvenerdì 14 dicembre 12 28
  29. 29. !CHAOS  Abstrac>on  of  the  Data 11/dic/2011 2venerdì 14 dicembre 12 29
  30. 30. Abstrac>on !CHAOS  use  a  structured  serializa@on  for  data  descrip@on  and   serializa@on •  Data  is  managed  by  CDataWrapper  cpp  object  that  wrap  the   serializa@on;   •  BSON  (hBp://bsonspec.org/)  has  been  chosen,  very  similar  to  JSON   but  it  is  binary; •  serializa@on  is  wrapped  into  a  CPP  class  so  it  can  be  changed  in   future;venerdì 14 dicembre 12 30
  31. 31. Abstrac>on  it  is  used  in  (for  now): •Hardware  dataset  descrip@ons  and  acquired  data  packaging   (“channel  desc”  =  value) •RPC  message •Chaos  Query  Language •  Directory  Server  communicaIonvenerdì 14 dicembre 12 31
  32. 32. Abstrac>on  for  Hardware  Dataset •  Hardware  aBributes  and  I/O  channels  are  described  into  the  DATASET •  Each  aBribute  is  defined  by •  Name •  Descrip@on •  Type  (type  ex:  INT32,  INT64,  DOUBLE,  STRING,  BINARY,  STRUCT) •  Cardinality  (single  or  array) •  Flow  (Input,  Output,  Bidirec@onal) •  Range •  default  value •  other  will  camevenerdì 14 dicembre 12 32
  33. 33. Hardware  Dataset DATASET name:VOLT_CHANNEL_1 type:VOLT32 flow:output range:1-­‐20 card:  1 VOLT_CHANNEL_1 name:VOLT_CHANNEL_2 RESET type:VOLT32 flow:output range:1-­‐20 VOLT_CHANNEL_2 card:  1 name:  RESET type:  BYTE flow:  input card:  1 11/dic/2011venerdì 14 dicembre 12 33
  34. 34. Hardware  DATASET The  JSON  visualiza>on   of  BSON  DS  representa>on {“device_id”: DEVICE_ID VOLT_CHANNEL_1   “device_ds”: { RESET "name" : "VOLT_CHANNEL_1", "desc" : "Output volt...", "attr_dir" : 1, "type" : 4, "cardinality" : 1 VOLT_CHANNEL_2 ! } .... } 11/dic/2011 3venerdì 14 dicembre 12 34
  35. 35. !CHAOS  Communica>on  Systems 11/dic/2011 3venerdì 14 dicembre 12 35
  36. 36. Communica>on  Systems   !CHOAS  has  three  different  communicaIon  system: RPC,  used  for  command  (messages  that  need  to  be  securely  received  and   recorded  and  for  message  that  need  a  response  [  asynchronous  ]) Events,  used  for  inform  to  a  wide  range  of  nodes,  that  something  has   happened  (device  heartbeat,  alert  on  device  channel,  received  command,   etc..) Direct  I/O,  used  for  streams  to  and  from  the  live,    historical  data  or  inter   node  data  exchangevenerdì 14 dicembre 12 36
  37. 37. Communica>on  Systems   Every  !CHAOS  node  has  at  last  one   client  and  server  instance  of  any  of   the  three  systemvenerdì 14 dicembre 12 37
  38. 38. RPC  System  •RPC  layer  is  implemented  as  a  plug-­‐ins  System,  its  is  abstracted  to  internal   CHAOS  framework  so  we  con  change  it•  used  to  send  important  messages  that  need  to  be  sure  of  its  delivering   status•  can  be  used  to  receive  asynchronous  responses•  the  set  command  for  input  channel  on  control  unit  is  done  with  RPCvenerdì 14 dicembre 12 38
  39. 39. RPC  System   RPC  Message  Flow •  request  message  is  constructed  with  BSON,  serialized  and  sent  to  RPC   end  point •  if  the  requester  want  a  response  it  put  answer-­‐code  and  sender-­‐ address  in  BSON  pack,and  automa@cally  RPC  System  wait  for  the  answer •  the  request  and  response  involves  the  forwarding  of  the  ack  message,   from  the  receiver  to  the  sender  in  synchronous  mode Message Answer  [opIonal] Messagge forwarding Response forwarding Node A Node B Node A Node B ack [sync] ack [sync]venerdì 14 dicembre 12 39
  40. 40. !CHAOS  rpc  system •  Two  kind  of  data  forwarding: •Message,  the  informaBon  are  sent  to  remote  host •Request,  the  informaBon  are  sent  to  remote  host,  and  an  answer  is   awaited •In  either  types  of  data  forwarding,  a  result  of  submissions  is  sent  to  the   client  when  a  server  has  received  the  datavenerdì 14 dicembre 12 40
  41. 41. RPC  ac>on  message •In  either  types  of  data  forwarding,  the  data  informaBon  are  stored  into  a   k/v  message: •NODE_DST,  internal  MessageBroker  address[string] •ACT_ALIAS_DST,  alias  for  the  call  acBon  on  the  node[string] •ACT_MSG,  k/v  object  used  as  param  for  the  acBon[BSON] BSON Document for the message “NODE_DST”:  “3b95d9b5” “ACT_ALIAS_DST”:  “alias” “ACT_MSG”:  messagevenerdì 14 dicembre 12 41
  42. 42. RPC  ac>on  request the  request,  add  some  other  informaBon  to  the  message BSON Document for the request message “NODE_DST”:  “3b95d9b5” layer “ACT_ALIAS_DST”:  “alias” “ACT_MSG”:  message “ANSW_ID”:  1 request “ANSW_NODE”:  “44342364” layer “ANSW_ACT”:  “alias” “ANSW_IP”:  “ip:port”venerdì 14 dicembre 12 42
  43. 43. !CHAOS  submessage •  a  message  can  bring  a  sub  message •  the  sub  message  is  dispatched  aLer  the  main  message  has  been   successfully  executedvenerdì 14 dicembre 12 43
  44. 44. !CHAOS  submission  result •the  message  or  request  submission  is  a  synchronous  operaBon •aLer  received  the  server  send  back  to  the  client  the  result,  wrapped  into   a  BSON  object  with  these  key: •SUBMISSION_ERROR_CODE,  integer  number  that  represents  the   error  code[0  mean  no  error] •SUBMISSION_ERROR_MESSAGE,  string  that  represents  what  is   happened •SUBMISSION_ERROR_DOMAIN,  string  that  represents  the  locaBon   where  the  error  is  occurred “SUBMISSION_ERROR_CODE”:  1 submission result “SUBMISSION_ERROR_MESSAGE”:  “error  string” “SUBMISSION_ERROR_DOMAIN”:  “class::method”venerdì 14 dicembre 12 44
  45. 45. !CHAOS  answer  to  a  request •the  answer  to  a  request,  is  composed  by: •  the  error  fields  shown  in  previous  slides,  in  case  of  error  there  will   be  all  tree •  the  BSON  object  that  represents  the  answer,  in  case  of  success “SUBMISSION_ERROR_CODE”:  1 answer result “ACT_MSG”:  Result  BSON  Docvenerdì 14 dicembre 12 45
  46. 46. Event  System   •  Event  is  implemented  as  a  plug-­‐ins  System  its  is  abstracted  to  internal   CHAOS  framework  so  we  con  change  it •  there  are  four  type  of  event: •  Alert  (highest  priority) •  Instrument •  Command •  Custom  (user  specific  event  type) •  all  informaAon  are  packed  into  1024  byte  and  transported  by  protocols   that  support  mulAcast  (UDP,  PGM[hVp://en.wikipedia.org/wiki/ PragmaAc_General_MulAcast]) 09/dic/2011venerdì 14 dicembre 12 46
  47. 47. Event  System   Some  numbers  about  event  data •  Every  event  type  has  2^16  sub  type •  Every  sub  type  can  have  up  to  2^16  priority  level •  Every  subtype  can  have  up  to  255  byte  of  informaAon 09/dic/2011venerdì 14 dicembre 12 47
  48. 48. Direct  I/O  System   •  Direct  I/O  is  implemented  as  a  plug-­‐ins  system •  used  to  make  insert  or  select  into  the  live  and  the  history  data •  can  be  used  as  stream  for  receive  result  data  of  the  query  or  read  the   stored  raw  data  file  of  the  devices •  fastest  way  to  transfer  data  between  node 09/dic/2011venerdì 14 dicembre 12 48
  49. 49. Communica>on  Systems   CommonToolkit,  implementa  a  so_ware  switch  that  permit,  to  all  other   class,  to  access  all  three  protocoll. Action Action Action Handler Answer Event RPC Channel Channel Node Node NetworkBroker RPC Event Direct I/O Client Server Client Server Client Server Networkvenerdì 14 dicembre 12 49
  50. 50. NetworkBroker  For  RPC  and  Event CPP Class RegisterAction Network DeclarAction Broker addActionDescritionInstance RegisterEventAction EventAction virtual void handleEvent(const event::EventDescriptor * const event) = 0;venerdì 14 dicembre 12 50
  51. 51. NetworkBroker  For  RPC  and  Event NetworkBroker SERVER Domain  Queue  is  filled data the  dispatcher find  des>na>on  queue data data ac>on Client data Communication data Implementationvenerdì 14 dicembre 12 51
  52. 52. NetworkBroker  For  RPC  and  Event DeviceMessageChannel MessageChannel Get Message Channel API MDSMessageChannel Network Broker AlertEVTChannel DeviceEVTChannel EventChannel Get Event Channel API CmdEVTChannel CustomEVTChannelvenerdì 14 dicembre 12 52
  53. 53. NetworkBroker  For  RPC  and  Event MessageChannel GetMessageChannel security check for Network who can send what Broker (TODO) EventChannel GetEventChannelvenerdì 14 dicembre 12 53
  54. 54. !CHAOS  ControlUnit  toolkit 11/dic/2011venerdì 14 dicembre 12 54
  55. 55. CU  Toolkit •CUToolkit  help  the  Control  Unit  development •A  ControlUnit  is  and  “Instrument”  driver  that  aBach  an  hardware   to  !CHAOS •Developer  need  to  extend  only  one  class  to  create  a  CHAOS  driver,   the  “AbstractControlUnit” •AbstractControlUnit  expose  all  the  APIs  needed  for  interac@ng   with  !CHOASvenerdì 14 dicembre 12 55
  56. 56. Write  a  Control  Unit AbstractControlUnit  is  an  abstract  cpp  class,  that  force  developer  to   implement  some  method developer  need  to  extends  this  class  to  create  a  !CHOAS  device  driver AbstractControlUnit Control Unit Toolkit •defineAc@onAndDataset Common Toolkit •init •run •stop •deinitvenerdì 14 dicembre 12 56
  57. 57. Write  a  Control  Unit defineAcIonAndDataset,  is  the  method  where  the  developer  needs,   using  api,  describe  the  aBached  device. Add  a  new  device  iden@fica@on  to  the  control  unit void addDeviceId(string); Add  a  global  default  scheduling  rate  for  run  method void setDefaultScheduleDelay(int32_t); Add  a  new  aBribute  to  the  device  dataset void addAttributeToDataSet(const char*const deviceID, const char*const attributeName, const char*const attributeDescription, DataType::DataType attributeType, DataType::DataSetAttributeIOAttribute attributeDirection);venerdì 14 dicembre 12 57
  58. 58. Write  a  Control  Unit Add  a  new  input  aBribute  of  the  type  XXXX  to  the  device  dataset  giving  ha   handler  for  manage  it template<typename T> void addInputXXXXAttributeToDataSet(const char*const deviceID, const char*const attributeName, const char*const attributeDescription, T* objectPointer, typename TDSObjectHandler<T, XXXX>::TDSHandler objectHandler) The  handler  is  every  method  of  an  class  that  respect  this  prototype: typedef void (T::*TDSHandler)(const std::string & , const XXXX&);venerdì 14 dicembre 12 58
  59. 59. Write  a  Control  Unit A  device  driver  can  also  define  custom  ac@on template<typename T> AbstActionDescShrPtr addActionDescritionInstance(T* actonObjectPointer, typename ActionDescriptor<T>::ActionPointerDef actionHandler, const char*const actionAliasName, const char*const actionDescription); An  ac@on  is  every  method  of  a  class  that  respect  this  prototype: typedef CDataWrapper* (T::*ActionPointerDef)(CDataWrapper*, bool&);venerdì 14 dicembre 12 59
  60. 60. Write  a  Control  Unit init,  is  the  method  where  the  developer  needs  to  ini@alize  his  hardware. The  local  dataset  is  filled,  from  controlling  client,  with  the  current  default   value,  for  every  channel  or  custom  property.  The  developer  can  read   these  value  and  setup  some  useful  things.venerdì 14 dicembre 12 60
  61. 61. Write  a  Control  Unit run,  is  the  method  where  the  developer  needs,  using  api,  check  the  state   of  the  device  and  acquire  the  value  of  output  channel,  or  make  control   logic All  the  data  acquired  in  the  driver,  that  are  associated  to  the  output   channels,  need  to  be  insert  into  a  CDataWrappedObject  to  be  published Return  a  new  CDataWrapper  prepared  to  be  push  on  data  output  queue CDataWrapper *getNewDataWrapperForKey(const char*); The  data  is  filled  with  the  default  chaos  key  for:  @mestamp,  deviceID,  triggerCode Send  a  data  buffer  to  the  output  buffer  for  a  determinated  device  id void pushDataSetForKey(const char *deviceID, CDataWrapper*);venerdì 14 dicembre 12 61
  62. 62. Write  a  Control  Unit stop,  inform  the  developer  that  has  been  stopped  the  device deinit,  inform  the  developer  that  has  been  requested  the  deini@aliza@on   of  the  device.  So  all  resource  of  the  hardware  need  to  be  deallocated.venerdì 14 dicembre 12 62
  63. 63. Write  a  Control  Unit summarizing •  A  CU  is  a  C++  class.   •  CUToolkit  help  to  create  a  deamon,  and  take  care  to: •  talk  with  MDS  for  registra@on; •  make  periodical  check  for  network(s@ll  in  development)   availability; •  take  care  to  configure  all  driver  with  all  the  ip  for  the  server  that   will  manage  the  acquired  data; •  other  secret  things...venerdì 14 dicembre 12 63
  64. 64. Write  a  Control  Unit A  typical  CUToolkit  main  for  a  CU,  is  the  code  below: int main (int argc, char* argv[] ) { // Initialize the CUToolkit ChaosCUToolkit::getInstance()->init(argc, argv); // Add a new control unit identified by the class WorkerCU ChaosCUToolkit::getInstance()->addControlUnit(new WorkerCU()); // Start the all the server and client for RPC and execute all // work to startup and register the instance of WorkerCU class // and wait an shutdown command via RPC or SIGINT signal ChaosCUToolkit::getInstance()->start(); return 0; } the  method ChaosCUToolkit::getInstance()->addControlUnit(...) can be called for other instance of WorkerCU Class or other control unitvenerdì 14 dicembre 12 64
  65. 65. !CHAOS  UI  Toollkit 11/dic/2011 6venerdì 14 dicembre 12 65
  66. 66. UI  Toolkit UIToolkit  helps  the  user  interface  development Gives   API   that   help   to   develop:   monitoring   processes,   device   controller  etc... Gives   an   high   performance   queue   using   "locking   free"   design.     The   queue   can   manage   quan@za@ons   ,   over   @me,   for   the   dataset   data   acquisi@on UIToolkit DeviceControl DeviceControl InterThread Cache Logic Live Acquisition Thread Pool AbsractIO AbsractIO Driver Driver Networkvenerdì 14 dicembre 12 66
  67. 67. UI  Toolkit At  the  moment  the  whole  package  of  UI  is  in  study  but  some  things   has  been  done  to  help  developer  to  control  hardware. The  DeviceController  class  can  help  developer  to  connect  to  a   specific   device  using  the  IDvenerdì 14 dicembre 12 67
  68. 68. UI  Toolkit DeviceID MDSChannel Directory  Server DeviceController DeviceChannel EventHandler Live Data Driver Memcached   Instances (for  now)venerdì 14 dicembre 12 68
  69. 69. UI  Toolkit A  “c”  interface  is  provide  to  access  UIToolkit  into  sofware  that  don’t   permit  to  use  C++  as  developer  language. This   permit   to   realize   an   user   interface   or   controller   algorithm   into   another  soZware  as  plugin,  as  for  example  LabView. The   “C”   proxy   has   a   minimal   set   of   interface   to   simplify   the   device   controller  alloca@on,  dealloca@on  and  usage.venerdì 14 dicembre 12 69
  70. 70. !CHAOS  EU  Toolkit 11/dic/2011 7venerdì 14 dicembre 12 70
  71. 71. EU  Toolkit helps  the  development  of  distributed  algorithms  (Execu@on  Unit)   that  can  be  used  to  compute  or  automate  the  devices  control The   EU  can  have,   in   input   the   device   channels   or   all   the  Dataset   descrip@on,  so  it  work  on  a   "Class"  of  device  not  directly   on  one   device. The   EU   can   be   also   used   to   perform   calcula@on   instead   control   algoritm; on   EU   can   be   preset   in   many   instance   distributed   on   different   fiscal  servervenerdì 14 dicembre 12 71
  72. 72. EU  Toolkit scheme  of  flow  of  input  or  output  of  an  EU Specialized Execution Unit Execution Abstract Execution Unit Result Queue Queue Input Parameter Output Parameter Input Param Pack Output Result Packvenerdì 14 dicembre 12 72
  73. 73. EU  Toolkit excep@on  could  be  concatenated  crea@ng  a  sequence  of  any  combina@on   of: •  CompuIng  algorithm •  Control  algorithm Specialized Execution Unit Execution Abstract Execution Unit Result Queue Queue Input Parameter Output Parameter Input Param Pack Output Result Pack Input Param Pack Specialized Execution Unit Execution Abstract Execution Unit Result Queue Queue Input Parameter Output Parametervenerdì 14 dicembre 12 73
  74. 74. Execu>on  Unit  Example  1 Thinks  about  an  EU  that  make  some  logic  on  CU  data  and  control  another  CU The  EU  read  the  needed  data  from  the  Output  aBribute  of  the  HW  from  the     CU_1  data The  EU  write  the  result  of  computa@on  to  a  CU_2  for  set  the  Input  aBribute   of  the  HW CU_1 CU_2 14/dic/2012 74venerdì 14 dicembre 12 74
  75. 75. Execu>on  Unit  Example  1 this  is  the  real  data  flow Ouput  aBribute  data  is The  ABribute  is  set  on  HW read  from  device some  logic  is  done by  CU_2 CU_1 CU_2 Output  aBribute  data   The  Input  ABribute  on  CU_2 is  pushed is  set  with  RPC  command on  live  memory  block from  CU_1 Output  CU_1  aBribute data  is  read  from  live CU_1  DATA  BLOCK live  data 14/dic/2012 75venerdì 14 dicembre 12 75
  76. 76. Execu>on  Unit  Example  2 the  idea  is  to  create  a  simil-­‐language  for  describe  a  “program”  that  can  be   executed  by  a  “sequence”  of  EU; Now  think  to  an  execu@on  unit  that  send   output  to  another  execu@on  unit  and  so  on... 14/dic/2012 76venerdì 14 dicembre 12 76
  77. 77. !CHAOS  Data  Management 11/dic/2011 7venerdì 14 dicembre 12 77
  78. 78. Data  Management !CHOAS  has  two  different  management  system  for  the  data  apparatus   controlled  environment  data Live  Data History  Datavenerdì 14 dicembre 12 78
  79. 79. !CHAOS  Live  Data ui_1 push data live dev_1 ui_2 cache eu_1 11/dic/2011 7venerdì 14 dicembre 12 79
  80. 80. Live  Data The  system  of  the  Live  Data,  is  ideated  for  the  sharing  of  the  real  @me   state  of  the  instruments  (CU)  to  other  nodesvenerdì 14 dicembre 12 80
  81. 81. Live  Data 1.  every  node  can  rpushes  the  data  of  central  channels  into  tme[polling] Every  instrument   ead  data  from  the   his  I/O   ache  at  any  R he  shared   cached  through  the  controller  CU 2.  the  nodes  can  register  to  central  cache  for  receive  update[push] ui_1 push data live dev_1 ui_2 cache eu_1venerdì 14 dicembre 12 81
  82. 82. Live  Data scalability  &  reliability ui_1 push data live dev_1 cache ui_2 All  data  of  all  instruments  are   push data distributed  across  many  cache  server   live dev_2 cache eu_1venerdì 14 dicembre 12 82
  83. 83. Live  Data scalability  &  reliability X ui_1 with  the  algorithm  previously  shown,   live dev_1 cache when  one  server  goes  down,  all  client   push data ui_2 that  pushs,  or  read,  data  on  that   server,  use  another  server,  with   push data live dev_2 cache eu_1 highest  priorityvenerdì 14 dicembre 12 83
  84. 84. Live  Data The  usage  of  the  two  modes  permits  to  manage,  more  efficiently,  the   bandwidth  of  the  flows  of  the  real  @me  data  (read  and  write) not  all  nodes  need  real@me  data  at  the  same  speed: •  the  mode  (1)  is  for  monitoring  and  control •  the  mode  (2)  is  for  control  algorithm  that  can’t  lose  packagesvenerdì 14 dicembre 12 84
  85. 85. !CHAOS  History  Data 11/dic/2011 8venerdì 14 dicembre 12 85
  86. 86. Live  &  History  Data ChaosQL  Proxy Proxy CQL Interpreter Abstract History Logic Cache Logic Common Layer Storage Driver Cache Driver Index Driver the  live   data   management  is  merged   into  history   data   management  into   the  ChaosQL  node  with  the  Cache  Drivervenerdì 14 dicembre 12 86
  87. 87. History  Data The   System   for   History   Data   has   been   designed   to   efficiently   store  the  serialized  data  pack,  manage   different  type  of   index  on   demand,  and  managing  data  filevenerdì 14 dicembre 12 87
  88. 88. History  Data It  is  realized  by  three  different  services  and  two  driver ChaosQL  Proxy Indexer  Node Management  Node Storage Management Logic Management Index Logic Abstract History Logic Abstract History Logic Abstract History Logic Storage Driver Storage Driver Storage Driver CQL Interpreter Index Driver Index Driver Index Driver Common Layer Common Layer Common Layer Storage  Driver  abstrac@on Index  Driver  abstrac@on Storage Driver Index Drivervenerdì 14 dicembre 12 88
  89. 89. Chaos  Query  Language  ChaosQL  implement  the  following  operaBon  on  data •  push  history/live  data  per  device •  create  index  on  device::aXribute::{rule} •  delete  index  on  device::aXribute •  retrieve  data  with  logic  operaBon  on  aXribute  and  indexvenerdì 14 dicembre 12 89
  90. 90. History  Data Manages    the  insert  and  select  opera@ons     by  parsing  the  Chaos  Query  Language ChaosQL  Proxy Insert   -­‐  as  fast   as  possible  stores   the   data   Abstract History Logic Storage Driver CQL Interpreter Index Driver into  the  storage  system Common Layer select   -­‐   decodes   the   query,   asks   at   the   indexer   the   posi@on   in   the   storage   of   the   data  that  match  the  criteriavenerdì 14 dicembre 12 90
  91. 91. History  Data UI or CU or EU UI-CU-EU Toolkit Chaos Common All   nodes,   using   the   ChaosQL   Driver   (that   CQL I/O Driver Direct I/O Driver uses   direct   I/O)  can   query   the   history   and   live  data. Query Result Proxy CQL Interpreter the   results   are   received   in   asynchronous   Abstract History Logic mode Common Layer Storage Driver Index Drivervenerdì 14 dicembre 12 91
  92. 92. Storage Store  Data get  data   pack oxy Q L Pr C Launch   Management   Index Store   Cluster Index Indexer   CUTK xy Cluster L Pro CommonTK CQ xy ro LP CQ Push  Data Index  Database 11/dic/2011venerdì 14 dicembre 12 92
  93. 93. Management   Cluster Storage get  pack  at   CUTK locaIon Indexer   CommonTK Cluster oxy Q L Pr C execute  search   on  idx  DB xy Pro Query  Live   CQ L Data send  search   on  idx EUTK Receive   xy CommonTK ro LP History  Data CQ Index  Database UITK Query   CommonTK History  Data 11/dic/2011venerdì 14 dicembre 12 93
  94. 94. History  Data UI or CU or EU the   query  can   be   spliXed   in  more   than   one   Bme   UI-CU-EU Toolkit period  from  the  receiving  proxy.   QueryChannel Chaos Common The   spliXed   query   are   sent   to   other   proxy   that   CQL I/O Driver Direct I/O Driver CQLPack will  process  the  sub  query Original Re ry lt Result sul ue su t Re Q Proxy CQL Interpreter Proxy CQL Interpreter Proxy CQL Interpreter Abstract History Logic Abstract History Logic Abstract History Logic Common Common Storage Storage Common Driver Driver Driver Driver Layer Layer Index Index Storage Driver Driver Layer Index CQLPack CQLPack CQLPack 1 2 3 Sub Queryvenerdì 14 dicembre 12 94
  95. 95. History  Data Executes  the  necessary  opera@ons  to  index   Indexer  Node the  data  saved  on  the  storage  system Management Index Logic Abstract History Logic Storage Driver Index Driver two  type  of  index: Common Layer •  TMI  -­‐  Time  Machine  index •  DSI  -­‐  Dataset  Indexvenerdì 14 dicembre 12 95
  96. 96. History  Data  TMI  -­‐  Time  Machine  index •This  index  permits  to  catalog   every  data  pack   for  a  device  for  the   @mestamp.   •  all  the  data  pack  are  recorded  and  associated  to  their  @mestamp •  this  index  is  managed  by  system  venerdì 14 dicembre 12 96
  97. 97. History  Data  DSI  -­‐  Dataset  Index •This   index   permits   to   catalog   aBributes   of   the   dataset   for   a   determinated  instrument •  there  will  be  possibility  of  customize  the  index  algorithm •  this  index  is  created  on  demand  and  on  specific  @me  periodsvenerdì 14 dicembre 12 97
  98. 98. History  Data Executes   management   opera@ons   on   files   Management  Node that  store  the  data  packs Storage Management Logic Abstract History Logic Storage Driver Index Driver some  data  pack  could  be  subject  to  ageing Common Layer the   ageing   state   will   determine   the   erase   or  long  term  storage  of  a  data  packvenerdì 14 dicembre 12 98
  99. 99. Project  Status C User Interface LV User Interface MatLab UI User Interface Toolkit 20% Common Toolkit distributed distributed Directory Live Data History server infrastructure service Data service Common Toolkit CU Toolkit Control Unit Control Unit I/O Unit Exec Unit CU-LV interface CU modules (config) CU modules (init) CU modules (set) CU modules (de-init) CU modules (stop) I/O modules I/O modules I/O modules EU modules EU modules EU modules CU modules (config) CU modules (init) CU modules (set) CU modules (de-init) CU modules (stop) INFN-­‐Vitrociset  mee@ng 23/02/12venerdì 14 dicembre 12 99
  100. 100. developments’  details QT User Interface: prototype ready (Qt) UI Toolkit: prototype ready C wrapper UI Toolkit: prototype ready LV User Interface: prototype in preparation (shlib for LV) Live Data: single-instance (no-distributed) prototype ready RPC client/server: prototype ready (msgpack and zmq impl) Event client/server: prototype ready (boost asio impl) infrastructure Direct I/O c/s: todo History Data service: performance test (mongodb) ongoing (UniTV) Directory services: DS simulator ready; DS under development scalability & availability: test facility in preparation Common Toolkit: prototype ready CU Toolkit: prototype ready CU modules: prototype ready (C++) CU-LV Interface prototypes & modules: almost ready INFN-­‐Vitrociset  mee@ng 23/02/12venerdì 14 dicembre 12 100
  101. 101. ...closing in   the   next   year   all   development   force   will   be   focused   to   realize   the   system  for  the  history  data    and  for  the   directory  service,  a  prototype  for   each  system  will  be  available  for  2013  2Qvenerdì 14 dicembre 12 101