XReplicator

784 views

Published on

Presentation of the project XReplicator in EMC (26.04.2010). XReplicator is the library and utility provides replication of XAM storage systems.

sourceforge.net/projects/xreplicator/

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
784
On SlideShare
0
From Embeds
0
Number of Embeds
94
Actions
Shares
0
Downloads
2
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

XReplicator

  1. 1. Developers:<br />Burmak AlexanderMirov YuriyKovalev Igor<br />Trainer:<br />Alexeev Peter<br />XAM<br />XReplicator project<br />
  2. 2. Contents<br />About XAM Technology<br />Why replication?<br />What we’ve done<br />Architecture<br />XReplicator library<br />XUtility<br />Demonstration<br />Resources<br />
  3. 3. About XAM Technology<br />eXtensible Access Method<br />Technology providing a unified interface between applications and storage systems<br />Industry standard API for Fixed Content Aware Storage<br />Makes developers independent from hardware<br />Easy migration between storages and applications<br />
  4. 4. Why replication?<br />Benefits of data replication<br />Reliability<br />Accessibility<br />Performance<br />Security<br />Shortage of analogues of client applications<br /> for XAM Storage Systems replication<br />
  5. 5. What we’ve done<br />XReplicator library, which enables XAM storages replication <br />XUtility, which uses XReplicator API and provides console user interface<br />XOnce, a Domain-Specified Language to make easy query to storage<br />
  6. 6. Architecture<br />XReplicator<br />XUtility<br />XAM<br />VIM<br />VIM<br />VIM<br />
  7. 7. XReplicator library<br />Replication – use RBaseAlgorithm or algorithm from third-party developers<br />XSystems collection manipulation<br />Direction of replication – mode parameter (in, out, inout)<br />Constraint for XSystem synchronization data:<br />XAM Query<br />Set of XUID<br />Configuration (XConfig)<br />Logging<br />
  8. 8. Replication – RBaseAlgorithm<br />Uses replication history (state of storage after replication)<br />Serialization/Deserialization to/from file<br />Uses XAM import/export mechanism<br />Solves “dirty” commit time problem<br />Saves timeCommit and timeSynchronize parameters<br />Solves conflicts of similar XSets<br />Supports XReplicator modes and constraints<br />
  9. 9. Logger<br />Logs information about replication, exceptions and configuration changes<br />Logs statistics about replication<br />Two types of log format:<br />XML<br />Simple<br />Supports log levels<br />Log entry content – time, method, level, message<br />Uses proxy-classes for incapsulation<br />Log file(s) settings – path, append, size limit , count<br />
  10. 10. XReplicator API architecture<br />XConfig<br />RAlgorithm<br />XReplicator factory<br />XHashSet<br />InvocationHandler<br />XReplicator<br />XReplicator Module<br />(Guice)<br />XAM<br />
  11. 11. XUtility features<br />Remote control<br />Banning enabled<br />Multiple connections<br />Scripting<br />Reading text file with XUtility commands and executing it (like a command language)<br />Program args enabled (e.g. for third party planners)<br />Encrypt<br />Blowfish encryption<br />Corporate security raise<br />Out of the box planner for replication<br />
  12. 12. XUtility features<br />Multithreading<br />Every command runs in a different thread<br />Enabled background threads (e.g remote control, planner)<br />Commands to manage threads (e.g monitor, kill)<br />Extensibility<br />Dynamic plugins loading enabled (plugin = simple impl. of XOperation)<br />
  13. 13. XUtility architecture<br />Parameters<br />XOp. Thread<br />XOp. Thread<br />Plugin 1<br />Plugin N<br />XOperation<br />Collector<br />XUtility<br />Operation <br />Set<br />XReplicator<br />Thread <br />Set<br />XAM<br />
  14. 14. XOnce features<br />Implements XAM Query<br />Similar SQL<br />Uses MPS<br />Flexible<br />Saves time<br />
  15. 15. XOnce in action<br />XOnceAPI<br />Select from myXSystem<br /> where "my.field" < 5<br /> to myCollection<br />XAM API<br />XSet query = myXSystem.createXSet(XSet.MODE_RESTRICTED);<br />query.createProperty(XSet.XAM_JOB_COMMAND,<br /> false, XSet.XAM_JOB_QUERY);<br />XStreamqueryStream = query.createXStream(<br />XSet.XAM_JOB_QUERY_COMMAND, false,<br />XAMLibrary.TEXT_PLAIN_MIME_TYPE);<br />queryStream.write ("select ".xset.xuid" where "my.field" < 5”.getBytes());<br />queryStream.close();<br />query.submitJob();<br />boolean finished = false;<br />while (!finished) {<br /> if (query.containsField(XSet.XAM_JOB_ERROR))<br /> if (query.getString(XSet.XAM_JOB_ERRORHEALTH).equals<br />XSet.XAM_JOB_ERRORHEALTH_ERROR)) {<br />query.haltJob();<br /> throw new QueryException(); }<br /> if (query.getString(XSet.XAM_JOB_STATUS).equals<br /> (XSet.XAM_JOB_STATUS_COMPLETE)) {<br /> finished = true; }<br />}<br />//.........<br />
  16. 16. XReplicator<br />Demonstration<br />
  17. 17. Resources<br />www.snia.org – XAM 1.0.1<br />developer.emc.com – Many help about XAM<br />groups.google.com/groups/xam-developers-group – Many help too<br />code.google.com/p/google-guice/ – GuiceIoC and DI<br />www.jetbrains.com/mps/ – MPS. Needed to create DSL XOnce<br />maven.apache.org – Maven build manager<br />www.junit.org – Unit tests<br />
  18. 18. XReplicaton on Open Source<br />XReplicator.sourceforge.net<br />
  19. 19. Thank you for attention!<br />Questions?<br />

×