SlideShare a Scribd company logo
Advanced Domino Replication
Top-Level Commands
• importTemplate (string)
– Imports another named script into this file to re-use the definition
• filter (string)
– Filters documents by the provided JSON query (as with Darwino selective replication on mobile)
• storeId (string)
– Sets the target store ID
• useSecurityFields (boolean)
– Sets the top-level default for whether or not to specially process reader/author fields (true by default)
• restrictToDefinedForms (boolean)
– Sets whether documents with forms other than those defined in the script should be ignored (false by
default)
• securityModel (SecurityModel enum)
– Specifies the expected security behavior (more on this later)
Top-Level Commands (Con’t)
• formConverter (closure)
– Provides a closure to convert document form names (e.g. to standardize old form
names before processing)
• locale (string or closure)
– Sets a locale name (e.g. en-us) to use for default conversions
• plugin (string)
– Loads a named converter plugin (provided via OSGi)
Field Options
• to (string)
– the Darwino field name (defaults to the same name, lowercase)
• type (DominoType enum)
– Specifies the type of the field (e.g. DATETIME, RICHTEXT)
• flags (DominoFlag enum)
– Flags to control conversion (e.g. DATE_ONLY)
• trueValue (object) and falseValue (object)
– Specifies how boolean values are stored in Domino
• nameRegex (boolean)
– Whether the field name should be used as a regex to match multiple fields
• force (boolean)
– Whether to evaluate any converters even when the field does not exist
• locale (string or closure)
– Sets the field-level locale to use for
Field Converters
• toDarwino (value, JsonObject, NSFNote)
– Provides a hook to translate a Domino value to Darwino
– Closure receives the original value, the target JsonObject, and the source
NSFNote
– Note: this is a NAPI wrapper object (more on that later)
• toDomino (value, ReplicationSourceDocument, Document)
– Provides a hook to translate a Darwino value to Domino
– Closure receives the original value, the sourceDarwino document wrapper, and
the target Document
• These can contain arbitrary code (new fields, lookups, network access,
whatever)
Array Fields
• Provide a mechanism for concatenating related fields in Domino to a single
value in Darwino
• Intended to cover a common historic pattern from the Notes client days
• For example, “FirstName_1”, “FirstName_2”, etc. -> single “firstname” array
Array Fields (Con’t)
• delimiter (string)
– Specifies the delimiter to use before or after the base name to match the index (e.g. “_” for “FirstName_1”)
• prefix (boolean)
– Specifies whether the number comes before or after the field name (defaults to false)
• zeroBased (boolean)
– Specifies whether the field indexing starts with 0 or 1 (defaults to false)
• initialIndexed (boolean)
– Specifies whether the first entry has an index
• pattern (String)
– Overrides “delimiter” and “prefix” to instead provide a pattern in the form of “Some_%_Text” to match the number
• compact (boolean)
– Specifies whether values should be compated into an array before running through conversion (defaults to false)
Security Considerations
• Darwino’s security differs from Domino’s in a few ways
– Darwino calls authors “writers” (as Domino does internally)
– Darwino has “excluded readers” and “excluded writers” – these are ignored in
Domino replication
– Darwino security is by default stricter than Domino’s: writer fields also restrict
reading
• Replicators assume that the target Darwino DB is in “Notes-like” security
mode, but this can be adjusted with the “securityModel” top-level command
• Usually, this should be left as-is and the target Darwino DB should be
configured to match Notes behavior
Converter Plugins
• Converter plugins provide a name (e.g. “processWorkflow”) and a set of
events
• Plugins are provided withhooks to the documents before and after
replication to and from Darwino
– There will likely be additional capabilities added in the future
<extension point="com.darwino.domino.replicator.documentconverterplugin">
<DocConverterPluginFactory class="com.example.PluginFactory"/>
</extension>
Converter Plguins (Con’t)
• boolean
preConvertDarwinoToDomino(ReplicationSourceDocument
jsonDocument)
• boolean postConvertDarwinoToDomino(Document doc)
• boolean preConvertDominoToDarwino(NSFNote note)
• boolean postConvertDominoToDarwino(JsonHolder jsonHolder)
Darwino Domino NAPI
• Many converter events deal with “NSFNote” objects instead of “Document”
• NSFNote is a wrapper object for the Domino C API
• The com.darwino.domino.napi package contains a JNI wrapper for the C API
as well as higher-level objects to ease the use
• For the most part, these objects act similarly to the normal Domino Java API,
but with some additional capabilities and more Java-like semantics
Example: Endeavour Replicator
// Concatenate the structured array-ish fields from the Safety forms
form("Safety") {
arrayField "$SafetyIssue_Organization", type:TEXT, pattern:
"SafetyIssue_%_Organization"
arrayField "$SafetyIssue_Issue", type:TEXT, pattern: "SafetyIssue_%_Issue"
arrayField "$SafetyIssue_OrgName", type:TEXT, pattern:
"SafetyIssue_%_OrgName"
arrayField "$FormResponse_Key", type:TEXT, pattern: "FormResponse_%_Key"
arrayField "$FormResponse_Question", type:TEXT, pattern:
"FormResponse_%_Question"
arrayField "$FormResponse_Response", type:TEXT, pattern:
"FormResponse_%_Response"
}
Example: Endeavour Reporter
// Combine the two meetingdocid fields in MeetingItems and related docs
def meetingDocIdRef = {
// Since these aren't arrayField type, join "manually”
field "MeetingDocIDRef", force: true,
toDarwino: { value, dest, note ->
// Concatenate, but merge duplicates and remove empty values
(Arrays.asList(note.get("OpenMeetingDocID")) +
Arrays.asList(note.get("MeetingDocID"))).unique(false) - [null, ""]
},
toDomino: {
null
}
}
form("MeetingItem", meetingDocIdRef)
form("ActionItem", meetingDocIdRef)
form("Issue", meetingDocIdRef)
Thank you for your attention!

More Related Content

What's hot

Unit 3
Unit  3Unit  3
Unit 3
siddr
 
Filesinc 130512002619-phpapp01
Filesinc 130512002619-phpapp01Filesinc 130512002619-phpapp01
Filesinc 130512002619-phpapp01
Rex Joe
 
basics of file handling
basics of file handlingbasics of file handling
basics of file handling
pinkpreet_kaur
 

What's hot (20)

Files and file objects (in Python)
Files and file objects (in Python)Files and file objects (in Python)
Files and file objects (in Python)
 
File handling
File handlingFile handling
File handling
 
COM1407: File Processing
COM1407: File Processing COM1407: File Processing
COM1407: File Processing
 
File handling in Python
File handling in PythonFile handling in Python
File handling in Python
 
Java nio ( new io )
Java nio ( new io )Java nio ( new io )
Java nio ( new io )
 
Linux intro 5 extra: makefiles
Linux intro 5 extra: makefilesLinux intro 5 extra: makefiles
Linux intro 5 extra: makefiles
 
7 streams and error handling in java
7 streams and error handling in java7 streams and error handling in java
7 streams and error handling in java
 
Files nts
Files ntsFiles nts
Files nts
 
Distributed Systems Naming
Distributed Systems NamingDistributed Systems Naming
Distributed Systems Naming
 
Python - Lecture 8
Python - Lecture 8Python - Lecture 8
Python - Lecture 8
 
Unit 3
Unit  3Unit  3
Unit 3
 
File Handling and Command Line Arguments in C
File Handling and Command Line Arguments in CFile Handling and Command Line Arguments in C
File Handling and Command Line Arguments in C
 
Pf cs102 programming-8 [file handling] (1)
Pf cs102 programming-8 [file handling] (1)Pf cs102 programming-8 [file handling] (1)
Pf cs102 programming-8 [file handling] (1)
 
12 linux archiving tools
12 linux archiving tools12 linux archiving tools
12 linux archiving tools
 
Files and streams
Files and streamsFiles and streams
Files and streams
 
Java
JavaJava
Java
 
Chapter 12 - File Input and Output
Chapter 12 - File Input and OutputChapter 12 - File Input and Output
Chapter 12 - File Input and Output
 
File handling
File handlingFile handling
File handling
 
Filesinc 130512002619-phpapp01
Filesinc 130512002619-phpapp01Filesinc 130512002619-phpapp01
Filesinc 130512002619-phpapp01
 
basics of file handling
basics of file handlingbasics of file handling
basics of file handling
 

Similar to 11 advanced replication

Hadoop 20111117
Hadoop 20111117Hadoop 20111117
Hadoop 20111117
exsuns
 

Similar to 11 advanced replication (20)

What is new in CFEngine 3.6
What is new in CFEngine 3.6What is new in CFEngine 3.6
What is new in CFEngine 3.6
 
Hadoop HDFS Concepts
Hadoop HDFS ConceptsHadoop HDFS Concepts
Hadoop HDFS Concepts
 
04 darwino concepts and utility classes
04   darwino concepts and utility classes04   darwino concepts and utility classes
04 darwino concepts and utility classes
 
Fantastic DSL in Python
Fantastic DSL in PythonFantastic DSL in Python
Fantastic DSL in Python
 
System Programming and Administration
System Programming and AdministrationSystem Programming and Administration
System Programming and Administration
 
Developing a custom Kafka connector? Make it shine! | Igor Buzatović, Porsche...
Developing a custom Kafka connector? Make it shine! | Igor Buzatović, Porsche...Developing a custom Kafka connector? Make it shine! | Igor Buzatović, Porsche...
Developing a custom Kafka connector? Make it shine! | Igor Buzatović, Porsche...
 
Helberg acl-final
Helberg acl-finalHelberg acl-final
Helberg acl-final
 
Data Migration with Spark to Hive
Data Migration with Spark to HiveData Migration with Spark to Hive
Data Migration with Spark to Hive
 
Concurrency at the Database Layer
Concurrency at the Database Layer Concurrency at the Database Layer
Concurrency at the Database Layer
 
Hadoop 20111117
Hadoop 20111117Hadoop 20111117
Hadoop 20111117
 
Introduction to firebidSQL 3.x
Introduction to firebidSQL 3.xIntroduction to firebidSQL 3.x
Introduction to firebidSQL 3.x
 
Protocol Buffers
Protocol BuffersProtocol Buffers
Protocol Buffers
 
Java I/O
Java I/OJava I/O
Java I/O
 
Hadoop HDFS Concepts
Hadoop HDFS ConceptsHadoop HDFS Concepts
Hadoop HDFS Concepts
 
Writing and using php streams and sockets
Writing and using php streams and socketsWriting and using php streams and sockets
Writing and using php streams and sockets
 
25dom
25dom25dom
25dom
 
archive A-Z linux
archive A-Z linuxarchive A-Z linux
archive A-Z linux
 
How to use the new Domino Query Language
How to use the new Domino Query LanguageHow to use the new Domino Query Language
How to use the new Domino Query Language
 
Dom
Dom Dom
Dom
 
Commands
CommandsCommands
Commands
 

More from darwinodb

More from darwinodb (15)

17 open ntf-v30-darwinoorg
17   open ntf-v30-darwinoorg17   open ntf-v30-darwinoorg
17 open ntf-v30-darwinoorg
 
16 endeavour reporter
16   endeavour reporter16   endeavour reporter
16 endeavour reporter
 
15 darwino script & command line
15   darwino script & command line15   darwino script & command line
15 darwino script & command line
 
14 integrating watson services
14   integrating watson services14   integrating watson services
14 integrating watson services
 
13 deploying cloud applications
13   deploying cloud applications13   deploying cloud applications
13 deploying cloud applications
 
12 ibm connections integration
12   ibm connections integration12   ibm connections integration
12 ibm connections integration
 
10 domino integration
10   domino integration10   domino integration
10 domino integration
 
09 business apis
09   business apis09   business apis
09 business apis
 
08 mobile development
08   mobile development08   mobile development
08 mobile development
 
07 darwino rest services
07   darwino rest services07   darwino rest services
07 darwino rest services
 
06 web applications
06   web applications06   web applications
06 web applications
 
05 darwino db
05   darwino db05   darwino db
05 darwino db
 
02 configuration
02   configuration02   configuration
02 configuration
 
01 introduction to darwino
01   introduction to darwino01   introduction to darwino
01 introduction to darwino
 
IBM ConnectED SPOT104: Lightning-Fast Development of Native Mobile Apps for I...
IBM ConnectED SPOT104: Lightning-Fast Development of Native Mobile Apps for I...IBM ConnectED SPOT104: Lightning-Fast Development of Native Mobile Apps for I...
IBM ConnectED SPOT104: Lightning-Fast Development of Native Mobile Apps for I...
 

Recently uploaded

How to Position Your Globus Data Portal for Success Ten Good Practices
How to Position Your Globus Data Portal for Success Ten Good PracticesHow to Position Your Globus Data Portal for Success Ten Good Practices
How to Position Your Globus Data Portal for Success Ten Good Practices
Globus
 
Providing Globus Services to Users of JASMIN for Environmental Data Analysis
Providing Globus Services to Users of JASMIN for Environmental Data AnalysisProviding Globus Services to Users of JASMIN for Environmental Data Analysis
Providing Globus Services to Users of JASMIN for Environmental Data Analysis
Globus
 

Recently uploaded (20)

2024 RoOUG Security model for the cloud.pptx
2024 RoOUG Security model for the cloud.pptx2024 RoOUG Security model for the cloud.pptx
2024 RoOUG Security model for the cloud.pptx
 
Multiple Your Crypto Portfolio with the Innovative Features of Advanced Crypt...
Multiple Your Crypto Portfolio with the Innovative Features of Advanced Crypt...Multiple Your Crypto Portfolio with the Innovative Features of Advanced Crypt...
Multiple Your Crypto Portfolio with the Innovative Features of Advanced Crypt...
 
Field Employee Tracking System| MiTrack App| Best Employee Tracking Solution|...
Field Employee Tracking System| MiTrack App| Best Employee Tracking Solution|...Field Employee Tracking System| MiTrack App| Best Employee Tracking Solution|...
Field Employee Tracking System| MiTrack App| Best Employee Tracking Solution|...
 
How Does XfilesPro Ensure Security While Sharing Documents in Salesforce?
How Does XfilesPro Ensure Security While Sharing Documents in Salesforce?How Does XfilesPro Ensure Security While Sharing Documents in Salesforce?
How Does XfilesPro Ensure Security While Sharing Documents in Salesforce?
 
In 2015, I used to write extensions for Joomla, WordPress, phpBB3, etc and I ...
In 2015, I used to write extensions for Joomla, WordPress, phpBB3, etc and I ...In 2015, I used to write extensions for Joomla, WordPress, phpBB3, etc and I ...
In 2015, I used to write extensions for Joomla, WordPress, phpBB3, etc and I ...
 
Paketo Buildpacks : la meilleure façon de construire des images OCI? DevopsDa...
Paketo Buildpacks : la meilleure façon de construire des images OCI? DevopsDa...Paketo Buildpacks : la meilleure façon de construire des images OCI? DevopsDa...
Paketo Buildpacks : la meilleure façon de construire des images OCI? DevopsDa...
 
How to Position Your Globus Data Portal for Success Ten Good Practices
How to Position Your Globus Data Portal for Success Ten Good PracticesHow to Position Your Globus Data Portal for Success Ten Good Practices
How to Position Your Globus Data Portal for Success Ten Good Practices
 
Globus Compute wth IRI Workflows - GlobusWorld 2024
Globus Compute wth IRI Workflows - GlobusWorld 2024Globus Compute wth IRI Workflows - GlobusWorld 2024
Globus Compute wth IRI Workflows - GlobusWorld 2024
 
Software Testing Exam imp Ques Notes.pdf
Software Testing Exam imp Ques Notes.pdfSoftware Testing Exam imp Ques Notes.pdf
Software Testing Exam imp Ques Notes.pdf
 
Beyond Event Sourcing - Embracing CRUD for Wix Platform - Java.IL
Beyond Event Sourcing - Embracing CRUD for Wix Platform - Java.ILBeyond Event Sourcing - Embracing CRUD for Wix Platform - Java.IL
Beyond Event Sourcing - Embracing CRUD for Wix Platform - Java.IL
 
Exploring Innovations in Data Repository Solutions - Insights from the U.S. G...
Exploring Innovations in Data Repository Solutions - Insights from the U.S. G...Exploring Innovations in Data Repository Solutions - Insights from the U.S. G...
Exploring Innovations in Data Repository Solutions - Insights from the U.S. G...
 
Cyaniclab : Software Development Agency Portfolio.pdf
Cyaniclab : Software Development Agency Portfolio.pdfCyaniclab : Software Development Agency Portfolio.pdf
Cyaniclab : Software Development Agency Portfolio.pdf
 
Using IESVE for Room Loads Analysis - Australia & New Zealand
Using IESVE for Room Loads Analysis - Australia & New ZealandUsing IESVE for Room Loads Analysis - Australia & New Zealand
Using IESVE for Room Loads Analysis - Australia & New Zealand
 
Globus Connect Server Deep Dive - GlobusWorld 2024
Globus Connect Server Deep Dive - GlobusWorld 2024Globus Connect Server Deep Dive - GlobusWorld 2024
Globus Connect Server Deep Dive - GlobusWorld 2024
 
Designing for Privacy in Amazon Web Services
Designing for Privacy in Amazon Web ServicesDesigning for Privacy in Amazon Web Services
Designing for Privacy in Amazon Web Services
 
Providing Globus Services to Users of JASMIN for Environmental Data Analysis
Providing Globus Services to Users of JASMIN for Environmental Data AnalysisProviding Globus Services to Users of JASMIN for Environmental Data Analysis
Providing Globus Services to Users of JASMIN for Environmental Data Analysis
 
Into the Box 2024 - Keynote Day 2 Slides.pdf
Into the Box 2024 - Keynote Day 2 Slides.pdfInto the Box 2024 - Keynote Day 2 Slides.pdf
Into the Box 2024 - Keynote Day 2 Slides.pdf
 
Accelerate Enterprise Software Engineering with Platformless
Accelerate Enterprise Software Engineering with PlatformlessAccelerate Enterprise Software Engineering with Platformless
Accelerate Enterprise Software Engineering with Platformless
 
De mooiste recreatieve routes ontdekken met RouteYou en FME
De mooiste recreatieve routes ontdekken met RouteYou en FMEDe mooiste recreatieve routes ontdekken met RouteYou en FME
De mooiste recreatieve routes ontdekken met RouteYou en FME
 
How Recreation Management Software Can Streamline Your Operations.pptx
How Recreation Management Software Can Streamline Your Operations.pptxHow Recreation Management Software Can Streamline Your Operations.pptx
How Recreation Management Software Can Streamline Your Operations.pptx
 

11 advanced replication

  • 2. Top-Level Commands • importTemplate (string) – Imports another named script into this file to re-use the definition • filter (string) – Filters documents by the provided JSON query (as with Darwino selective replication on mobile) • storeId (string) – Sets the target store ID • useSecurityFields (boolean) – Sets the top-level default for whether or not to specially process reader/author fields (true by default) • restrictToDefinedForms (boolean) – Sets whether documents with forms other than those defined in the script should be ignored (false by default) • securityModel (SecurityModel enum) – Specifies the expected security behavior (more on this later)
  • 3. Top-Level Commands (Con’t) • formConverter (closure) – Provides a closure to convert document form names (e.g. to standardize old form names before processing) • locale (string or closure) – Sets a locale name (e.g. en-us) to use for default conversions • plugin (string) – Loads a named converter plugin (provided via OSGi)
  • 4. Field Options • to (string) – the Darwino field name (defaults to the same name, lowercase) • type (DominoType enum) – Specifies the type of the field (e.g. DATETIME, RICHTEXT) • flags (DominoFlag enum) – Flags to control conversion (e.g. DATE_ONLY) • trueValue (object) and falseValue (object) – Specifies how boolean values are stored in Domino • nameRegex (boolean) – Whether the field name should be used as a regex to match multiple fields • force (boolean) – Whether to evaluate any converters even when the field does not exist • locale (string or closure) – Sets the field-level locale to use for
  • 5. Field Converters • toDarwino (value, JsonObject, NSFNote) – Provides a hook to translate a Domino value to Darwino – Closure receives the original value, the target JsonObject, and the source NSFNote – Note: this is a NAPI wrapper object (more on that later) • toDomino (value, ReplicationSourceDocument, Document) – Provides a hook to translate a Darwino value to Domino – Closure receives the original value, the sourceDarwino document wrapper, and the target Document • These can contain arbitrary code (new fields, lookups, network access, whatever)
  • 6. Array Fields • Provide a mechanism for concatenating related fields in Domino to a single value in Darwino • Intended to cover a common historic pattern from the Notes client days • For example, “FirstName_1”, “FirstName_2”, etc. -> single “firstname” array
  • 7. Array Fields (Con’t) • delimiter (string) – Specifies the delimiter to use before or after the base name to match the index (e.g. “_” for “FirstName_1”) • prefix (boolean) – Specifies whether the number comes before or after the field name (defaults to false) • zeroBased (boolean) – Specifies whether the field indexing starts with 0 or 1 (defaults to false) • initialIndexed (boolean) – Specifies whether the first entry has an index • pattern (String) – Overrides “delimiter” and “prefix” to instead provide a pattern in the form of “Some_%_Text” to match the number • compact (boolean) – Specifies whether values should be compated into an array before running through conversion (defaults to false)
  • 8. Security Considerations • Darwino’s security differs from Domino’s in a few ways – Darwino calls authors “writers” (as Domino does internally) – Darwino has “excluded readers” and “excluded writers” – these are ignored in Domino replication – Darwino security is by default stricter than Domino’s: writer fields also restrict reading • Replicators assume that the target Darwino DB is in “Notes-like” security mode, but this can be adjusted with the “securityModel” top-level command • Usually, this should be left as-is and the target Darwino DB should be configured to match Notes behavior
  • 9. Converter Plugins • Converter plugins provide a name (e.g. “processWorkflow”) and a set of events • Plugins are provided withhooks to the documents before and after replication to and from Darwino – There will likely be additional capabilities added in the future <extension point="com.darwino.domino.replicator.documentconverterplugin"> <DocConverterPluginFactory class="com.example.PluginFactory"/> </extension>
  • 10. Converter Plguins (Con’t) • boolean preConvertDarwinoToDomino(ReplicationSourceDocument jsonDocument) • boolean postConvertDarwinoToDomino(Document doc) • boolean preConvertDominoToDarwino(NSFNote note) • boolean postConvertDominoToDarwino(JsonHolder jsonHolder)
  • 11. Darwino Domino NAPI • Many converter events deal with “NSFNote” objects instead of “Document” • NSFNote is a wrapper object for the Domino C API • The com.darwino.domino.napi package contains a JNI wrapper for the C API as well as higher-level objects to ease the use • For the most part, these objects act similarly to the normal Domino Java API, but with some additional capabilities and more Java-like semantics
  • 12. Example: Endeavour Replicator // Concatenate the structured array-ish fields from the Safety forms form("Safety") { arrayField "$SafetyIssue_Organization", type:TEXT, pattern: "SafetyIssue_%_Organization" arrayField "$SafetyIssue_Issue", type:TEXT, pattern: "SafetyIssue_%_Issue" arrayField "$SafetyIssue_OrgName", type:TEXT, pattern: "SafetyIssue_%_OrgName" arrayField "$FormResponse_Key", type:TEXT, pattern: "FormResponse_%_Key" arrayField "$FormResponse_Question", type:TEXT, pattern: "FormResponse_%_Question" arrayField "$FormResponse_Response", type:TEXT, pattern: "FormResponse_%_Response" }
  • 13. Example: Endeavour Reporter // Combine the two meetingdocid fields in MeetingItems and related docs def meetingDocIdRef = { // Since these aren't arrayField type, join "manually” field "MeetingDocIDRef", force: true, toDarwino: { value, dest, note -> // Concatenate, but merge duplicates and remove empty values (Arrays.asList(note.get("OpenMeetingDocID")) + Arrays.asList(note.get("MeetingDocID"))).unique(false) - [null, ""] }, toDomino: { null } } form("MeetingItem", meetingDocIdRef) form("ActionItem", meetingDocIdRef) form("Issue", meetingDocIdRef)
  • 14. Thank you for your attention!