The document provides an introduction to the ConfD Database (CDB), which is a hierarchical, ACID-compliant database used by ConfD to store configuration and operational data modeled with YANG. CDB stores data both in-memory and persistently on disk, supports transactions, replication, and automatic schema upgrades. Managed objects use CDB APIs to read configuration data at startup and subscribe to relevant configuration changes via a subscription socket.

1. 1©2015 Tail-f Systems all rights reservedFebruary 13, 2015
CDB Technical Introduction

2. 2©2015 Tail-f Systems all rights reservedFebruary 13, 2015
CDB – ConfD Database
NETCONF SNMP REST CLI Web
CDB
Database
Data Provider API
CDB API
Management Agent API
App AppApp
ConfD Core Engine
App
External
DatabaseOperational
Data
App
• Hierarchical database
• ACID test compliant
• Atomicity, consistency, isolation, durability
• Fast, lightweight, fault-tolerant
• Compact binary XML format
• Memory resident with journal in persistent storage
• Schema automatically derived from YANG
• Supports 1:N data replication
• Supports automatic schema version
up/downgrades
• CDB can hold some or all data
• Configuration data and/or Operational data
• External database can be used in addition to or instead of
• Automatic loading of initial data
• Applications read data, then subscribe to relevant
configuration changes
• Subscription notifications with priority level
ordering
• Multiple datastores per NETCONF standards
• Startup, running, candidate, operational

3. 3©2015 Tail-f Systems all rights reservedFebruary 13, 2015
Hierarchical
• CDB uses an Hierarchical (XML) data model
• Ideal for configuration data, which is tree structured
• No Object-Relational-Mapping (ORM) required
/ dhcp/
aaa/
voip/
defaultLe…
maxLeas…
SubNets/ subNet/
mask
range
lowAddr
highAddr
logFacility
net

4. 4©2015 Tail-f Systems all rights reservedFebruary 13, 2015
Object Relational Mapping (ORM)
Hierarchical Relational
/ dhcp/
aaa/
voip/
defaultLe…
maxLeas…
SubNets/ subNet/
mask
range
lowAddr
highAddr
logFacility
net

5. 5©2015 Tail-f Systems all rights reservedFebruary 13, 2015
ACID
The four properties that define a transaction:
• Atomicity
• Transactions are indivisible, all-or-nothing
• Consistency
• Transactions are all-at-once
• {create A, create B} and {create B, create A} are identical
• Independence
• Transactions do not interfere with each other
• Durability
• Committed data remains in the system even in case of a restart, etc

6. 6©2015 Tail-f Systems all rights reservedFebruary 13, 2015
Database Storage
MO = Managed Object
(i.e. configuration consumer)
JOURNAL
START TRANSACTION
WRITE 14 TO
/some/leaf/somewhere
WRITE “hello” TO
/some/other/leaf
COMMIT
START TRANSACTION
WRITE -1 TO /foo/bar
ABORTED
START TR
ON DISK
Values stored
binary
Paths stored as
sequences of
32-bit hashes
ConfD
CDB
IN MEMORY
MO
MO
MO

7. 7©2015 Tail-f Systems all rights reservedFebruary 13, 2015
Data Model Becomes Database Schema
Network
Element
NETCONF SNMP REST CLI Web
CDB
Database
Data Provider API
CDB API
Management Agent API
OSS/BSS
App AppApp
ConfD Core Engine
App
External
DatabaseOperational
Data
App
Management Agent API
Auto-rendered management interfaces and data schema
YANG
Data Models

8. 8©2015 Tail-f Systems all rights reservedFebruary 13, 2015
JOURNAL
ConfD
CDB
IN MEMORY
MO
MO
MO
Database Replication
ACTIVE NODE
Journal
records
JOURNAL
ConfD
CDB
IN MEMORY
MO
MO
MO
STANDBY NODE

9. 9©2015 Tail-f Systems all rights reservedFebruary 13, 2015
CDB Start
• CDB in-memory contents blank at this point
• Read up schema from disk
• Start transaction
• Read up data from disk
• Data on disk consistent with schema?
• If not: Trigger automatic schema upgrade (see next page)
• CDB doesn’t differentiate between up- or downgrade,
it just notices that the schema is different
• CDB does not rely on version in the namespace declaration
• Validate
• Commit

10. 10©2015 Tail-f Systems all rights reservedFebruary 13, 2015
CDB Automatic Schema Upgrade
• Elements with default value added to schema
• Default values added to new database
• Elements removed from schema
• Values removed from new database
• Elements with trivial type changes
• Value promoted to new type in new database
• Data is converted to string representation, then to new type
• Elements with complex type changes
• No value in new database
• All other elements
• Copied as is to new database
Check the devel.log for messages about what conversions were made

11. 11©2015 Tail-f Systems all rights reservedFebruary 13, 2015
CDB Programmatic Schema Upgrade
Then add your own upgrade clients
• Complement to the automatic schema upgrade
• Connect in start phase 0
• Compute values for the new database
• Read any old or new database values
• Compute new values
• Write (overwrite) new values
• If the schema is changed cautiously,
CDB automatic schema upgrade might be able to handle everything

12. 12©2015 Tail-f Systems all rights reservedFebruary 13, 2015
CDB Schema Upgrade Examples
• 32-bit flag field becomes 64-bits
• New flags should be initialized to all zeros
• Automatic upgrade sufficient
• New Maximum Transfer Unit (MTU) field added
• Should be 1500 for Eth interfaces, 48 for ATM interfaces
• Upgrade client reads InterfaceType value, classifies it and writes the appropriate MTU value
• InterfaceType changed from string to enumeration
• All strings that match enumeration value are auto upgraded
• Remaining values left unset: set by upgrade client
• Time field changed from seconds to nanoseconds
• Upgrade client reads the old value, multiplies by 109 and overwrites the automatically
promoted value

13. 13©2015 Tail-f Systems all rights reservedFebruary 13, 2015
Data Stores
• Each data store is a full copy of the configuration
• Each data store can be mapped to CDB or external data base
• Running can be split across CDB and several data bases
• 5 combinations (S+Rro, S+R, R, R+C, Rro+C)
• Operational data store
• Store status, performance or application data
CandidateStart-up Running
Operational

14. 14©2015 Tail-f Systems all rights reservedFebruary 13, 2015
Data Stores
• Start-up
• If enabled, start-up configuration is copied to (overwriting) the running configuration at system
start
• If not enabled, configuration in running remains unchanged from last time system was running
• The first time the system starts, configuration is read from factory default files ( *_init.xml )
• Running
• Holds the currently running configuration
• Candidate
• Data store for future configuration under construction
• Copied to running to make it take effect

15. 15©2015 Tail-f Systems all rights reservedFebruary 13, 2015
Sessions
• Session Start (login)
• Display values
• Update values
• Request Commit
• Lock data store (running, candidate, start-up)
• Run validation sequence
• Run Transaction (next slide)
• Wait for acknowledgement from standby node(s)
• Inform subscribers of changes
• Unlock data store
• Session Continues

16. 16©2015 Tail-f Systems all rights reservedFebruary 13, 2015
Transactions

17. 17©2015 Tail-f Systems all rights reservedFebruary 13, 2015
The Life of a Managed Object
• Subscribe to relevant configuration changes
• Read out configuration at startup
• Enter event loop
• If subscription event, react to configuration change
RESTCLINETCONF SNMP
Management Agent API
Managed Object API
MO MO
CDB
ConfD Core Engine
CDB API
Data Provider API
Web

18. 18©2015 Tail-f Systems all rights reservedFebruary 13, 2015
Read out Configuration at Startup
• Initialization
• cdb_connect()
• cdb_start_session()
• cdb_set_namespace()
• Read elements
• cdb_get_u_int32()
• cdb_get_bool()
• cdb_get_str()
• cdb_get_ipv4()
• cdb_get_enum_hash()
• … (~30 more)
• Read many
• cdb_get_object()
• cdb_get_values()
• Count elements
• cdb_exists()
• cdb_num_instances()
• Move in data tree
• cdb_cd()
• cdb_pushd()
• Finalization
• cdb_end_session()
• cdb_close()

19. 19©2015 Tail-f Systems all rights reservedFebruary 13, 2015
Read out Configuration at Startup
Sessions
• CDB protected with Single Writer-
Multiple Reader lock
• No configuration changes can occur
between cdb_start_session() and
cdb_end_session()
• Don’t keep the session open
unnecessarily, delays configuration
changes
• Timeout after some minutes
Namespaces
• Each data model is defined in an
XML Namespace
• ConfD handles multiple data
models/namespaces
• Each subsystem should be separate
namespace
• Generate header file with
namespace hash constant
sock = socket(PF_INET, SOCK_STREAM, 0);
cdb_connect(sock, CDB_READ_SOCKET, &confd_ip_port, size);
cdb_start_session(sock, CDB_RUNNING);
cdb_set_namespace(sock, dhcpd__ns);

20. 20©2015 Tail-f Systems all rights reservedFebruary 13, 2015
Read out Configuration at Startup
Keypaths
• Each element in CDB is addressed by its path
cdb_get_duration(sock, &dur, “/dhcp/defaultLeaseTime”)
• Paths may include keys
cdb_get_ipv4(sock,&ip,”/dhcp/SubNets/subNet{10.0.0.0 255.0.0.0}/lowAddr”)
cdb_get_ipv4(sock, &ip, ”/dhcp/SubNets/subNet{%s %s}/lowAddr”, net, mask)
cdb_get_ipv4(sock, &ip, ”/dhcp/SubNets/subNet{%x %x}/lowAddr”, net, mask)
• Index may be used in place of keys
n = cdb_num_instances(sock, "/dhcp/SubNets/subNet");
for (i=0; i<n; i++) {
cdb_cd(sock, "/dhcp/SubNets/subNet[%d]", i);
do_subnet(…);
}

21. 21©2015 Tail-f Systems all rights reservedFebruary 13, 2015
Triggers when
anything in
scope changes
Subscribe to Configuration Changes
subsock = socket(PF_INET, SOCK_STREAM, 0);
cdb_connect(subsock, CDB_SUBSCRIPTION_SOCKET, &confd, size);
cdb_subscribe(subsock, 3, dhcpd__ns, &spoint, "/dhcp");
cdb_subscribe_done(subsock);
/ dhcp/
aaa/
voip/
defaultLe…
maxLeas…
SubNets/ subNet/
mask
range
lowAddr
highAddr
logFacility
net
Managed Objects typically have
several subscription points

22. 22©2015 Tail-f Systems all rights reservedFebruary 13, 2015
Enter Event Loop
• Event loop needs to monitor subscription socket
• Typically using poll()/select()
while(1) {
set[0].fd = subsock;
set[0].events = POLLIN;
set[0].revents = 0;
poll(set, 1, -1); // Monitor one socket, wait forever
if (set[0].revents & POLLIN) {
cdb_read_subscription_socket(subsock, sub_points, &reslen);
// React on changes here…
// sub_points[] contains triggering subscription points,
// reslen how many
}
cdb_sync_subscription_socket(subsock, CDB_DONE_PRIORITY);
}
• How to handle CDB subscription notifications will be covered in the training module
“CDB Subscribers”

23. 23©2015 Tail-f Systems all rights reservedFebruary 13, 2015