The document discusses a migration from RDBMS to MongoDB at Bayerisches Landesamt, highlighting the benefits of simpler learning and reduced complexity for developers. It outlines techniques for unit testing and performance improvements through the use of MongoDB's driver, as well as challenges such as maintaining referential integrity. The summary emphasizes the overall positive experience and satisfaction in developing the data layer using MongoDB.

1. MongoDB@Bayern
Migration from RDBMS, Problems, Unit-Testing

2. Christian Brensing
Bayerisches Landesamt für
Statistik und Datenverarbeitung
Rechenzentrum Süd

3. • Template-Processor (~ JSP)
• ODF and RTF
• PDF / PostScript postprocessing
• Groovy, Python, Ruby, Tcl
• In Production using RDBMS
since 2008
• Central HR system (SAP) uses
BayText to create output

4. Client for developing the templates (BayText-IDE)

5. ODF-Template using Ruby

6. Generated document

7. 2 Million
Generated documents per year

8. Tiny dataset
50,000 records and 10 GB BLOB

9. a small system after all

10. So why migrate?

11. Because we can

12. developers.pop()
Smaller teams

13. Easy to learn
Read one book and you're done

14. Simple mapping

15. permissions templates documents
codes groups document_usages
folder_configurations requests folders
nodes structures template_tags
dynafields components roles_roles
roles bundles scripts

16. nodes
roles
properties
GridFS

17. {!
"_id" : ObjectId(),!
"_type" : "Document",!
"updated_at" : ISODate("2007-12-06T10:04:00.543Z"),!
"updated_by" : "maggie",!
"version" : 1,!
"path" : "foo.bar.Document",!
"description" : "",!
"language" : "ruby",!
"format" : "odf",!
"autoload_uplevel" : -1,!
"requests" : [!
{!
"name" : "Test",!
"description" : "foo", !
"xml" : zlib("<?xml ...?>")!
}, !
... !
],!
"structure" : [!
{ !
"alias" : "some alias",!
"component_path" : "foo.bar.Document$Document",!
"parent", -1 !
},!
... !
], !
} !
"tags" : ["foo", "bar", "baz"]!
}

18. public class RoleReadConverter implements Converter<DBObject, Role> {!
@Override!
public Role convert(DBObject source) {!
Role role = new Role();!
role.setName((String) source.get("name"));!
...!
return role;!
}!
}!
!
!
public class RoleWriteConverter implements Converter<Role, DBObject> {!
@Override!
public DBObject convert(Role source) {!
return new BasicDBObject()!
.append("name", role.getName())!
.append(...);!
}!
}!
!
!
// Convert a DBObject to an Entity!
conversionService.convert(collection.findOne(...));
Easy mapping using Spring-ConversionService

19. Performance
up to 10x

20. ORM no more
The MongoDB driver is all you need

21. Obstacles?

22. Mentality

23. Referential Integrity
Multi-Document-Update

24. {!
"_id" : ObjectId(),!
"_type" : "Folder", !
"updated_at" : ISODate("2007-12-06T10:04:00.543Z"),!
"updated_by" : "maggie",!
"version" : 1,!
"path" : "foo.bar.Folder",!
"description" : "bla",!
"bundles" : [!
... !
]!
"document_usages" : [!
{!
"name" : "foo",!
"description" : "bla", !
"exec_order" : 5, !
"print_copies" : 2,!
"type": "FAIR_COPY",!
"document_path" : "a.b.Document",!
"bundle" : "bar"!
},!
...!
}
Linking documents via a path instead of ObjectId

25. But paths are mutable!
All references must be updated

26. Approach
• Pseudo-Transaction with an update locking
mechanism in a custom oplog collection
• Periodical repair jobs finishing cancelled
operations
• Extended interpretation of eventual consistency

27. Unit-Tests
Continuous Integration

28. F.I.R.S.T.

29. Isolated
Tests are using a unique DB per JVM

30. @Configuration!
@Profile("test")!
public class TestSpringConfiguration extends SpringConfiguration {!
@Override!
public void init() {!
setTestProperties();!
!
super.init();!
!
// Test-DB löschen!
db().dropDatabase();!
}!
!
private void setTestProperties() {!
String dbname = env.getProperty("db.name", "baytext_test_" + getSimpleUsername());!
String host = env.getProperty("db.host", "mongodb-dev.db.rz-sued.bybn.de");!
String port = env.getProperty("db.port", "27016");!
!
cmProperties.setProperty("servers", String.format("%s:%s", host, port));!
cmProperties.setProperty("name", dbname);!
}!
!
// OS-Username without prefix (e.g. maggie instead of lfstad-maggie)!
private static String getSimpleUsername() {!
String username = SystemUtils.USER_NAME;!
int indexOfDash = username.indexOf('-');!
return indexOfDash != -1 ? username.substring(indexOfDash + 1) : username;!
}!
}
Test-ApplicationContext

31. Repeatable
Collections are purged before each test method

32. public class MongoTestExcecutionListener extends AbstractTestExecutionListener {!
@Override!
public void beforeTestMethod(TestContext testContext) throws Exception {!
purgeCollections();!
}!
!
private void purgeCollections() {!
DB db = MongoDBHolder.getDB();!
for (String collectionName : db.getCollectionNames()) {!
if (collectionName.startsWith("fs.") || collectionName.startsWith("system.")) {!
continue;!
}!
DBCollection collection = db.getCollection(collectionName);!
if (!collection.isCapped()) {!
collection.remove(new BasicDBObject());!
}!
}!
}!
}
Spring-TestExecutionListener

33. @RunWith(SpringJUnit4ClassRunner.class)!
@ActiveProfiles({"test"})!
@ContextConfiguration(classes = TestSpringConfiguration.class)!
@TestExecutionListeners({!
DependencyInjectionTestExecutionListener.class,!
DirtiesContextTestExecutionListener.class,!
MongoTestExcecutionListener.class!
})!
public abstract class MongoTestSupport {!
}!
!
!
!
public class DocumentRepositoryTest extends MongoTestSupport {!
}!
Test base class

34. 95% Coverage

35. Summary
• Developing the data layer is fun again
• Reduced complexity
• Flat learning curve compared to SQL/ORM
• use_mongo() unless transactions_required

36. Thank you!