How test-driven development enabled Gnowsis to reimplement Refinder's basic data model. Talk at Warsaw DjangoPiwo meetup on 2011-11-02.

1. TDD

2. RED
GREEN
REFACTOR

3. Effect?

4. More code

5. More simple code

6. More simple code,
delivered faster

7. Cleaner
interfaces

8. Better
understanding
of codebase

9. Fast & precise
regression
discovery

10. Easy
refactoring

11. Case
study

12. Things come together.
Semantic social
collaboration platform

13. Things come together.
Python/Django
PostgreSQL
Apache Solr
Custom Java backend

14. RDF-based APIs
Graphs of linked data
URIs as identifiers
Subject-Predicate-Object
triples

15. <_:me> <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <http://xmlns.com/foaf/0.1/Person> .
<_:me> <http://xmlns.com/foaf/0.1/family_name> "Pasternacki" .
<_:me> <http://xmlns.com/foaf/0.1/givenname> "Maciej" .
<_:me> <http://xmlns.com/foaf/0.1/homepage> <http://www.pasternacki.net/> .
<_:me> <http://xmlns.com/foaf/0.1/mbox> <mailto:maciej@pasternacki.net> .
<_:me> <http://xmlns.com/foaf/0.1/name> "Maciej Pasternacki" .
<_:me> <http://xmlns.com/foaf/0.1/nick> "japhy" .
<_:me> <http://xmlns.com/foaf/0.1/workplaceHomepage> <http://www.gnowsis.com/> .
@prefix : <#> .
@prefix foaf: <http://xmlns.com/foaf/0.1/> .
:me a foaf:Person .
:me foaf:family_name "Pasternacki" .
:me foaf:givenname "Maciej" .
:me foaf:homepage <http://www.pasternacki.net/> .
:me foaf:mbox <mailto:maciej@pasternacki.net> .
:me foaf:name "Maciej Pasternacki" .
:me foaf:nick "japhy" .
:me foaf:workplaceHomepage <http://www.gnowsis.com/> .

16. Item Relation
uuid : uuid type : uri
rdf_type : URI *
label : string
created : dateTime
lastModified: dateTime Description
text : string
target (1)
Thing Annotation Rating
value : normalizedFloat
InformationElement Wiki
text : string
Property
File
type : uri
path : string
lexicalForm : string
mimeType : string
... Webpage
url : string
summary : string StringProperty
Email
subject : string value : string
... IntProperty
value : integer
Blogpost Tweet DatetimeProperty
timestamp : dateTime author : string value : datetime
author : string text : string
... ...

17. def promote(self):
if not hasattr(self, '_promoted'):
if isinstance(self, self.get_class()):
self._promoted = self
else:
self._promoted = getattr(
self, getattr( self.get_class(),
self._root_class_name + '_ptr'
).field.related_query_name() )
return self._promoted
SELECT … FROM "pim_annotation"
LEFT OUTER JOIN "pim_related"
ON ("pim_annotation"."id" = "pim_related"."annotation_ptr_id")
WHERE ( "pim_annotation"."thing_id" = E'∴'
OR "pim_related"."related_to_id" = E'∴' )
ORDER BY "pim_annotation"."lastModified" DESC;

19. NoSQL?

20. NoSQL?

21. “How FriendFeed
uses MySQL to store
schema-less data”
-- http://bret.appspot.com/entry/how-friendfeed-uses-mysql

22. Use PostgreSQL
as RDF document-
oriented database

23. Single Thing class
Many concrete
Annotation classes
TEXT field `attributes’
for free-form RDF

24. No JOINs

25. Rules
Keep existing code
(when possible)
Keep existing behaviour
Keep magic localized

26. I. New code
RDFField & URIRefField using
rdflib
Customized RDF serialization

27. II. Analysis
Extract existing schema from
model code
Declare schema in RDF
Glue code for attribute access to
RDF data

28. nfo:Bookmark rdfs:subClassOf pimo:Thing ;
grfs:className "Bookmark" ;
grfs:label "nfo:Bookmark" ;
grfs:pluralLabel "nfo:Bookmarks" ;
grfs:isGuiCreatable "true"^^xsd:boolean ;
grfs:labelProperties ( dct:title rdfs:label nfo:bookmarks ) ;
grfs:shortSummaryInfoProperties ( nfo:bookmarks ) .
nao:description a rdfs:Property ;
rdfs:domain pimo:Thing, pimo:Person, pimo:Event, nfo:Bookmark,
pimo:Project, pimo:Person, nfo:FileDataObject, pimo:Topic, pimo:Location,
pimo:Task, nmo:Email, pimo:Question, pimo:Organization, pimo:Note,
pimo:Collection;
grfs:attributeName "description" ;
grfs:ftsFullText "true"^^xsd:boolean ;
grfs:additionalShortSummaryInfo "false"^^xsd:boolean .
nfo:bookmarks a rdfs:Property ;
rdfs:domain nfo:Bookmark ;
grfs:attributeName "bookmarks" ;
grfs:ftsFullText "true"^^xsd:boolean ;
grfs:additionalShortSummaryInfo "true"^^xsd:boolean .

29. SELF_URI = URIRef('self:#')
def __getattr__(self, attr):
property_uri = attribute_uri(attr)
oo = list(self.attributes.objects(
self.SELF_URI,
property_uri))
if not oo:
raise AttributeError(attr)
if len(oo) == 1:
return oo[0].toPython()
return set([o.toPython() for o in oo])

30. III. Details
Run test suite
Find most common error
Adapt test or change code
Repeat for three weeks

31. IV. Tweaking
Data migration
Refactor APIs
Optimize

32. Success!

33. Things come together.
http://www.getrefinder.com/