The Development Infrastructure of the TYPO3 Project
Brussels, 02. - 03.02.2013 1The Development Infrastructure of the TYPO3 project Lizenz: CC BY-SA 3.0 (pictures of other authors excluded)
Brussels, 02. - 03.02.2013What to expect from this talk? 2✘ Learn to set up a perfect development infrastructure✘ See the perfect infrastructure of the TYPO3 project✔ Learn how people in another FLOSS project work✔ See an infrastructure built from 100% open-source? Find inspiration for your project? Get in touch with me/us for collaboration firstname.lastname@example.org - we’re happy to share!
Brussels, 02. - 03.02.2013Steffen Gebert 3 • TYPO3 CMS Core Team Member (2010-2012) • TYPO3 Server Admin Team Member (since 2011) • PhD Student in Computer Science (Communication Networks, since 2011) • E-Mail: email@example.com Twitter: @StGebert
Brussels, 02. - 03.02.2013 Questions! 4 • Happy to answer your questions! • Come down here after the talk! comments included! • Slides availableFoto “Question mark”: Yello-Doghttp://www.sxc.hu/photo/1279316 http://slideshare.net/stephenking/nice that you found the slides - have fun ☺
Brussels, 02. - 03.02.2013What’s TYPO3? 5 • TYPO3 CMS • Started: 1997, ﬁrst release: 1999 • More than 5M downloads • PHP-based, GPLv2 • New products • TYPO3 Flow: PHP framework • “TYPO3 Flow and the Joy of Development” by Robert Lemke from yesterday • TYPO3 Neos • New CMS based on TYPO3 Flow - alpha state
Brussels, 02. - 03.02.2013Who’s TYPO3? 6 • TYPO3 is community-driven open-source • no company behind, TYPO3 Association for ﬁnancial support • Core teams steering development (99.x% volunteer work) • CMS with ~25ppl., Flow/Neos with ~12ppl. Photo: Tobias Liebig
Brussels, 02. - 03.02.2013 7 Requirements of an Open Source ProjectPhoto by brokenarts:http://www.sxc.hu/photo/207291
Brussels, 02. - 03.02.2013 Requirements 8 • Collaboration & Communication • Source Code & Versioning • Quality Assurance & Testing • Documentation & TranslationPhoto by brokenarts:http://www.sxc.hu/photo/207291 There are couple of temas working on tasks other than development - Design, Usability, Marketing, Screencast, Event teams - Teams and committees of the TYPO3 Association (Board, Expert Advisory Board, Budget Control Committee)
Brussels, 02. - 03.02.2013Using own infrastructure 9 • We’ve been there before Github ☺ • Sourceforge.net initially used for CVS • Got hacked, often unreliable • Owning our data (esp. issue tracker) • Own account management • Provide central place and connected services for the TYPO3 community • Nevertheless, open towards great, free hosted services for less critical things • Human resources are a problem, of course
Brussels, 02. - 03.02.2013 10 Communication and CollaborationImage by fangol:http://www.sxc.hu/photo/1269811
Brussels, 02. - 03.02.2013Communication in the TYPO3 Project 11 • Mailing lists and news groups used since 2003 • Ten years later (last Monday): forum.typo3.org
Brussels, 02. - 03.02.2013 Forum and Ofﬁcial Communication 12 • Forum • Based on FUDforum • Bi-directional sync with news groups • Easier access to ofﬁcial channels (we hope..) • Decisions take place in lists • Transparency towards more people • Avoid hidden communication • Avoid surprises (well, we still fail too often)Image by svilen001:http://www.sxc.hu/photo/1165445
Brussels, 02. - 03.02.2013 Forge Platform 13 • Based on Redmine • Project space for 2400+ projects • Core projects, Community Extensions, Other teams • Features Want to contribute? ➪ See who’s responsible • Membership listing not the • Issue tracker best one ☹ • Wiki, Repo Browser, Roadmap News, Time Tracking forge.typo3.orgIt’s often a bit chaotic in a big open source community. Being able to ﬁnd the persons incharge is important!Forge is the place where to search.We know that we’re running a very outdated redmine version. We’re sorry and ashamed.We’re working on an upgrade.
Brussels, 02. - 03.02.2013 14 Source Code and VersioningSource:http://nvie.com/posts/a-successful-git-branching-model/
Brussels, 02. - 03.02.2013 Source Code Versioning 15 • TYPO3 is using Git since 2010/11 • .. but we use a special ﬂavor of Git • Unfortunately still lot of extensions restricted to subversionYes.. it’s true. The high Subversion usage is our (=admin team) fault.. we still haven’tlaunched official git support for community extensions. But it’s near.
Brussels, 02. - 03.02.2013 16 Quality Assurance and TestingPhoto by bredmakers:http://www.sxc.hu/photo/1280927
Brussels, 02. - 03.02.2013 Motivation for Code Reviews 17 • Bugs do exist • Ensure code quality through peer-reviews (4 eyes in our case) • Discussing code improves it • Reviews before merge • Learn from each other • Make development transparent, involves community • Using Gerrit Code Review as Git server and review toolPhoto by asifthebes:http://www.sxc.hu/photo/912841 every change has to be reviewed by at least two other persons, one of them being team member
Brussels, 02. - 03.02.2013 Gerrit Code Review 18 review.typo3.orgAll code-changes for core projects are peer-reviewed by at least two people (one of themteam member)(Disclaimer: except trivial changes)
Brussels, 02. - 03.02.2013More About Gerrit 21 Interested in more Gerrit details? “Using Gerrit Code Review in an open-source project” by myself :-) Here at 15:40
Brussels, 02. - 03.02.2013Continuous Integration and Code Quality 22 • ci.typo3.org runs our Jenkins server • PHP lint (syntax check) • PHP CodeSniffer (enforcement of Coding Guide Lines) • PHP Mess detector • Automatic down-vote in Gerrit in case of violations • metrics.typo3.org runs Sonar • Code Metrics • For Core and Extensions, 5600+ projects
Brussels, 02. - 03.02.2013TravisCI 24 • Spins up virtual machines (using Vagrant and Chef) • Runs your unit tests in different an environments • Example: php: -‐ 5.2 -‐ 5.3 -‐ 5.4 PHP 5.2 PHP 5.3 PHP 5.4 env: -‐ DB=mysql MySQL ✔ ✔ ✔ -‐ DB=pgsql PostgreSQL ✔ ✘ ✔
Brussels, 02. - 03.02.2013 25 Documentation and TranslationPhoto by topfer: Photo by ftibor:http://www.sxc.hu/photo/969875 http://www.sxc.hu/photo/533821
Brussels, 02. - 03.02.2013 Translating TYPO3 26 • Pootle-based • Core + Extensions can be translated • Language packages generated, provided for download translation.typo3.orgPhoto by ftibor:http://www.sxc.hu/photo/533821
Brussels, 02. - 03.02.2013 Documentation 27 • Was a raw point for long time • Used OpenOfﬁce 1.x .sxw ﬁles since ages • Migration to DocBook • started / planned, but format turned out to be too complicated • Migration to reStructuredText (reST) • Started ~2011 • Milestone reached with TYPO3 6.0 (Oct. 2012): docs.typo3.org • TYPO3 CMS Core documentation converted to reST Photo by topfer: http://www.sxc.hu/photo/969875- During .sxw times, we used a wiki page to write down pending documentation changes,which sucked- docbook format is based on xml, pretty heavyweight. Probably because of markdown,people feel better with plain-text?
Brussels, 02. - 03.02.2013 reStructuredText 28 created for • Plain-text format, similar to markdown docs.python.org • Parsed and rendered through sphinx • Rendering chain at TYPO3 • Managed through TYPO3 Flow app lower barrier for adoption • Can import old .sxw ﬁles • You can view sources on docs.typo3.org! • Changes reviewed through GerritDocumentation is parsed and rendered through the tools docutils and sphinx (both createdfor docs.python.org).http://docutils.sourceforge.net/rst.htmlIntersphinx allows cross-manual referencingBeing able to import old manuals hopefully lowers the barrier to migrate documentation fromsxw to reST
Brussels, 02. - 03.02.2013 Solutions at a Glance 29 • Collaboration & Communication • Mailing lists / NNTP / Forum using FUDforum, Redmine • Source Code & Versioning • Git with Gerrit as server and review tool • Quality Assurance and Testing • Gerrit, Jenkins with couple PHP tools, TravisCI • Documentation & Translation • reStructuredText, sphinx, PootleHonestly, we use couple of more tools:- mailman runs our mailing lists.. what else (ah, we have a sympa server vor TYPO3 Assocmember lists, but I wouldn’t recommend that software)- etherpad-lite for meeting notes and collaborative writing: http://notes.typo3.org- BigBlueButton web conferencing server: http://bigbluebutton.typo3.org- and we have been using Mantis as bug tracker for years. Mantis (http://bugs.typo3.org) isnow archived and Redmine (http://forge.typo3.org) acts as tracker for *all* TYPO3 projects.- yes, we are still running a subversion server for some thousand projects-
Brussels, 02. - 03.02.2013 Questions! 30 • Happy to answer your questions! • @StGebert on Twitter • firstname.lastname@example.org comments included! • Slides availableFoto “Question mark”: Yello-Doghttp://www.sxc.hu/photo/1279316 http://slideshare.net/stephenking/
inspiring people to share inspiring people to share