Your SlideShare is downloading. ×
0
P2P Bug Tracking with SD
                        http://syncwith.us

curl fsck.com/sd|perl;
export PATH=~/sd/bin:$PATH;
sd...
Hi!
I’m Jesse (obra)
I own a small software
      company
(Best Practical)
I’ve been making issue
  trackers since 1995
Our software
has some bugs
All software
has some bugs
(All software
is made of bugs)
I spend a lot of time
    on airplanes...
...and at conferences
     with bad wifi
I need to keep track of
our bugs and our work
I’m the boss
I have no excuse for
 not doing my work
I need to keep track of
our bugs and our work
even when I don’t have
    Internet access
I’ve tried everything
Text files
Text files in
version control
IMAP Servers
RSS Feeds
Running RT on
  my laptop
Keeping browsers open
Nothing was quite right
So we built SD
SD is a Bug Tracker
SD is a Distributed
   Bug Tracker
SD Features
Tracks bugs            P2P Sync
CLI                    Trac Sync
Web UI                 RT Sync
Scriptable    ...
Principles of distributed
       computing
The network is reliable
Latency is zero
Bandwidth is infinite
The network is secure
Transport cost is zero
The network is
homogeneous
Topology doesn't
    change
There is one
administrator
Principles of distributed
       computing
ie s
      la c
 Principles of distributed
    l
F a       computing
Those are all LIES
The network is not
      reliable
SD isn’t network-
  dependent
Latency hurts
SD runs at the edge
Bandwidth is always
    a problem
SD knows which
changesets you’ve already
          seen
The network is
  insecure
SD doesn’t depend on a
 network security layer
Topology is
unpredictable
  and fluid
SD is topology-agnostic
There is no
administrator
(But there are many
  people who think
they’re administrators)
SD lets you implement
policy in the centralized
  systems it syncs to
Transport
costs money
SD doesn’t use much
 bandwidth & can use
non-network substrates
The network is
heterogeneous
SD is designed to sync to
     foreign systems
I didn’t make those up
Bill Joy, Tom Lyon and
  James Gosling did
They missed one
Data is clean
SD doesn’t expect to be
 able to enforce data
    integrity rules
SD runs locally
SD plays well
 with others
It syncs the way you do
Clone a project’s bug
database (over HTTP)
Work offline
Pull changesets from
anyone you work with
Publish your database
  replica with rsync
Topology doesn’t matter...
Don’t worry
It won’t break
SD learns how to
resolve each conflict...
...based on how
everyone else resolves it
Using SD (CLI)
Getting Started
SD Shell

$ sd
./
Getting help

$ sd help
Creating a new project

$ sd init
Project settings

$ sd settings
Config file

$ sd config
Create a bug

$ sd ticket create
Listing bugs

$ sd ticket list
Show a bug

$ sd ticket show
Update a bug

$ sd ticket update
Log

$ sd log
Git Integration

$ git sd
Using SD (Web)
Web? Isn’t SD an
 offline tool?
Local microserver
$ sd browser
Home
Create a ticket
Search
Show a bug
Update a bug
Comments
History
Working with others
Working with others
   (Using SD)
Any topology
It doesn’t matter who
     you sync with
You get all the updates
      eventually
Cloning

$ sd clone
clone makes a replica of
    someone else’s
       database
Pulling

$ sd pull
pull imports unseen
changes from another
   database replica
Publishing

$ sd publish
publish writes out a
copy of your database
  replica for sharing
(As SD changesets
 and static HTML)
Hackathon mode
 (using Bonjour)
Publish your replica

$ sd server
Pull updates

$ sd pull --local
Working with others
(Using other systems)
But you already have a
     bug tracker?
No Problem!
I use at least
 two others.
I wrote at least
  two others.
We designed SD talk to
 foreign bug trackers
RT
(RT::Client::REST)
Hiveminder
(Net::Jifty)
Trac
(Net::Trac)
Google Code
(Net::Google::Code)
GitHub
(Net::GitHub)
Redmine
(Read-only for now)
(Net::Redmine)
Want to help with
Bugzilla, Jira, FogBugz or
   something else?
Work with
  foreign
replicas in
   a Star
Topology
X

    An SD node
    can act as a
      gateway
Clone

$ sd clone
Clone from Google Code

$ sd clone --from gcode:k9mail
Clone from RT
$ sd clone --from "rt:https://rt.cpan.org|DBI|"
Clone from Trac
$ sd clone --from trac:https://trac.parrot.org/parrot
Clone from GitHub
$ git sd clone --from github:miyagawa/remedie
SD reverse engineers a
   database history
Pull

$ sd pull
SD reverse engineers a
partial database history
Push

$ sd push
SD figures out local
 updates and sends
  them upstream
(Then it does a bunch
  of book-keeping)
So what’s that all look
like in the real world?
Google Code                 Alice’s SD                              Bob’s SD

Create some bugs



                        ...
Installing SD
It’s time to get SD
   up and running
SD is in Perl
SD uses 45-90
CPAN modules
Are you afraid?
Are you afraid?
Don’t be
I have a novel idea for
   a Perl application
One-tweet install.
I’m installing #SD (http://syncwith.us)
  after seeing @obra’s talk at #OSCON!
80 bytes! I still have 60
    to work with!
I’m installing #SD (http://syncwith.us)
  after seeing @obra’s talk at #OSCON!
         curl fsck.com/sd|perl;
     export...
You’ll need:

curl, perl 5.8, C compiler
You won’t need:

    CPAN
You won’t need:

to answer prompts
You won’t need:

to fix dependencies
SD is in Perl
SD uses CPAN
  modules.
This is a blessing.
This is a curse.
CPAN
      =
Dependency Hell
When we first built
 SD, we used anything
we thought was useful.
(The first version of SD
 used a SVN backend)
The first ~useful
 version of SD:
123 Dependencies
...a couple hours later
54 Dependencies
Only one needs a
   compiler.
Shipwright gives us
one-command install
curl fsck.com/sd|perl
export PATH=$PATH:~/sd/bin
What’s that do?
curl
perl
$ head /Library/WebServer/Documents/sd

open (my $tar,'|tar xz 2>/dev/null');
while (<DATA>) {
    print $tar $_;
}
close ...
Shipwright installs a few
    Perl modules...
...in order
Scalar-List-Utils        WWW-Mechanize-GZip        Sub-Exporter
String-BufferStack       File-Slurp                Path-Di...
(104 dists with our
   sync plugins)
What’s wrong with SD?
$ sd clone --from http://fsck.com/~jesse/sd-bugs
Don’t want to
 install SD?
Point your browser at
http://fsck.com/~jesse/sd-bugs/html
What’s next?
Indexing
GPG-signed changesets
Actually releasing 1.0
Thanks!

      Questions?

curl fsck.com/sd | perl


  http://syncwith.us
  jesse@bestpractical.com
SD - A peer to peer issue tracking system
SD - A peer to peer issue tracking system
SD - A peer to peer issue tracking system
SD - A peer to peer issue tracking system
SD - A peer to peer issue tracking system
SD - A peer to peer issue tracking system
SD - A peer to peer issue tracking system
SD - A peer to peer issue tracking system
SD - A peer to peer issue tracking system
SD - A peer to peer issue tracking system
SD - A peer to peer issue tracking system
SD - A peer to peer issue tracking system
SD - A peer to peer issue tracking system
SD - A peer to peer issue tracking system
SD - A peer to peer issue tracking system
SD - A peer to peer issue tracking system
SD - A peer to peer issue tracking system
SD - A peer to peer issue tracking system
SD - A peer to peer issue tracking system
SD - A peer to peer issue tracking system
SD - A peer to peer issue tracking system
SD - A peer to peer issue tracking system
SD - A peer to peer issue tracking system
SD - A peer to peer issue tracking system
SD - A peer to peer issue tracking system
SD - A peer to peer issue tracking system
SD - A peer to peer issue tracking system
SD - A peer to peer issue tracking system
SD - A peer to peer issue tracking system
Upcoming SlideShare
Loading in...5
×

SD - A peer to peer issue tracking system

4,076

Published on

Published in: Technology, Business

Transcript of "SD - A peer to peer issue tracking system"

  1. 1. P2P Bug Tracking with SD http://syncwith.us curl fsck.com/sd|perl; export PATH=~/sd/bin:$PATH; sd jesse@bestpractical.com Jesse Vincent
  2. 2. Hi!
  3. 3. I’m Jesse (obra)
  4. 4. I own a small software company
  5. 5. (Best Practical)
  6. 6. I’ve been making issue trackers since 1995
  7. 7. Our software has some bugs
  8. 8. All software has some bugs
  9. 9. (All software is made of bugs)
  10. 10. I spend a lot of time on airplanes...
  11. 11. ...and at conferences with bad wifi
  12. 12. I need to keep track of our bugs and our work
  13. 13. I’m the boss
  14. 14. I have no excuse for not doing my work
  15. 15. I need to keep track of our bugs and our work even when I don’t have Internet access
  16. 16. I’ve tried everything
  17. 17. Text files
  18. 18. Text files in version control
  19. 19. IMAP Servers
  20. 20. RSS Feeds
  21. 21. Running RT on my laptop
  22. 22. Keeping browsers open
  23. 23. Nothing was quite right
  24. 24. So we built SD
  25. 25. SD is a Bug Tracker
  26. 26. SD is a Distributed Bug Tracker
  27. 27. SD Features Tracks bugs P2P Sync CLI Trac Sync Web UI RT Sync Scriptable Hiveminder Sync Works offline GitHub Sync Attachments Google Code Sync Comments Conflict Resolution Customizable workflow Git integration Custom properties Darcs integration
  28. 28. Principles of distributed computing
  29. 29. The network is reliable
  30. 30. Latency is zero
  31. 31. Bandwidth is infinite
  32. 32. The network is secure
  33. 33. Transport cost is zero
  34. 34. The network is homogeneous
  35. 35. Topology doesn't change
  36. 36. There is one administrator
  37. 37. Principles of distributed computing
  38. 38. ie s la c Principles of distributed l F a computing
  39. 39. Those are all LIES
  40. 40. The network is not reliable
  41. 41. SD isn’t network- dependent
  42. 42. Latency hurts
  43. 43. SD runs at the edge
  44. 44. Bandwidth is always a problem
  45. 45. SD knows which changesets you’ve already seen
  46. 46. The network is insecure
  47. 47. SD doesn’t depend on a network security layer
  48. 48. Topology is unpredictable and fluid
  49. 49. SD is topology-agnostic
  50. 50. There is no administrator
  51. 51. (But there are many people who think they’re administrators)
  52. 52. SD lets you implement policy in the centralized systems it syncs to
  53. 53. Transport costs money
  54. 54. SD doesn’t use much bandwidth & can use non-network substrates
  55. 55. The network is heterogeneous
  56. 56. SD is designed to sync to foreign systems
  57. 57. I didn’t make those up
  58. 58. Bill Joy, Tom Lyon and James Gosling did
  59. 59. They missed one
  60. 60. Data is clean
  61. 61. SD doesn’t expect to be able to enforce data integrity rules
  62. 62. SD runs locally
  63. 63. SD plays well with others
  64. 64. It syncs the way you do
  65. 65. Clone a project’s bug database (over HTTP)
  66. 66. Work offline
  67. 67. Pull changesets from anyone you work with
  68. 68. Publish your database replica with rsync
  69. 69. Topology doesn’t matter...
  70. 70. Don’t worry
  71. 71. It won’t break
  72. 72. SD learns how to resolve each conflict...
  73. 73. ...based on how everyone else resolves it
  74. 74. Using SD (CLI)
  75. 75. Getting Started
  76. 76. SD Shell $ sd
  77. 77. ./
  78. 78. Getting help $ sd help
  79. 79. Creating a new project $ sd init
  80. 80. Project settings $ sd settings
  81. 81. Config file $ sd config
  82. 82. Create a bug $ sd ticket create
  83. 83. Listing bugs $ sd ticket list
  84. 84. Show a bug $ sd ticket show
  85. 85. Update a bug $ sd ticket update
  86. 86. Log $ sd log
  87. 87. Git Integration $ git sd
  88. 88. Using SD (Web)
  89. 89. Web? Isn’t SD an offline tool?
  90. 90. Local microserver
  91. 91. $ sd browser
  92. 92. Home
  93. 93. Create a ticket
  94. 94. Search
  95. 95. Show a bug
  96. 96. Update a bug
  97. 97. Comments
  98. 98. History
  99. 99. Working with others
  100. 100. Working with others (Using SD)
  101. 101. Any topology
  102. 102. It doesn’t matter who you sync with
  103. 103. You get all the updates eventually
  104. 104. Cloning $ sd clone
  105. 105. clone makes a replica of someone else’s database
  106. 106. Pulling $ sd pull
  107. 107. pull imports unseen changes from another database replica
  108. 108. Publishing $ sd publish
  109. 109. publish writes out a copy of your database replica for sharing
  110. 110. (As SD changesets and static HTML)
  111. 111. Hackathon mode (using Bonjour)
  112. 112. Publish your replica $ sd server
  113. 113. Pull updates $ sd pull --local
  114. 114. Working with others (Using other systems)
  115. 115. But you already have a bug tracker?
  116. 116. No Problem!
  117. 117. I use at least two others.
  118. 118. I wrote at least two others.
  119. 119. We designed SD talk to foreign bug trackers
  120. 120. RT
  121. 121. (RT::Client::REST)
  122. 122. Hiveminder
  123. 123. (Net::Jifty)
  124. 124. Trac
  125. 125. (Net::Trac)
  126. 126. Google Code
  127. 127. (Net::Google::Code)
  128. 128. GitHub
  129. 129. (Net::GitHub)
  130. 130. Redmine (Read-only for now)
  131. 131. (Net::Redmine)
  132. 132. Want to help with Bugzilla, Jira, FogBugz or something else?
  133. 133. Work with foreign replicas in a Star Topology
  134. 134. X An SD node can act as a gateway
  135. 135. Clone $ sd clone
  136. 136. Clone from Google Code $ sd clone --from gcode:k9mail
  137. 137. Clone from RT $ sd clone --from "rt:https://rt.cpan.org|DBI|"
  138. 138. Clone from Trac $ sd clone --from trac:https://trac.parrot.org/parrot
  139. 139. Clone from GitHub $ git sd clone --from github:miyagawa/remedie
  140. 140. SD reverse engineers a database history
  141. 141. Pull $ sd pull
  142. 142. SD reverse engineers a partial database history
  143. 143. Push $ sd push
  144. 144. SD figures out local updates and sends them upstream
  145. 145. (Then it does a bunch of book-keeping)
  146. 146. So what’s that all look like in the real world?
  147. 147. Google Code Alice’s SD Bob’s SD Create some bugs clone --from gcode:k9mail Alice’s SD makes a full copy of the k9mail from google code Someone updates ticket 12 ticket update 13 ticket create pull --from k9mail SD integrates upstream changes from Google Code since Alice ran the ‘clone’ command. push --to k9mail SD figures out what’s changed locally since the original ‘clone’ command. SD then sends any changes that Google Code hasn’t seen upstream as ticket updates. Alice’s new ticket and her updates to ticket 13 are now part of the upstream bug database in Google Code publish --to alice.com:public_html/k9 SD exports a full copy of Alice’s bug database and rsyncs it up to alice.com/k9. Anyone browsing alice.com/k9 can clone her SD replica. (Or see it in static HTML) clone --from http://alice.com/k9 Bob makes a full copy of Alice’s SD replica of the k9mail database. He’s now free to make changes to his local database replica. ticket update 45 ticket create ticket create ticket update 45 publish --to bob.com:public_html/k9 Bob publishes his full K9 bug database. Anyone browsing bob.com/k9 can clone Bob’s K9 replica. Anyone who already has a copy of Alice or Bob’s database can incorporate Bob’s new ticket and his changes to ticket 45. pull --from http://bob.com/k9 Alice’s SD pulls all previously unseen updates from Bob’s published database replica and incorporates them. If Bob's edits to ticket 45 didn't conflict, Alice's SD incorporate them. If they did conflict, Alice will be prompted to resolve the conflict. push --to k9mail Alice publishes all updates her bug database has seen since she last sent updates to Google Code. Google Code now has all of Alice and Bob’s changes.
  148. 148. Installing SD
  149. 149. It’s time to get SD up and running
  150. 150. SD is in Perl
  151. 151. SD uses 45-90 CPAN modules
  152. 152. Are you afraid?
  153. 153. Are you afraid?
  154. 154. Don’t be
  155. 155. I have a novel idea for a Perl application
  156. 156. One-tweet install.
  157. 157. I’m installing #SD (http://syncwith.us) after seeing @obra’s talk at #OSCON!
  158. 158. 80 bytes! I still have 60 to work with!
  159. 159. I’m installing #SD (http://syncwith.us) after seeing @obra’s talk at #OSCON! curl fsck.com/sd|perl; export PATH=~/sd/bin:$PATH; sd
  160. 160. You’ll need: curl, perl 5.8, C compiler
  161. 161. You won’t need: CPAN
  162. 162. You won’t need: to answer prompts
  163. 163. You won’t need: to fix dependencies
  164. 164. SD is in Perl
  165. 165. SD uses CPAN modules.
  166. 166. This is a blessing.
  167. 167. This is a curse.
  168. 168. CPAN = Dependency Hell
  169. 169. When we first built SD, we used anything we thought was useful.
  170. 170. (The first version of SD used a SVN backend)
  171. 171. The first ~useful version of SD:
  172. 172. 123 Dependencies
  173. 173. ...a couple hours later
  174. 174. 54 Dependencies
  175. 175. Only one needs a compiler.
  176. 176. Shipwright gives us one-command install
  177. 177. curl fsck.com/sd|perl export PATH=$PATH:~/sd/bin
  178. 178. What’s that do?
  179. 179. curl
  180. 180. perl
  181. 181. $ head /Library/WebServer/Documents/sd open (my $tar,'|tar xz 2>/dev/null'); while (<DATA>) { print $tar $_; } close $tar; exec("cd sd-build; bin/shipwright-builder --install-base=$ENV{HOME}/sd"); __DATA__ ?I?Isd-build.tar?<is?F???_1?䀌yH?"#‫{$ۿ‬U???H?eH ...
  182. 182. Shipwright installs a few Perl modules...
  183. 183. ...in order
  184. 184. Scalar-List-Utils WWW-Mechanize-GZip Sub-Exporter String-BufferStack File-Slurp Path-Dispatcher Class-Accessor Test-MockModule Module-Pluggable Class-Data-Inheritable Net-GitHub Time-Progress Tree-DAG_Node MIME-Types Carp-Assert Test-Simple Class-Singleton Proc-InvokeEditor Sub-Uplevel Params-Validate Test-HTTP-Server-Simple Test-Exception version Module-Refresh Array-Compare ExtUtils-CBuilder Carp-Assert-More Test-Warn ExtUtils-ParseXS Test-LongString Template-Declare Test-Harness Test-WWW-Mechanize URI File-Temp Test-Script-Run HTTP-Server-Simple Module-Build prophet.git Params-Util DateTime-TimeZone Devel-StackTrace Class-Inspector List-MoreUtils Exception-Class File-ShareDir DateTime-Locale Error DBI DateTime RT-Client-REST DBD-SQLite File-MMagic Email-Address HTML-Tagset Net-Google-Code YAML HTML-Parser Term-ReadLine-Perl Path-Class HTML-Tree Digest-SHA1 Clone Crypt-SSLeay Digest-HMAC Hash-Merge JSON Net-IP UNIVERSAL-isa YAML-Syck Net-DNS UNIVERSAL-can JSON-XS Net-Bonjour Test-MockObject JSON-DWIW TermReadKey Net-Jifty JSON-Any Data-UUID Lingua-EN-Inflect Mouse XML-Atom-SimpleFeed Text-CSV Any-Moose Digest-SHA Net-Trac Compress-Raw-Zlib Exporter-Lite boolean Compress-Raw-Bzip2 IPC-Run3 Time-Piece IO-Compress MIME-Base64-URLSafe Test-MockTime libwww-perl Data-UUID-Base64URLSafe DateTime-Format-Natural HTTP-Response-Encoding Sub-Install sd.git WWW-Mechanize Data-OptList
  185. 185. (104 dists with our sync plugins)
  186. 186. What’s wrong with SD?
  187. 187. $ sd clone --from http://fsck.com/~jesse/sd-bugs
  188. 188. Don’t want to install SD?
  189. 189. Point your browser at http://fsck.com/~jesse/sd-bugs/html
  190. 190. What’s next?
  191. 191. Indexing
  192. 192. GPG-signed changesets
  193. 193. Actually releasing 1.0
  194. 194. Thanks! Questions? curl fsck.com/sd | perl http://syncwith.us jesse@bestpractical.com
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×