VisibleGovernment.ca Expense Visualizer Pilot - Montreal on Rails

Citizen Activism using Scrubyt and RoR

 Only partially available online  Formatted as web page or PDF  Hard to search  Can’t subscribe  Can’t visualize  Can’t re-use

Publishing Structured Data Visualization Feeds • Makes it easy to find new • Ability to subscribe to patterns. interesting data • Data streams can be ‘mashed’ in new ways. Collaborative Crowdsourcing Organization • Combines skills and input of • Tagging, Voting, Sharing large numbers of people

• Governments publish Governments data streams publish data streams • 3rd parties create tools for analysis and oversight 3rd Issues are Party Citizens monitor data • Citizens collaboratively resolved Tools streams monitor their government • Citizens detect issues, Issues are detected give feedback • Issues are resolved

 Government has little incentive ▪ Usually has disincentive Why can’t the  Don’t want a single government do monolithic solution everything? ▪ Want to allow evolution of best- of-breed tools  Tools created by citizens, for citizens

 Focus:  US Congress  California Legislature  Gives grants to online transparency tools  $3.5 M Seed

A recent US Congress bill Groups for Groups bill against bill

Votes Donations

Publishing Structured Data Visualization Feeds • MAPLight makes relationship • MAPLight is a mashup of between money and votes data streams from different visible. sources. Collaborative Crowdsourcing Organization • Thousands of journalists, • Advocacy group tags advocates, and citizens can donating companies as browse data and flag issues. belonging to interest groups.

 Accelerate online transparency Ideas  Raise Awareness  With public  With government Skills  Raise Money  Fund External Development: Funds  Grants  Contests

Prove Concept Get Publicity Direct Attention and Money and to Online Tools For Transparency Raise Awareness Show What’s Possible

 2003 Directive: Must publish travel and hospitality expenses on the web  No standards for presentation defined

124 Departments - All different

Standardize Stream Visualize • Scrape data into • Publish RSS • Provide basic standard format feeds visualization app • Run contest

1. LEARNING TEMPLATE 2. PRODUCTION SCRAPER Input Input • Example Page • Any Page with • Example Text Same Format Output: Output: • XML • Production Scraper • XML

 Create a system where non-coders can train a scraper.

PRO CON  Ability to use ‘learning’  Learning mode fails hard example (sometimes)  Doesn’t always learn  Syntax integrates XML builder  Supports all hpricot Xpath operations Note: For compatibility reasons, this project uses an older version of scrubyt. Issues may be fixed in newer version.

 Create a system where non-coders can train a scraper. .... Didn’t work.

Still need coders w/ the following expertise: 1. XPath XML resolution 2. Regular Expressions 3. Firebug

1. Open This Link 2. Paste This Text

...created in the background

Go To Next Level

Split Level: Two Types of Links Open This Link

Select Element Get the XPath

Split Level: Two Types of Links

...created in the background

Test Random Reports Send Home

 Goal: Finish scraping in one day  12/124 Completed: 112 to go  5-20 Volunteers  5-20 min. per department  Downloadable app w/ setup instructions  Integrated examples  Benefits:  Excuse to use scrubyt, firebug  On-site tutorial + guidance  Easy intro to a Rails App

Jennifer Bell visiblegovernment.ca

VisibleGovernment.ca Expense Visualizer Pilot - Montreal on Rails

by Jennifer Bell on Nov 19, 2008

Accessibility

Categories

Upload Details

Usage Rights

1 Embed 1

Statistics

VisibleGovernment.ca Expense Visualizer Pilot - Montreal on Rails — Presentation Transcript