How we use Bottle and Elasticsearch

Uploaded on

An Adventure in Sinar Project

An Adventure in Sinar Project

More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads


Total Views
On Slideshare
From Embeds
Number of Embeds



Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

    No notes for slide


  • 1. How We Use Bottle and Elasticsearch An Adventure in Sinar Project
  • 2. A background of Malaysian BillWatcher● The big idea, is to let citizen knows what is happening in parliament, bills being debated and pass● No it doesnt know the bills that is not debated yet.● Its is scraper based, just because the parliament site tend to be stand alone, and nobody bother to go there.● And cheat on twitter notification
  • 3. The result● Didnt quite worked out.● That is not the point of this talk● You can talk to us later on this
  • 4. What we use?● We use bottle micro framework.● Elasticsearch via pyes● sqlalchemy for db abstraction(seriously thinking to move to mongodb instead, idea?)● We use bootstrap for css and their js plugin● with jquery● Beaufifulsoup for scraping(that is our data source)We only going to cover pyes and bottle(talk to me later for everything else)
  • 5. Bottle● A micro framework● Similar to flask!!!● Less feature though● But it is OKAY.
  • 6. Bottle a views● Watcher/blob/master/billwatcher/● Watcher/blob/master/billwatcher/
  • 7. What happen?● Bottle can return a dict, and it automatically output json, if the object in dict is compatible● Or with View Decorator, the dict can be feed to a template
  • 8. bottle the template● Watcher/blob/master/billwatcher/views/list.tpl●
  • 9. What happen?● Bottle read template from a dict/object● like most templating language does.● Have some logic like other, different syntax same idea● Dont have good inheritance though
  • 10. elasticsearch!!!● OK I lied, this is written in JAVA● But hey, it got RESTful API● and pyes did a lot of the stuff for us already
  • 11. Before index● before we do that● we dont need to create a schema● But we do it because we can control the search priority
  • 12. Indexer● Watcher/blob/master/billwatcher/
  • 13. then we index● First flatten your result● convert to dict● index
  • 14. The indexer● Watcher/blob/master/billwatcher/● Watcher/blob/master/billwatcher/
  • 15. now we search!!!!● There is some field inside the search result● Watcher/blob/master/billwatcher/
  • 16. The future● Do localization● Find new data source● Move template to jinja● Seriously thinking of moving to mongodb● Seriously still trying to see a better way to scrape websites