Hybrid Apps (Native + Web)
       via QtWebKit

                    SF BAY MEEGO NETWORK
                         JUNE 22, 2011




      ARIYA HIDAYAT
   ENGINEERING DIRECTOR
whoami
Overview
Going Hybrid?

   Platform Integration      Security



                            App Store/
    Advanced Technologies   Marketplace
WebKit Everywhere

 Browser



 Devices



Runtime
~2000 commits/month
            History
            90000
            80000
            70000
            60000
Revisions




            50000
            40000
            30000
            20000
            10000
               0
                    0   1   2   3   4     5     6    7     8     9    10
                                        Years
Extensive Tests



 ≈ 20,000 tests    tests
                  904 MB   the rest
                           229 MB
Workflow
                             quality control


    1     Every commit needs to be reviewed



    2     Broken commit must be reverted

                     zero-regression policy
Level of Involvement
    Contributor
                       after 10-20 patches




                                              after 80 patches
                       Committer

               ≈ 150                                     Reviewer

     checks in reviewed patches              ≈ 90
                                                           accept or reject patches
WebKit Reviewers
              Apple
               39




                              Google
                                25
      Misc
       11

       1     RIM
                      Nokia
              7
                        7
Components of WebKit
        DOM              CSS

              WebCore             SVG

 HTML
                   rendering
                                        JavaScriptCore


                 WebKit Library
Platform Abstractions

       Network     Unicode      Clipboard


       Graphics     Theme        Events


       Thread     Geolocation    Timer
WebCore
Di erent “Ports”                                      graphics


GraphicsContext
                      Mac        Chromium            Qt     Gtk


                                   Skia                          Cairo
                  CoreGraphics
                                                 QPainter


                                          graphics stack
Use
Web Browsers
                          Arora




  Demo Browser         http://arora.googlecode.com

       demos/browser
QWebView, QWebPage, QWebFrame
                    QWebView (widget)




                  QWebPage (object)


                            QWebFrame (object)

                     At least one, i.e. the main frame of the page
Using WebView


 QWebView webView;
 webView.show();
 webView.setUrl(QUrl("http://meego.com"));
Contents via String


 QWebView webView;
 webView.show();
 webView.setContent("<body>Hello, MeeGo!</body>");
Contents via Resource
                      <RCC>
                          <qresource prefix="/">
                              <file>content.html</file>
                          </qresource>
                      </RCC>


QWebView webView;
webView.show();
webView.setUrl(QUrl("qrc:/content.html"));
Capture to Image

QWebPage page;
QImage image(size, QImage::Format_ARGB32_Premultiplied);
image.fill(Qt::transparent);
QPainter p(&image);
page.mainFrame()->render(&p);
p.end();
image.save(fileName);



           http://labs.qt.nokia.com/2009/01/15/capturing-web-pages/
SVG Rasterizer




     http://labs.qt.nokia.com/2008/08/06/webkit-based-svg-rasterizer/
Search + Preview




     http://labs.qt.nokia.com/2008/11/04/search-with-thumbnail-preview/
Bridging the Two Worlds
Exposing to the Web world

QWebFrame::addToJavaScriptWindowObject(QString, QObject*)



                                  Public functions
                                  Object properties
                                    Child objects
Exposing to the Web world
page()->mainFrame()->addToJavaScriptWindowObject("Dialog", new Dialog);



                 class Dialog: public QObject
                 {
                     Q_OBJECT

                 public:
                     Dialog(QObject *parent = 0);

                 public slots:
                     void showMessage(const QString& msg);
                 };
Exposing to the Web world

   <input type="button" value="Try this"
   onClick="Dialog.showMessage('You clicked me!')">




    instance of
                             public slot
   Dialog object
Signal and Slot
              signal


   foobar.modified.connect(refresh);

    QObject instance       JavaScript function


  foobar.modified.connect(obj, refresh);
                             any object
Triggering Action from Native
class Stopwatch: public QObject
{                                     Stopwatch::Stopwatch(QObject *parent)
    Q_OBJECT                              : QObject(parent)
                                          , m_index(0)
public:                               {
    Stopwatch(QObject *parent = 0);       QTimer *timer = new QTimer(this);
                                          timer->setInterval(1000);
signals:                                  connect(timer, SIGNAL(timeout()), SLOT(update()));
    void tick(int t);                     timer->start();
                                      }
private slots:
    void update();                    void Stopwatch::update()
                                      {
private:                                  emit tick(m_index++);
    int m_index;                      }
};
Triggering Action from Native
   instance of
 Stopwatch object
                                  signal


      <script>
      Stopwatch.tick.connect(function(t) {
          document.getElementById('tick').innerText = t;
      });
      </script>
Coming back to the Native

  QVariant QWebFrame::evaluateJavaScript(QString)


      mostly key-value pair
     (like JavaScript objects)
Other Bridging Solutions


                                           Custom network protocol
                                           QNetworkAccessManager




       http://labs.qt.nokia.com/2010/11/16/some-webkit-hybrid-stu /
Platform Integration

                   Menu and Menu Bar

                                              Dialogs
                          Application
   System Access



                               Notifications
Debugging


     Web Inspector




settings()->setAttribute(QWebSettings::DeveloperExtrasEnabled, true);
Deployment

     O ine                Packaging
 Cache Manifest      QtWebKit boilerplate
  Local Storage
                               Tools

              PhoneGap
          AppUp Encapsulator
Real-world Hybrid Apps




      Ext Designer       Sencha Animator
Technologies
Key Technologies
JavaScript
Libraries and Frameworks
Content Editing




      http://labs.qt.nokia.com/2009/03/12/wysiwyg-html-editor/
Consume Web 2.0




http://labs.qt.nokia.com/2009/03/08/creating-a-google-chat-client-in-15-minutes/
Vector Graphics




           http://raphaeljs.com/polar-clock.html
Canvas-based Game




      http://ariya.blogspot.com/2010/09/invade-destroy.html
Diagrams & Visualization




    JavaScript InfoVis Toolkit   http://thejit.org/
CSS3 Animations




      http://mozillademos.org/demos/planetarium/demo.html
GPU FTW

Accelerated Composition

                                          Element    GL texture


                                               Movement
                                          Transformation matrix

http://www.webkit.org/blog-files/leaves/
Sencha Animator
Device Access




          http://ariya.github.com/js/marblebox/
WebGL for 3-D




        http://webglsamples.googlecode.com/hg/aquarium/aquarium.html
PhiloGL: WebGL Framework




            http://senchalabs.github.com/philogl/
Tools
Web Inspector
Network Log

28: GET http://www.google.com/m/gp
292: Response 200 application/xhtml+xml; charset=UTF-8 0 bytes http://www.google.com/m/gp
311: GET data:image/gif;base64,R0lGODlhiA...
312: GET data:image/gif;base64,R0lGODlhJA...
312: GET data:image/gif;base64,R0lGODlhGA...
312: Response 0 image/gif 3611 bytes data:image/gif;base64,R0lGODlhiA...
312: Finish fail data:image/gif;base64,R0lGODlhiA...
312: Response 0 image/gif 284 bytes data:image/gif;base64,R0lGODlhJA...
312: Finish fail data:image/gif;base64,R0lGODlhJA...
312: Response 0 image/gif 178 bytes data:image/gif;base64,R0lGODlhGA...
312: Finish fail data:image/gif;base64,R0lGODlhGA...
317: Response 200 application/xhtml+xml; charset=UTF-8 0 bytes http://www.google.com/m/gp
324: Finish fail http://www.google.com/m/gp
328: GET http://www.google.com/m/gn/user?...
329: Finish success http://www.google.com/m/gn/user?...
Typical Scenario
                          This is
                         awesome!




                    un
             t to r
       f orge ests
           the t
Test Framework

specrunner SpecRunner.html
5 specs, 0 failures in 0.013s

specrunner SpecRunner.html
FAIL: 5 specs, 1 failure in 0.014s




               Selenium, Watir, Squish Web, JSUnit, Jasmine,
                                 QUnit, ...
Headless WebKit

if (phantom.state.length === 0) {
    phantom.state = 'pizza';
    phantom.open('http://www.google.com/m/local?site=local&q=pizza+in+new+york');
} else {
    var list = document.querySelectorAll('div.bf');
    for (var i in list) {
        console.log(list[i].innerText);
    }
    phantom.exit();
}




                               http://phantomjs.org
UI Designer
IDE: AKShell
IDE: Cloud9
Recorder and Replayer
Get + Compile
Using git

  git clone git://git.webkit.org/WebKit.git
  cd WebKit



                       ≈ 1.2 GB .git
Build

Tools/Scripts/build-webkit --qt




      --debug for “Debug” mode
Launch

Tools/Scripts/run-launcher --qt
Conclusion
Today
       Web technologies are moving really fast
Various frameworks and libraries boost the productivity
         Hybrid approach helps the migration
                Tools need to catch-up
Future
 More bindings to the native world
 Platinum-grade productivity tools
Ubiquitous mesh and cloud solutions
THANK YOU!

         ariya.hidayat@gmail.com


         ariya.blogspot.com


         ariyahidayat

Hybrid Apps (Native + Web) via QtWebKit

  • 1.
    Hybrid Apps (Native+ Web) via QtWebKit SF BAY MEEGO NETWORK JUNE 22, 2011 ARIYA HIDAYAT ENGINEERING DIRECTOR
  • 2.
  • 3.
  • 4.
    Going Hybrid? Platform Integration Security App Store/ Advanced Technologies Marketplace
  • 5.
  • 6.
    ~2000 commits/month History 90000 80000 70000 60000 Revisions 50000 40000 30000 20000 10000 0 0 1 2 3 4 5 6 7 8 9 10 Years
  • 7.
    Extensive Tests ≈20,000 tests tests 904 MB the rest 229 MB
  • 8.
    Workflow quality control 1 Every commit needs to be reviewed 2 Broken commit must be reverted zero-regression policy
  • 9.
    Level of Involvement Contributor after 10-20 patches after 80 patches Committer ≈ 150 Reviewer checks in reviewed patches ≈ 90 accept or reject patches
  • 10.
    WebKit Reviewers Apple 39 Google 25 Misc 11 1 RIM Nokia 7 7
  • 11.
    Components of WebKit DOM CSS WebCore SVG HTML rendering JavaScriptCore WebKit Library
  • 12.
    Platform Abstractions Network Unicode Clipboard Graphics Theme Events Thread Geolocation Timer
  • 13.
    WebCore Di erent “Ports” graphics GraphicsContext Mac Chromium Qt Gtk Skia Cairo CoreGraphics QPainter graphics stack
  • 14.
  • 15.
    Web Browsers Arora Demo Browser http://arora.googlecode.com demos/browser
  • 16.
    QWebView, QWebPage, QWebFrame QWebView (widget) QWebPage (object) QWebFrame (object) At least one, i.e. the main frame of the page
  • 17.
    Using WebView QWebViewwebView; webView.show(); webView.setUrl(QUrl("http://meego.com"));
  • 18.
    Contents via String QWebView webView; webView.show(); webView.setContent("<body>Hello, MeeGo!</body>");
  • 19.
    Contents via Resource <RCC> <qresource prefix="/"> <file>content.html</file> </qresource> </RCC> QWebView webView; webView.show(); webView.setUrl(QUrl("qrc:/content.html"));
  • 20.
    Capture to Image QWebPagepage; QImage image(size, QImage::Format_ARGB32_Premultiplied); image.fill(Qt::transparent); QPainter p(&image); page.mainFrame()->render(&p); p.end(); image.save(fileName); http://labs.qt.nokia.com/2009/01/15/capturing-web-pages/
  • 21.
    SVG Rasterizer http://labs.qt.nokia.com/2008/08/06/webkit-based-svg-rasterizer/
  • 22.
    Search + Preview http://labs.qt.nokia.com/2008/11/04/search-with-thumbnail-preview/
  • 23.
  • 24.
    Exposing to theWeb world QWebFrame::addToJavaScriptWindowObject(QString, QObject*) Public functions Object properties Child objects
  • 25.
    Exposing to theWeb world page()->mainFrame()->addToJavaScriptWindowObject("Dialog", new Dialog); class Dialog: public QObject { Q_OBJECT public: Dialog(QObject *parent = 0); public slots: void showMessage(const QString& msg); };
  • 26.
    Exposing to theWeb world <input type="button" value="Try this" onClick="Dialog.showMessage('You clicked me!')"> instance of public slot Dialog object
  • 27.
    Signal and Slot signal foobar.modified.connect(refresh); QObject instance JavaScript function foobar.modified.connect(obj, refresh); any object
  • 28.
    Triggering Action fromNative class Stopwatch: public QObject { Stopwatch::Stopwatch(QObject *parent) Q_OBJECT : QObject(parent) , m_index(0) public: { Stopwatch(QObject *parent = 0); QTimer *timer = new QTimer(this); timer->setInterval(1000); signals: connect(timer, SIGNAL(timeout()), SLOT(update())); void tick(int t); timer->start(); } private slots: void update(); void Stopwatch::update() { private: emit tick(m_index++); int m_index; } };
  • 29.
    Triggering Action fromNative instance of Stopwatch object signal <script> Stopwatch.tick.connect(function(t) { document.getElementById('tick').innerText = t; }); </script>
  • 30.
    Coming back tothe Native QVariant QWebFrame::evaluateJavaScript(QString) mostly key-value pair (like JavaScript objects)
  • 31.
    Other Bridging Solutions Custom network protocol QNetworkAccessManager http://labs.qt.nokia.com/2010/11/16/some-webkit-hybrid-stu /
  • 32.
    Platform Integration Menu and Menu Bar Dialogs Application System Access Notifications
  • 33.
    Debugging Web Inspector settings()->setAttribute(QWebSettings::DeveloperExtrasEnabled, true);
  • 34.
    Deployment O ine Packaging Cache Manifest QtWebKit boilerplate Local Storage Tools PhoneGap AppUp Encapsulator
  • 35.
    Real-world Hybrid Apps Ext Designer Sencha Animator
  • 36.
  • 37.
  • 38.
  • 39.
  • 40.
    Content Editing http://labs.qt.nokia.com/2009/03/12/wysiwyg-html-editor/
  • 41.
  • 42.
    Vector Graphics http://raphaeljs.com/polar-clock.html
  • 43.
    Canvas-based Game http://ariya.blogspot.com/2010/09/invade-destroy.html
  • 44.
    Diagrams & Visualization JavaScript InfoVis Toolkit http://thejit.org/
  • 45.
    CSS3 Animations http://mozillademos.org/demos/planetarium/demo.html
  • 46.
    GPU FTW Accelerated Composition Element GL texture Movement Transformation matrix http://www.webkit.org/blog-files/leaves/
  • 47.
  • 48.
    Device Access http://ariya.github.com/js/marblebox/
  • 49.
    WebGL for 3-D http://webglsamples.googlecode.com/hg/aquarium/aquarium.html
  • 50.
    PhiloGL: WebGL Framework http://senchalabs.github.com/philogl/
  • 51.
  • 52.
  • 53.
    Network Log 28: GEThttp://www.google.com/m/gp 292: Response 200 application/xhtml+xml; charset=UTF-8 0 bytes http://www.google.com/m/gp 311: GET data:image/gif;base64,R0lGODlhiA... 312: GET data:image/gif;base64,R0lGODlhJA... 312: GET data:image/gif;base64,R0lGODlhGA... 312: Response 0 image/gif 3611 bytes data:image/gif;base64,R0lGODlhiA... 312: Finish fail data:image/gif;base64,R0lGODlhiA... 312: Response 0 image/gif 284 bytes data:image/gif;base64,R0lGODlhJA... 312: Finish fail data:image/gif;base64,R0lGODlhJA... 312: Response 0 image/gif 178 bytes data:image/gif;base64,R0lGODlhGA... 312: Finish fail data:image/gif;base64,R0lGODlhGA... 317: Response 200 application/xhtml+xml; charset=UTF-8 0 bytes http://www.google.com/m/gp 324: Finish fail http://www.google.com/m/gp 328: GET http://www.google.com/m/gn/user?... 329: Finish success http://www.google.com/m/gn/user?...
  • 54.
    Typical Scenario This is awesome! un t to r f orge ests the t
  • 55.
    Test Framework specrunner SpecRunner.html 5specs, 0 failures in 0.013s specrunner SpecRunner.html FAIL: 5 specs, 1 failure in 0.014s Selenium, Watir, Squish Web, JSUnit, Jasmine, QUnit, ...
  • 56.
    Headless WebKit if (phantom.state.length=== 0) {     phantom.state = 'pizza';     phantom.open('http://www.google.com/m/local?site=local&q=pizza+in+new+york'); } else {     var list = document.querySelectorAll('div.bf');     for (var i in list) {         console.log(list[i].innerText);     }     phantom.exit(); } http://phantomjs.org
  • 57.
  • 58.
  • 59.
  • 60.
  • 61.
  • 62.
    Using git git clone git://git.webkit.org/WebKit.git cd WebKit ≈ 1.2 GB .git
  • 63.
    Build Tools/Scripts/build-webkit --qt --debug for “Debug” mode
  • 64.
  • 65.
  • 66.
    Today Web technologies are moving really fast Various frameworks and libraries boost the productivity Hybrid approach helps the migration Tools need to catch-up
  • 67.
    Future More bindingsto the native world Platinum-grade productivity tools Ubiquitous mesh and cloud solutions
  • 68.
    THANK YOU! ariya.hidayat@gmail.com ariya.blogspot.com ariyahidayat