Advanced Web Development

Advanced Web Development Software Strategies for Online Applications Rob Stein, Charlie Moad, Ed Bachta The Indianapolis Museum of Art

Purpose of Today’s Workshop

You will leave today with tools and techniques for building their own online applications .

You will understand the overarching concepts and strategies behind building dynamic online applications and some shortcuts to accelerate your development

Outline

Web App or Web Page

Trends in Web Software Development

BYO API’s

Bite-sized Web Components

Collaborative Development

Drupal Bootcamp

More…

Web Page or Web Application… What’s the Difference?

Web Pages are Built by Producers

They convey a set amount of information

The creator decides what the information is

Generally the user consumes the content once

Web Apps are Driven by the User

Information presented is dynamic and unknown before hand

The user decides the information they want to see

Web applications can be used many times by the same user.

Web Page

Web Application

Web Page

Web Application

The More Things Change The More They Stay the Same…

Attributes of a Web Application

CORE COMPETENCIES OF WEB 2.0 COMPANIES:

Services, not packaged software, with cost-effective scalability

Control over unique, hard-to-recreate data sources that get richer as more people use them

Trusting users as co-developers

Harnessing collective intelligence

Leveraging the long tail through customer self-service

Software above the level of a single device

Lightweight user interfaces, development models, AND business models

Tim O’Reilly – What is Web2.0

So what are the keys to building great web applications?

We’re in Luck!

People have been building robust applications for over 30 years

They’ve figured out many smart strategies for building applications that are easy to grow and maintain.

We’ll be sharing those strategies and techniques with you today and applying them to building great web applications.

Differences from Last Year

Last year we spent a lot of time talking about the foundations of creating solid software applications…

This is still important when you are looking to provide access to your own controlled information to a larger audience

However…

Differences from Last Year

Since last year’s talk software services and API’s / SDK’s of web software has really taken hold in a serious way.

Building True Museum Applications from SAS Components is not so crazy anymore

Last year it would have seemed foolish to suggest that Flickr could be used as the basis for a digital asset management system… this year… not so much (Flickr-LOC, machine tags, geotag support and integration)

How crazy is it to think that we could combine Google Gears, Spreadsheets, Form Creator and Sites to create a simple collection management system?

WebDev 2008

Today, web developers are more like a systems integrator than a hardware manufacturer.

WebDev 2008

The craft of web development is now how you choose to use components to communicate about your content.

WebDev 2008

Skills required are now more about knowing what’s out there and how to structure relationships between components and do the integration pieces needed to bring engaging content to users…

X

That Being Said…

There are still many reasons why you should consider building your own API’s and application infrastructure.

Reasons to build your own API’s

If you have unique information that others are unlikely to support adequately

If you care about security of your data

If you wish to control the distribution of your information

If a guarantee of service is important

Where’s the API for my data?

Strategies for rapidly building an API

Model-driven development

Last year == XMLSchema (still valid)

This year == database as your model

You most likely already have this model

Object Relational Mappers

Object oriented access to a database

No SQL needed

Data model changes picked up automatically

Mostly database agnostic

Object Relational Mappers

ORMs have come a long way

They understand you might want to execute queries directly

Allow custom method additions to objects

http://wiki.rubyonrails.org/rails/pages/ActiveRecord http://www.hibernate.org/ http://www.sqlalchemy.org/ http://propel.phpdb.org/trac/

steve.museum

Model Driven Development

Class generation from database tables

Model changes are picked up automatically

**Auto-generated Code** Do Not Edit! /** * Entity representing the table: term * @package steve.core.data_objects * * @author tech-dudes <steve.tech@steve.museum> * @version 0.001alpha */ class Term extends Entity { function getTableName() { return "steve_term";} function getKey() { return "term_id";} var $termId; function getTermId() { return $this->termId; } function setTermId($term_id) { $this->termId = $term_id;} var $sessionId; function getSessionId() { return $this->sessionId; } function setSessionId($session_id) { $this->sessionId = $session_id;} … PHP

steve.museum

Formalized API

Directs remote access

Explicit security

/** * @package steve.core.api * @author tech-dudes <steve.tech@steve.museum> * @version 0.001alpha */ interface iTermDAO { /** * Create a new term object * @param string $token the auth token * @param int $mimeId the image id to associate the term with * @param string $termText the text for the term * @return Term result * @throws termException * @secure authToken $token */ function createTerm($token, $mimeId, $termText); /** * Save an existing term into the database * @param string $token the auth token * @param Term $term the term to be saved * @throws termException * @secure authToken $token */ function saveTerm($token, $term); ...

steve.museum

Bringing it all together

Implement interfaces

Code access to types

class TermDAO extends ADOHelper implements iTermDAO { /** * Create a new term object * @param string $token the auth token * @param int $mimeId the image id to associate the term with * @param string $termText the text for the term * @return Term result * @throws termException * @secure authToken $token */ function createTerm($token, $mimeId, $termText) { $termObject = new Term(); $termObject->setTerm($termText); $this->create($termObject); // makes new DB row return $termObject; } /** * Save an existing term into the database * @param string $token the auth token * @param Term $term the term to be saved * @throws termException * @secure authUserCheckId $token */ function saveTerm($token, $term) { $this->save($term); // issues a DB update return ; } ...

Introducing Mercury

Adaptive layer for the IMA Collection

Collections Management Digital Asset Management User Generated Content Mercury www.imamuseum.org Exhibition Micro-sites In-Gallery Applications

PHP Adapter Layer

Adding a custom method Using the custom method

Rich Client Platform

Not (just) an IDE

Support for virtually every programming language you will need

Cross platform

http://www.eclipse.org/

No! It’s not just for Java.

Demo

Search

Users expect good search functionality

Full-text indexes to the rescue

Index of terms that map to documents

Many databases include FT support now

What if these don’t meet our needs…

… then meet Lucene

http://gears.google.com/ http://lucene.apache.org/ http://drupal.org/

Fast -- Low memory usage

Extensible

Tools built on Lucene

Nutch - web crawler

Solr - they did the REST for you

ranked searching -- best results returned first

many powerful query types: phrase queries, wildcard queries, proximity queries, range queries

fielded searching (e.g., title, author, contents)

date-range searching

sorting by any field

multiple-index searching with merged results

allows simultaneous update and searching

Bite Sized Web Components

Small, specialized software building blocks allow you to leverage someone else’s work w/o costly dev or debugging.

They key is staying abreast of what’s out there and ways of integrating these systems together…

WebApp Toolbox

Storage

Amazon S3 http://aws.amazon.com/s3

$0.15 per GB/month storage

$0.10 per GB transfer IN

$0.18 per GB transfer OUT

Nirvanix http://www.nirvanix.com

$0.18 per GB/month storage

$0.18 per GB/month transfer IN/OUT

XDrive http://www.xdrive.com

$9.95 / month for 50GB (no transfer?)

Box.net http://enabled.box.net

$19.95 / month for 15GB (no transfer?)

Video

YouTube ( http://code.google.com/apis/youtube/overview.html )

Search Videos, Control ratings, comments, Upload videos, control and customize Flash player

Interface via the Google Data protocols (HTTP and XML), Java and PHP

Revver ( http://developer.revver.com/ )

blip.tv ( http://blip.tv/about/api/ )

Images

Flickr http:// www.flickr.com/services/api /

Uploads, Tag, Search, Browse photos and users / groups, Machine Tags provide the ability to extend metadata arbitrarily.

Support for REST, XML_RPC, SOAP, serialized PHP, and JSON

Picassa http:// code.google.com/apis/picasaweb/overview.html

Also Uses the GData API’s like YouTube

Supports population and retrieval of several different types of metadata, creation of groups / albums, etc…

Compute

Linux based Virtual Machines

Defaults include fedora core +/- apache +/- mysql

Custom configuration of instances are supported

Instances are persistent, but billable based on use

Amazon Elastic Compute Cloud

http://aws.amazon.com/EC2

$0.10 - $0.80 per instance/hour

$0.10 Trans IN, $0.80 Trans OUT

3Tera APPLogic

2-1024 CPU’s

2-2048 GB RAM

0.75 – 512 TB Disk

Web Hosting Setup

Single Server Instance per Month

$16.80 (small), $64.00 (large), $134.40 (xlarge)

$100 1TB per Month Transfer IN

$180 1TB per Month Transfer OUT

Total Cost = $296 - $414

Yearly Cost = $3500 - $5000

Server plus all power and bandwidth

High availability

Amazon Web Services

Great Example: Monster Muck Mashup - Mass Video Conversion Using AWS

Convert 500 videos to MPEG4 for iPod

Uses Amazon’s EC2, S3, and Simple Queue Web Service’s

Compute Time: 20min

Cost: $1.78

Authentication Services

OpenId – http://www.openid.net

Currently supported by AOL, LiveJournal, WordPress, Ma.gnolia, Basecamp and Yahoo!

Allows you to uniquely identify yourself to service providers.

OAuth – http://www.oauth.net

Library support for all major programming languages

Enables you to grant access to your data to a third party. (i.e. giving a slideshow mashup permission to access your Flickr photos w/o giving that site your login information on flickr)

Social Computing

Undoubtedly a force to consider when creating your web apps.

Future development scenarios are a bit uncertain.

Facebook /OpenSocial

Winner is currently unknown

Best bet today… stick with the eyeballs and be ready to change.

Facebook API

http://developers.facebook.com

3 components

Interface API

Query Language (FQL)

FBML (Facebook Markup Language)

Interface API

Interface API

REST based API covering ways to manage / retrieve information about friends / users / applications / profiles and messaging

Yech! This is a whole lot of bother for a list of friends…

friends.get(api_key, session_key, call_id, secret_key_md5, version, [format], [callback], [flid]);

Interface API

Fortunately the Developer’s Platform comes with helper libraries that encapsulate the current facebook API

Much Simpler!

friends.get([format], [callback], [flid]);

friends.get() Response <?xml version="1.0" encoding="UTF-8"?> <friends_get_response xmlns= http://api.facebook.com/1.0/ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://api.facebook.com/1.0/ http://api.facebook.com/1.0/facebook.xsd" list="true"> <uid>222333</uid> <uid>1240079</uid> </friends_get_response> [222333,1240079] XML JSON

Query Language

Query Language (FQL)

SQL-like query language that provides an alternate way to query the system.

API methods cover 90% of what you need here by there are 10% of things that I don’t think you can do w/o making use of the Query language

FQL has some advantages

Reduces the cost of parsing API’s more complicated XML structures

Potentially reduces the number of calls to the API (i.e. joins)

SELECT uid2 FROM friend WHERE uid1=loggedInUid friends.get() in FQL

FBML (Facebook Markup Lang)

An Evolved Set of HTML Tags plus Facebook specific features

All facebook content is passed through an FBML interpreter

Adds Security Controls and expanded interface features.

USE THIS WHENEVER POSSIBLE

It’s a great set of shortcuts and is maintained by someone else!

FBML Examples <fb:pronoun uid=123 useyou=‘true’ possessive=‘false’ reflexive=‘false’ objective=‘false’ usethey=‘true’capitalize=‘false’> Rob updated his profile. -possessive Charlie poked himself -reflexive You added Pat as a friend -useyou, usethey, capitalize and wrote on their wall <fb:friend-selector uid="123445" name="uid" idname="friend_sel" />

FBML Examples <fb:board xid="titans_board" canpost="true" candelete="false" canmark="false" cancreatetopic="true" numtopics="5" returnurl="http://apps.facebook.com/myapp/titans/"> <fb:title>Discuss the Titans</fb:title> </fb:board>

Facebook Application Model FB Data Store API Application Host

Widget Frameworks

Google Gadgets API

http://code.google.com/apis/gadgets/docs/overview.html

Support for Widget Integration with OpenSocial, Google Maps, Finance, Ads and Calendar

Run widgets in iGoogle or on Any Webpage

Widget Frameworks

Netvibes UWA

Run widgets across many platforms including

Netvibes, Mac Dashboard, iPhone, iGoogle, Opera, Vista, Windows Live and Yahoo! Widgets

XHTML for content and structure + css for style + javascript for interaction

Is AIR a Widget Killer?

Adobe AIR

Flash-based runtime for Desktop applications

Web-based deployment across platforms

Can run as background proc’s

Can access desktop files, clipboard, drag n’ drop

Awesome graphics, styling and effects

95% Market Penetration for Flash 9 (Dec 2007)

Content Management – Back to Legos

The CMS is the container for all these components.

A good CMS will allow you to quickly integrate these into a new site with a minimum of effort

CMS’s also enable easier internationalization

Some good CMS’s to start with

Joomla, Wordpress, Drupal, Plone

A CMS Tour

Joomla

steve (www.steve.museum)

IMA Digital (ima-digital.org)

WordPress

The IMA Blog (imamuseum.org/blog)

walkerart.org

Drupal

The IMA Dashboard (dashboard.imamuseum.org)

The IMA Website (imamuseum.org)

breaking the mode, nhmc

Show me the money!

The Romans Are Coming : Estimated cost $Y

Breaking the Mode : Estimated cost $X

Team Troubles $total = 0; $entrance_data = array (); for ($i = 0; $i < $entRawRS->RecordCount(); $i++) { $entRawRS->Move($i); $entAdjRS->Move($i); $cnt = getDayCount($entRawRS->fields[ 'dateData' ], $entRawRS->fields[ 'raw' ], $entAdjRS->fields[ 'adj' ], 300); $entrance_data[] = $cnt; $total += $cnt; } $chart->addRow( 'Entrances' , $entrance_data); div#header h1#site-name { background-image: url(../gfx/ima_logo.gif); background-repeat: no-repeat; display: block; height: 67px; width: 174px; float: left; top: 46px; position: absolute; left: 18px; z-index: 0; } <div class="copyleft"> <h3>Available Tours</h3> </div> <div class="copyright"> <div id="anchors"> <ul> <li>Spotlight a Collection </li> <li>Special Exhibition </li> <li>Lilly House Gardens of the IMA </li> </ul> </div> “ SELECT t.nid as task_nid FROM athena_project_task pt " . "INNER JOIN athena_task t ON t.nid = pt.task_nid " . "INNER JOIN node n ON n.nid = t.nid " . "WHERE pt.project_nid = %d AND t.state = %d " . "ORDER BY n.changed DESC, n.title" var i = 0; var found_date = false; $('select').each(function(){ i ++; var words = this.id.split('-'); var last_word = words.pop(); switch(last_word) { case 'year' : $(this).addClass('split-date'); $($(this).parent()[0]).addClass('cal-form-item'); this.id = words.join('-'); found_date = true; break;

Divide and Conquer

Allow your team members to focus on their area of expertise

Minimize potential conflicts when editing files

Make it easier to swap out content or themes

div#header h1#site-name { background-image: url(/gfx/ima_logo.gif); background-repeat: no-repeat; “ SELECT t.nid as task_nid FROM athena_project_task pt " . "INNER JOIN …

MVC – An architectural basis

View (user interface)

Renders content

Requests content updates

Sends user input to controller

Model (data management)

Manages persistent data

Responds to data queries

Notifies views of changes

Controller (interaction logic)

Defines behavior

Selects views

Updates the model based on user action

user actions view selection model updates data requests change notification

MVC – Applied to the web

View (user interface)

Page components (HTML, XML)

Page style & layout (CSS, XSLT)

Effects (Javascript)

Model (content management)

Databases (MySQL)

Back-end logic (PHP)

Controller (interaction logic)

State & API handling (PHP, Javascript)

user actions, navigation page & theme selection content updates and requests content requests (AJAX) Web Designer Web Programmer Web Programmer Content Creators Content

How do we make use of MVC?

We use content management systems

Let it take care of the heavy lifting (model code)

Content creators will be happy

We use templates

Templates provide a boundary between controller and view

Think of templates as an interface, and design accordingly

Web Designers will be happy

We have a process: Design, implement, evaluate, iterate

First, plan out what you want to achieve in phase one

Divide the work based on skill sets

Implement and test as you go

Come together again as a team and evaluate your progress

Plan your next phase…

Subversion (SVN)

An improvement on the concurrent versions system (CVS)

Allows multiple developers to work simultaneously without overwriting each other’s code

Basic operations:

checkout

add & remove

update

commit

Code developed or updated on a beta installation can

be quickly reflected in a deployment

It works for designers too!

Clients: Eclipse plugin, Tortoise SVN

Bringing in non-technical members

In the planning phase:

Non-techies have great ideas too

They can also provide great insight for interface design

In the evaluation phase:

They will do things you hadn’t thought of

And they won’t be afraid to let you know…

Hey Ed, What if you leave a project you really wish you would have deleted? Is there any way to return to abandoned projects and delete them for the sake of tidying up? Or is it easier to create an automated process to delete projects with no files and active members? It would be interesting to put in a kind of drop down list by department where we could then just check off members, rather than typing them in. I don't know if this is possible, but for all that I know it's all magic and anything is possible.

Drupal Boot Camp Drop and give me 20!

Let’s get started…

Make sure you have the required components

Apache or IIS web server

PHP

MySQL

Download Drupal from http://drupal.org

We’re using 5.7 to demonstrate the CCK and Views modules

Unzip/extract Drupal to disk

Configure your web server to serve the directory

Set up the database

That was quick!

Create some content

Pick a new theme

Install some modules

Craft a custom content type

Organize content with views

Peek at user permissions

Look at where to put custom code

Listen up! Here’s the game plan…

Create some content

Choose a new theme

Content type creation with CCK

CCK is the Content Construction Kit

Download CCK from http://drupal.org/project/cck

Extract the contents to sites/all/modules/cck

Create a new Content Type

Add a custom field

Configure the field widget

Create your content

Take a peek at the result

Organizing content with Views

Download the Views module from http://drupal.org/project/views

Extract to sites/all/modules/views

Enable Views in Site Building

Add a view

Configure the view

A quick look at menu items

Add a link to the new view Change your front page in site information

Behold, the new front page…

and the Works of Art page

For the ambitious

Custom code should be added at sites/default

Create a modules directory to contain custom modules

Create a themes directory to contain custom themes

Eclipse users:

Create a project at sites/default

For team development:

Create an SVN repository from sites/default

Install and configure each development site

Perform a check out from the project repository

on each workstation

Note: Drupal 6.x has new theme customization features

Recommended modules

CCK & Views, as shown in previous slides (not yet available for Drupal 6.x)

ldapauth – Authenticate against existing LDAP systems

Image – Upload, resize, and render images

Image Cache – Easily define derivative image sizes

Devel – Extra tools for developers

jQuery Update – Utility to update Drupal’s jQuery library

reCAPTCHA – Allow visitors to add content without requiring a login

Client Side Development The Customer is Always Right

Client Interfaces

The web is now a platform for hosting all kinds of online applications

So what are the basics of this application platform?

Let’s talk about the DOM…

What about the DOM?

What’s a DOM?

Document Object Model

This is a platform and language neutral model of a web page

The DOM model applies to HTML and XML documents

What’s in a DOM?

Javascript

Scripting language that web browsers can execute when user loads a page.

For adding interactivity to web pages both Javascript and Actionscript are used frequently.

Mostly by modifying and manipulating the DOM

They both also support OO techniques and models

Slick Effects and Interactive Features are a hallmark of what people think of web 2.0

Most of these effects can be accomplished using Javascript

Javascript Toolkits

Making Javascript a little more OO

Why you need one…

Avoid (some of) the cross browser problems

Rapid development

Javascript Toolkits

A variety of toolkits have become available that make manipulating the DOM in Javascript easy

Prototype: ( http:// www.prototypejs.org / )

Scriptaculous: ( http:// script.aculo.us / )

Lightbox: ( http://www.huddletogether.com/projects/lightbox/ )

Greybox, Thickbox

jQuery (http:// jquery.com )

Moo.tools (http:// mootools.net /)

Moo.fx ( http://moofx.mad4milk.net/ )

Yahoo User Interface Library

http://developer.yahoo.com/yui/

Google Web Toolkit:

http:// code.google.com/webtoolkit /

jQuery Example

Interface Components

Just like server side web components developers can also leverage reusable components for graphics design and layout.

Back to the Toolbox

Thickbox - http://jquery.com/demo/thickbox/

Back to the Toolbox

reCAPTCHA - http://recaptcha.net/

Back to the Toolbox

Image Carousels

ImageFlow 0.9 http://www.prototype-ui.com

Back to the Toolbox

Rating Modules

In Support of CSS Boilerplates

Cross browser tweaking of CSS layouts can take a lot of effort.

Why not treat CSS layout like another software component of web design.

Where are the reusable components?

http://www.intensivstation.ch http://www.thenoodleincident.com/

Standardize Your CSS Usage

Structured naming of <div> tags or <class>’s enables drop in use of server side content generation and JavaScript effects.

Seek CSS Enlightenment Grasshopper

OSWD

http:// www.oswd.org

CSS Zen Garden

http:// www.csszengarden.com

http:// www.mezzoblue.com/zengarden/alldesigns /

XML in the Real World You mean people actually use this stuff?

G.O. – Global Origins

XML Driven Content

Plugin to NASA’s WorldWind

Google Maps Mashup

Art Metadata, Imagery, Audio, App Settings, UI Components, etc …

G.O. – Global Origins One Ring XML Data Description to Rule Them All

G.O. – Global Origins

Config File is a Schema Instance

XmlSerializer artobjSer = new XmlSerializer(typeof( GlobalExplorer )); goData = (GlobalExplorer)artobjSer.Deserialize(File.OpenRead(this.xmlFile));

Useful Eclipse Plugins

SVN Integration

Subversive , Subclipse

PHP

PDT , PhpEclipse

Hibernate Tools

WTP – WST

Ruby – RadRails

Python - PyDev

Questions?

Rob Stein

[email_address]

Ed Bachta

[email_address]

Charles Moad

[email_address]

Slideshare of this talk

http://www.slideshare.com/.....

Advanced Web Development

0 comments

Notes on slide 1

15 Favorites & 1 Group