BP203: A Look Under the Hood
at a World-Class IBM Lotus Domino Web Application
Scott Good
President
Teamwork Solutions, Inc.
Henry Newberry
Director of Technical Services
Teamwork Solutions, Inc.
®

Scott Good – Bio
President, Teamwork Solutions, one of Lotus’ earliest BPs
Started with Notes 2.1 (1992)
Extensive experience building workflow applications for both Notes
and the web
Have published over 50 articles on Notes & Domino application
development (LotusScript, JavaScript, CSS, AJAX, JSON, etc)
Contributing Editor, Lotus Advisor magazine
Regular speaker at Lotusphere and Advisor conferences

Henry Newberry – Bio
Director of Technical Services at Teamwork Solutions, Inc.
Formerly President of Newbs Consulting, Inc. (2000-2005)
Cofounder of Synergistics, Inc. (1993-2000)
Notes Pioneer at CBIS (1989-1993)
Lotus Advisor Contributor, Speaker, and Editorial Advisory Panel
since 1996
Speaker at IBM Lotus events since 2001

We were …
Asked by Kevin Cavanaugh and Bob Balaban to re-build the Lotus
on-line forums to take advantage of Web 2.0 capabilities
And, to do it using Domino
The result is a really interesting Domino web application which
we’re going to share with you today

Agenda
Introduction to the problem
System setup and definition
Handling of views and navigation
Handling individual threads
Creating new threads and responses
Dojo Editor
Searching
Within one forum
Across multiple forums
Most Popular Topics

Introduction to the ------------------ Opportunity
problem
Existing forums were Domino-based but…
Old
Slow
Ugly
Every click required a full page refresh
View twisties
Each response in a thread
Etc
Goal was to rebuild the forums to take advantage of AJAX, JSON,
DHTML and CSS to make an attractive, fast, easy-to-use site
And, to NOT lose all information currently in the old forums
And, wherever possible, to take advantage of Domino’s strengths

Before and After

System Setup Overview
A hub-and-spoke arrangement of a single Central database and
multiple external Forums (databases)
IBM
Presentations
Forum
Support
Forums
“Home”
IBM IBM
Documents Spreadsheets
Forum Forum

System Setup: Forum Databases
One central database per “suite” (e.g., Lotus Symphony)
One database per “Forum”
IBM Lotus Spreadsheets Forum
IBM Lotus Documents Forum
IBM Lotus Presentations Forum
In each Forum, any number of “sections”
Feature Requests
Feedback (general discussion)
Hints and Tricks
Issues and Troubleshooting
Sections do not need to match from Forum to Forum
Each section contains the actual documents and responses

System Setup: Central Database
Central database
Contains forum and area configuration documents
Contains the main Home page with links to all forums and forum sections
Searches done from here reach into all forum databases
Collects “Most Popular Topic” information from all forums
Feature
Forums Requests
Home Section
Page Forum
Configuration
Feedback
Support Forums Section
Most
Central
Popular
Database
Topics
Hints &
Tips
Section
Multi-
Forum
Searching Issues &
Troubleshooting
Section

System Setup: Forums
IBM
Presentations From the Home page, links to Forums
Forum
Each Forum reads configuration
information in Central Db to get Sections
Feature
Requests
Central
Database
Feedback
Section IBM
Spreadsheets
Forum
Hints
IBM Spreadsheets
Section
Documents Forum
Forum Configuration
Issues
Section

System Setup: Creating a New Forum
Create new Forum database
from template
Create On-Line Forum
Configuration form in Central
database
Automatically creates a keyword
doc in the Forum database to let
Forum find the central database
Create one or more Forum
Section documents to define
that Forum’s sections
Close and reopen the browser
(to clear the cache)
Done

Central Database Home Page
Our plan…
Get something on-screen immediately (core page, header)
Get most important content next (Forum links)
Get less-important content last (Most Popular Topics)
Give the user something to look at ASAP

AJAX, then AJAX, then more AJAX
1 AJAX lookup in the onLoad to get all the Forums and sections
?ReadViewEntries&OutputFormat=JSON
Uses JavaScript to build the central page area’s HTML on the fly
2 Get the Most Popular Topics
Pulls top 5 topics from each Forum
(sequentially, with AJAX)
As a custom JSON
format…more about this later
Collates, sorts, builds HTML of top
Topics and puts them on the page
Results are cached during your
session

Home Page Links to Each Forum…
Primary link to the database (“IBM Spreadsheets”)
Links to each individual Section (“Feature Requests,” etc)
Information for each Section on…
Last post date/time
Number of main topics
Total number of posts
Number of views

Forum Views
AJAX content retrieval and pagination
Response hierarchy creation
UI Navigation approach (tracking via cookies, breadcrumbs)
User preferences options

Forum Views
Two options
Main Forum view
Forum Section
Both are the same view, same $$ViewTemplate
View: AllDateThreadedWeb
Section “views” don’t display the list of other sections
Main Forum view DOES have links to all sections
What you see is controlled by the URL, Cookies

When the View Opens…
Core HTML is loaded
And JavaScript
And CSS
Main view documents (top-level documents) are retrieved
Then Forum Section information (if needed) from Central Db
Cached AJAX query…very fast
THEN, if needed, documents for open response hierarchies
Each component is built and displayed as it is available

How the Views Work
Initial URL determines what will be shown
AllDateThreadedWeb?OpenView&RestrictToCategory=All
AllDateThreadedWeb?OpenView&RestrictToCategory=Feature%20Requests
Initial URL does not return any view information
$$ViewTemplateDefault does not have a $$ViewBody field
All view information is built on the fly from AJAX query returns and placed into
the existing HTML
$$ViewTemplateDefault DOES have…
HTML used for the rest of the page
Many computed JavaScript variables
Notes fields with lookups
JavaScript libraries
CSS libraries

AJAX 1…Main Documents
onLoad event of $$ViewTemplateDefault calls an initView() function
and starts the first of a series of AJAX calls
Gets the main View documents (using AJAX)
AllDateThreadedWeb?ReadViewEntries&OutputFormat=JSON&count=10
&RestrictToCategory=All&CollapseView&nocache=1196698255351
Pulls main docs data from a multi-column view using Domino’s
standard JSON format
Returned string needs converted to become “live” JSON
eval(\"forumLinksJSON = \" + forumLinksAjax.responseText);
JSON data is built into a ViewObj class
Within class, builds HTML for the “view,” places it on-screen
document.getElementById(\"ViewOutput\").innerHTML = …;

How to Know What to Show
Since all data is retrieved with AJAX, URL never changes
Without tracking current user position, returning to the view (e.g., “Back”
button) would always take you to the first page…not good
Have to keep track of where you are and rebuild accordingly
Current view location is managed with cookies
View position/lines per view (per Forum) 21~10
Expanded row numbers (per Forum) 21~21.1~21.1.1~22
Whenever view is opened, this is read and used to control what
part of the view is actually returned

AJAX 2…Forum Sections
Forum Section list is retrieved from the Central Database
Retrieve Forum Section names (a cached lookup)
/MainDb/ForumAreaLookups?ReadViewEntries&OutputFormat=json&count=200
Pulls necessary information from multi-column view, again using
Domino’s standard JSON format
If the view “category” is not “All,” this is ignored
Because it’s cached…it’s incredibly fast
Otherwise, JSON is converted using eval(), processed with
JavaScript, and added to the page

AJAX 3…Response Hierarchy in the View
Only top-level documents load when the view is first shown
Responses are loaded when any “twistie” is clicked
ALL descendants of a given topic are loaded when the top-level twistie is
clicked, but only displayed one level at a time
No descendants are loaded for any hierarchy not clicked
A delayed call in the onLoad event resets hierarchy when returning
setTimeout(\"reOpenViewToPriorSettings()\", 100);
Reads cookie to know which response sets to re-load
Reads cookie to know which elements of the hierarchy should be expanded
Gives users the expected experience when hitting “back” or otherwise returning
to the view from somewhere else

View Load Review
Initial HTML, including breadcrumbs
Main Documents
Forum Section links (if needed)
Response hierarchies as needed
Reset expanded response trees

Opening a Thread
Clicking on any document or any of its responses opens the top-
level document with all responses rendered into it
HTML for the main document loads with the form, all responses are
retrieved with AJAX
Response hierarchy is displayed with indentation
Every document and response has a “respond” link allowing users
to respond to any point in the thread (as you would expect)
Lateral navigation to next and previous documents in the view

Creating a New Document
Click “Post a Topic” link from virtually anywhere in the site to create
a new Topic (main document)
Click “respond” in any of the document hierarchies to respond to a
part of an existing thread
Both the document and response are almost identical and kept as
simple as possible
User name and e-mail address are stored in a cookie (user are not
required to be authenticated)

Saving a New Post
Returns you to where you were in the view
With the right things expanded
Traversing laterally through documents (using Next/Previous)
View location keeps track of where the document you’re in (now) is located
As you move to documents in the next segment of the view, returning to the
view puts you in the new location

Document “View” Counter
Each time a document is opened, a Profile document (used by all
documents) is incremented

Guest Preferences
Guest Preferences allow users to set values for
Number of rows displayed per page
Time Zone (which is ignored)
After changing settings, you’re returned where you came from with
the new settings in effect
Adjusts view paging to put you in the “same” place
From 7th page on a 10/page setting to the 3rd page on a 25/page setting

Searching
There are two types of searching
Just within the current Forum (from anywhere in a Forum)
Across all Forums (only available from the main Home page)
The two search types use two different search techniques
HTML-formatted search view (current Forum searching)
Custom JSON-formatted search views (multi-Forum searching)

Single-Database Searching
Search query is built with JavaScript into a URL
searchview?searchview&query=replication%20and%20conflict
SearchView is a view formatted in HTML to display nicely in the browser
Uses $$SearchTemplateDefault form to display results
$$ViewBody field controls placement of “view,” which is really just search results from the
view
Advanced Search option (on the $$SearchTemplate) expands the
parameters to further refine the search
searchview?searchview&query=replication%20and%20conflict&SearchOrder=1&SearchWv=
TRUE&SearchFuzzy=TRUE&SearchMax=250
These are all standard Domino capabilities/URL parameters
Again, constructed with JavaScript based on selections in the available fields
Requires database to be Full-Text Indexed

Cross-Database Searching
A little more complicated than single-database searching
Surprisingly—shockingly—fast
Is initiated from the Home page in the Central database
When started, makes an AJAX search query to each of the
associated Forums, one at a time
As one set of results is returned, the next query is sent
Continues until all Forums have been touched
Queries are made to custom JSON views
NOT Domino’s JSON format
Query results are organized using a special $$ViewTemplate for the
view

Custom JSON Search View
Used custom JSON to make it easier to get only the information we wanted
back
Enough to build a results page with links to documents in the various Forums
Our format for each line in the view…
{
sprid: \"WEBB76TFX3\",
unid: \"00EA8A8C84B104EC0525834F005AFC36\",
dbtitle: \"IBM Spreadsheets\",
dbpath: \"IBM\\SpreadsheetForum.nsf\",
subject: \"Full-text search not working?\",
date: \"09/07/2007\",
author: \"John Doe\",
numviews: \"12\",
dbform: \"Feedback\"
},
NOTE the closing comma

Custom Search ViewTemplate
$$ViewTemplate for MultiDbSearchView
JSON object contained in a <div> for JavaScript parsing purposes
More about this in a moment
Blank set of values at the end because each live of the view ends
with a comma

In the Home Page…
Starting a search puts query on the URL and calls SearchResults
form
window.location = \"SearchResults?Open&query=” +
escape(document.forms[0].SearchField.value);

SearchResults form…
Is very much like $$ViewTemplateDefault in layout, format
Includes fields for Advanced Search
Has onLoad event which starts the cascade of AJAX searches
doSearch() builds search URL and calls makeSearch() which makes all AJAX calls
function makeSearch(){
queryUrl = \"/\" + searchDbPaths[curDb] +
\"/multidbsearchview?searchview&searchmax=0\" + queryVal;
curDb++;
createAJAXRequest(queryUrl, \"handleMultiDbSearch\");
}
handleMultiDbSearch() continues to call makeSearch() until all
Forums have been searched

Parsing AJAX Return Values for JSON
$$ViewTemplate for MultiDbSearchView returns a big bunch of
HTML around the part you actually want

Parsing AJAX Return Values for JSON
$$ViewTemplate for MultiDbSearchView returns a big bunch of
HTML around the part you actually want
Can’t change the form to display as HTML or as “text/plain”
because neither will work with a form being “edited”
Have to live with the mess and parse it out
Hence, the <div> around the JSON object…a handle to grab with JavaScript

Getting the JSON Out of that Mess
It’s as simple as finding the locations of the <div> and </div> tags
in the string that is the HTML
var searchHTML = ajaxReq.responseText;
var divStart = searchHTML.indexOf(\"<div>\") + 5;
var divEnd = searchHTML.indexOf(\"</div>\");
var viewStr = searchHTML.substring(divStart, divEnd);
eval(\"var viewObj = \" + viewStr);
Of course, you have to know there are no <div> tags in the text

Why Not Use an Agent for Searching?
One reason: Speed
Using the Domino-standard searching against a Full-Text Indexed
database is amazingly fast
Agents give you flexibility you can’t really get out of a view
That flexibility comes with a cost…it’s slower
We didn’t need the flexibility

Finishing the Search
Once JSON is back from all Forums, page is built
JavaScript iterates through all JSON objects building HTML
Search results are segregated by Forum

Most Popular Topics
Similar to multi-Forum searching
On the initial Home page load, all Forums are
queried (via AJAX) for the Most Popular Topics
Based on the number of views
Query is made to a custom JSON view sorted by
view count
Only five documents are returned from each database
Results from all Forums are combined and sorted
Top five (plus ties) are displayed
Results are cached during the user’s session

Cascading Style Sheets
Our design is based on that used for interim Jive-based site
Most of our application was done before we got access to actual
CSS code
Built it from screen shots
Then re-built it to match, using CSS from IBM (ON IBM site) plus localized
adjustments needed for our system
Then IBM’s CSS changed…a little
Now we store ALL our own CSS in the databases

Lessons Learned
Don’t count on others to never change their CSS
JavaScript compression options can make an enormous difference
in file sizes (and corresponding page-load speeds)
Sequentially searching across multiple databases is amazingly fast if
you leverage the Domino search mechanisms
FireBug ROCKS!

Thank you!
Scott Good: sgood@teamsol.com
614-457-7100 x200
http://www.scottgood.com (blog)
Teamwork Solutions Henry Newberry: newbs@teamsol.com
614-457-7100 x206
http://www.henrynewberrry.com (blog)
http://www.teamsol.com
PLEASE FILL OUT YOUR EVALUATIONS