Presentation: Engage.ug March 2015 by Peter Presnell
Many organizations are now investing in the development of web interfaces for their existing (and new) Notes/Domino applications. Now is the perfect time to start paying down on the technical debt we have accumulated to ensure that we don’t pass on an unsustainable software development deficit to those that will follow. With this presentation we will challenge the notion that trickle down development based on XPages is the next logical step in that evolution. While XPages still has a place in the world of modern Domino development we will explore new approaches and alternative technologies that have the potential to deliver a brighter prosperous.
60. ModernizationA continuous journey with many twists and turns. We can never be quite sure what lies
around the next corner. But then… it is not knowing that creates the excitement.
69. 1990s: The Rise Of The
Citizen Developer
Citizen Developer: Noun
A user operating outside the scope of IT
and its governance who creates new
business applications for consumption
by others by scratch or by composition
Gartner
In this presentation we are going to look at the future of Web application development for Notes applications and challenge the belief that Xpages is the next logical step for applications originally developed for the Notes client.
My goal is to challenge conventional thinking in the the belief that opportunities exist to
Save time
Save money
And reduce the risk assocciated with modernizing Notes applications at the same time as possible
Helping to define the career choices that lie ahead for Notes developers.
Today’s presentation is divided into four sections.
Defining our goals by looking at what is modern in application development.
Understanding how we got to where we are today and the technical that has been accumulated along the way
Look at why XPages is not a good fit for meeting our future needs
And finally look at the architecture by which we expect modern Web applications to be developed.
Before we start… While this is a session targeted at Developers (amongst others) I would like to point out this session contains no code. It is largely a discussion about strategic direction and software architecture.
To me modernization is the process of narrowing the gap between what our applications currently deliver and what our users are expecting from those applications based upon what is now possible.
It is 2015 and it is clear that the sun is setting on the Notes client as an application development framework. If you are like the typical Notes shop you have many Notes applications that look something like
This
Or perhaps this
Or this
Maybe there are a few neglected applications looking like this…
Or even this… It is also quite likely that the users of those applications are starting
To look like this.
Because what they are expecting is desktop applications that look like this
If they have a tablet they expect applications that look like this
And on a smartphone applications such as this.
And soon they will be looking for applications on their watches!
So what does IBM have to say about all of this?
Well if we start with a typical Notes discussion database like this. There are some at IBM who are suggesting the way forward is to do this…
Place a firefox (or Internet Explorer) logo in the top left corner of the screen. Running the Notes client inside the web browser using what is (currently) known as the Notes Browser Plug-in.
Let me be very clear about this….. This is not MODERN. This is serving up the same tire old Notes applications. They still look the same… They still behave the same. And your users are still going to
React the same way!
SO what is modern. I would like to start with what I think has recently merged as one of the most compelling requirements… Design. Over the past couple of years the importance of design has emerged as a key need behind every great web applications.
This is in no small part due to the advent of mobile devices and the emergence of the application store in which users have become accustomed to downloading great looking applications for as little as 99cents – or even free. Very few of these applications come with instruction manuals or on-line training videos. They are expected to be intuitive, easy to use and look great on their latest sexy smartphone. This expectation is now starting to rub off as an expectation for business software, especially as the price tage for most business applications is well over 99 cents per user!
And on a smartphone applications such as this.
IBM is starting to get it. They have started to invest heavily in Design through the IBM Design Center. This is under the leadership of Phil Gilbert, perhaps the most inspirational IBM executive I have heard speak for a long time. It is clear Phil gets it and there is hope that IBM will understand the importance of Design.IBM aredefeinitely making a large investment in tgis area and are in the process of establishing the IBM Design Language.
One of the first products to have benefited from this new Design focus at IBM is Verse, which I am sure many of you have now seen.
And Connections next will follow soon.
Google are another major player that is starting to make a significant investment in design. Google has its material design philosphy that is being rolled out in many of the latest versions of its software. Importantly they also have toolkits that can be used to deliver applications that follow the Google Design standards.
Adding a web interface to Notes client applications is no longer enough. Over the past few years we have seen significant changes in the sizes, resolutions, and aspect ratios of monitors. The display size being used to access applications has been simultaneously getting larger and smaller.
Responsive web design has quickly become the expected standard for web design. Responsive web combines the concept of fluid grids with adaptive layouts allowing the way a user interacts with the application to automatically adjust to current size of the page in which it is displayed.
Twitter Bootstrap has quickly emerged as the de-facto standard for responsive Web and is the recommended approach to take when adding a web interface to Notes client applications.
IBM’s OneUI is constantly playing catch-up and perhaps best suited to Notes developers still new to XPages and Web development
For some applications responsive web designs may be adequate for delivery onto mobile devices, especially tablets.
In many cases there is a need to design applications for a mobile client that uses the same user experiences found in native apps. This can be achieved using HTML5 and CC3 or by developing either a native client or hybrid application.
For HTML-based applications the best mobile controls at this time are Kendo UI and jQuery mobile. TeamStudio’s Unplugged is also an option, especially when there is a need for offline access to data.
Dōjō mobile and the IBM XPages Mobile Controls are a good starting point for Notes client developers to learn about developing applications for mobile devices but it is difficult to see IBM being able to sustain a level of investment that will allow either dojo or XPages mobile controls being relevant in a few years time.
For mobile client development Mobile First is perhaps an option for large organizations looking to make huge investments into building an enterprise-grade mobile infrastructure. For smaller projects such as those likely to be tackled by Notes developers solutions such as Titanium or TeamStudio Unplugged.
When developing the mobile user experience it is suggested to consider the different ways in which each device is used. Tim Tripcony perhaps summarized this best… A desktop/laptop UI should be designed for use with a mouse, a tablet UI for use with the hand, and the phone UI for use with two thumbs.
When building a mobile client for an existing application don’t assume it is necessary to reproduce the existing functionality of the application on the mobile device. Consider different usage patterns for phones from tablets and laptops.
Wearables are likely to be the next big thing to impact application design. The pending release of the Apple Watch will most likely define the initial UX for this category of device.
Google’s recent withdrawal of Google Glass should be viewed as a delay rather than a cancellation of this type of wearable device.
Mobile phones have had an interesting history. First they got smaller as we found new ways to cram greater capabilities into smaller components. Then with the advent of the smartphone we saw them get larger again in order to deliver a larger display. It is not clear whether smart watches and other wearable devices will follow similar trends. With wearable devices we may see them acting as a first port of notification that is tied to
With wearables we will most likely be looking at a completely new design for application. One in which multiple devices are used to complete a transaction. An application notification may be displayed on a watch (or glasses) for an initial triage from the user’s smartphone before it is completed using either a tablet or laptop. In the past the concepts of transactions were based around the completion of a transaction from a single location following communication via email.
If you are just starting to address the needs of providing smartphone information for Notes applications originally designed for Desktop computers now might be a good time to consider how these applications can be engineered to display information on even smaller screens.
Each time a name appears in the data being displayed to a user we have a potential opportunity to connect that name with a wide range of options to provide analytics about that person and/or provide a range of options that facilitate communication with that person. We may want to see who that person is, where they fir inside their corporate hierarchy, how I am connected to that person and the options available to me to reach out and connect with that person. If it is somebody I already know I may want to see recent communication I have had with that person or know when we are next scheduled to attend the same meeting or event.
Social media applications such as Facebook, LinkedIn, and Skype all demonstrate a wide range of these capabilities. The latest wave of PIM such as IBM’s verse and Tempo also demonstrate how close integration between a contact information and names stored in other applications can deliver powerful business solutions.
We are fortunate that in Notes applications names are usually clearly identified via the use of Names fields. The Notes platform also has support for both a corporate directory and personal contact databases. These form a great starting point for delivering integrated contact details to existing applications. Contact details can also be supplemented by the content of other Notes databases containing contact information for customers or suppliers.
A great way to start is to consolidate the data sources into a single API that can be accessed from Applications. The development of a single “Contact” control can then be used to display contact details and expose a wide range of options for interacting with these contacts.
This is not unlike the way Notes client applications were able to provide Sametime integration through the addition of new properties for existing Names fields. What was missing from Notes at the time was a simple way to control the default behavior for all Names fields in an application.
The fastest way to implement this type of integration is to separate the presentation layer from the application itself. A data schema for fields on a form can identify dates and control which (optional) behaviors are provided for each name field.
Social software products such as Facebook and Twitter made activity streams and timelines a popular way of consuming information that has some aspect of time associated with it. Today we are seeing timelines and activity streams being added to wider range of software. Even the accounting package I use now supports a timeline as a way to show me the activities I have recently undertaken or need to attend to.
Again Notes applications have a great basis for exploiting this modern capability. Most Notes document track the date they were created and each time they were subsequently modified. The names of the people who most recently modified a document are also stored. It is also possible to track who has accessed a specific document, but this is a feature that must be enabled for a specific application. All this meta-data that is being collected can be used to provide timelines for the content of individual applications.
Social software products such as Facebook and Twitter made activity streams and timelines a popular way of consuming information that has some aspect of time associated with it. Today we are seeing timelines and activity streams being added to wider range of software. Even the accounting package I use now supports a timeline as a way to show me the activities I have recently undertaken or need to attend to.
Again Notes applications have a great basis for exploiting this modern capability. Most Notes document track the date they were created and each time they were subsequently modified. The names of the people who most recently modified a document are also stored. It is also possible to track who has accessed a specific document, but this is a feature that must be enabled for a specific application. All this meta-data that is being collected can be used to provide timelines for the content of individual applications.
The concepts of a Services Oriented Architecture have been around for a long time now. In recent years REST services has overtaken SOAP as the preferred approach to implementing SOA.
REST has the potential to provide access to data in a way that makes it transparent if the underlying data is held in a AQL database, a Notes database or any of the other emerging non-SQL database formats.
REST can also play an important part in encapsulating business logic important in developing the application layer. If business logic is accessed via REST it become largely irrelevant if the underlying programming language is C#, Java, LotusScript of @Formula.
Domino provides a wide range of options for creating REST APIs for Notes databases making it relatively easy to access the data inside Notes applications via REST. This forms the basis for accessing Ntes applications from IBM Worklight and IBM BlueMix.
The biggest challenge for Notes applications is reconstructing the application layer in a way that it can be invoked via REST services. Code written in a combination of LotusScript and @Formula is embedded into actions, buttons, and events on Forms and Views with only a low level of adoption of object oriented programming styles.
There are now quick outs here. It will take a lengthy period of time to re-engineer many Notes applications to take full advantage of REST. Without it there are few options for taking the existing logic and making it work outside the Notes client. A start can be made by moving the code into agents.
In some organizations (but certainly not all) there is still a stigma associated with installing software on a Domino server that is not fully supported by IBM. Some Domino Administrators have been reliant on IBM to deliver the entire code base for Notes client development for so long that it is sometimes difficult to adjust to the widespread use of OpenSource software in the modern Web development arena.
The OpenNTF Domino API (ODA) is just one example of essential software now being made available using the OpenSource model. Others include all Apache projects, Bootstrap, Java, jQuery, Dohjo, Angular.js, Eclipse, Android.
The concepts of a Services Oriented Architecture have been around for a long time now. In recent years REST services has overtaken SOAP as the preferred approach to implementing SOA.
REST has the potential to provide access to data in a way that makes it transparent if the underlying data is held in a AQL database, a Notes database or any of the other emerging non-SQL database formats.
REST can also play an important part in encapsulating business logic important in developing the application layer. If business logic is accessed via REST it become largely irrelevant if the underlying programming language is C#, Java, LotusScript of @Formula.
Domino provides a wide range of options for creating REST APIs for Notes databases making it relatively easy to access the data inside Notes applications via REST. This forms the basis for accessing Ntes applications from IBM Worklight and IBM BlueMix.
The biggest challenge for Notes applications is reconstructing the application layer in a way that it can be invoked via REST services. Code written in a combination of LotusScript and @Formula is embedded into actions, buttons, and events on Forms and Views with only a low level of adoption of object oriented programming styles.
There are now quick outs here. It will take a lengthy period of time to re-engineer many Notes applications to take full advantage of REST. Without it there are few options for taking the existing logic and making it work outside the Notes client. A start can be made by moving the code into agents.
React the same way!
$20,000 per application.
After alll…. There are 10 million Notes applications out there.
After alll…. There are 10 million Notes applications out there.
And 50 billion lines of proprietary code. It is difficult to see all of this going away any time soon.
With the average cost to rewrite code of 3 euros per line
This represents a technical debt of 150 billion
The avereage company’s share in the technical debt if 4 million
After alll…. There are 10 million Notes applications out there.
Most CIOS have become stuck doing nothing. As the sands of time pile up the situation only gest worse. Unless something is done to start paying off the technical debt it overcome and drown many….
And all this time I poor user still has that pained look on their face every time somebody mentions NOTES.
To address the issue of our technical debt we have three choices.
The Do Nothing (or COBOL) approach is favored by those companies where the customer expectations have not yet exceeded that delivered by the Notes client. It is also adopted where companies are not prepared to invest in modernizing their applications or where the remaining developers supporting these applications do not feel confident in their ability to master the new Web development languages. This approach includes the adoption of the Notes Browser plug-in.
The next option is that of using XPages or other similar server-based approaches such as Connections or SharePoint.
So Why not XPages? XPages does have a place and in some situations it may be an option. It is certainly better than doing nothing at all. But I would argue quite strongly it is not a strategic choice upon which to build a new future for existing Notes applications. And the reasons for this conclusion are as follows….
There is a small team of very dedicated people at IBM who are trying their best to deliver a future for XPages. But they are hamstrung. IBM are simply not investing at a level that will ensure XPages can establish a foothold in the application development market outside of a declining Notes population. And even if they did it is questionable if the architecture itself has a strong future moving forward. At best Xpages is a stop-gap offering from IBM targeted at existing customers only in the hope that they can find a way to move them over to another IBM product such as Websphere or Connections.
Notes recently turned 25. When we look at the roadmap over those 25 years
We see s steady pattern of major releases every 2-3 years. But that stopped in 2008 when we had the last major release of Notes – Notes 8.5. We should ignore 9.0 as we all know that this was going to be a point release 8.5.4 but a marketing decision was made to change the numbering to 9.0 to better support the marketing message around Social. Of course I have missed 2014, which we can see had absolutely no releases, not even a point release. And I am pretty sure every time we saw a roadmap from IBM prior to 2014 they we not showing an empty calendar for 2014!
In 2015 IBM continues to have a road-map tat includes Notes/Domino. You will hear strong statements about betting on the Domino SERVER as being part of the future and how the Notes client will be SUPPORTED moving forward but even IBM themselves are now moving away from the Notes client.
IBM is trying to get as many of its employees as possible across to using Verse by the end of 2015.
And we have asked them about tere plans for Notes applications. The response we have received Is that IBM will be adopting the Browser plug-in as part of its plans to ieliminate the use of the Notes client.
So we know there is at least one person not impressed with that decision. And maybe many more??? Many more companies can be expected to follow IBM’s lead, resulting in a signficant drop in revenue for the Notes platform
So by 2020 the road map for the product is likely to look more like this
And Lotus Notes will join a long list of other products that were in the original Lotus brand…
Which brings me to the issue of commitment.
As recently as Lotusphere 2010 IBM had a message to community about providing Continuity
Continuity that includes the discontinuance of the Lotus SmarSuite.
Domino.Doc… which was folded in QuikR
Only for QuickR itself to be disCONTINUED
Later the Lotus brand itself was disCONTINUED
As the same conference we heard about the convergence of Developer tools moving forward (Of which Xpages is a part).
In Notes 7 we saw DB2 Integration being added and then forgotten about by the time Notes 8 was released.
In Notes 8 we had Composite Applications… These lasted one Lotuspehere and the release of 8.5. before they too were forgotten.
And in 8.5 There was a big push for OneUI… Which is now also being forgotten.
Now even the LotusSphere event itself is being discontinued. I am told great event like this take a minimum of 12 months to plan and there is no clear plan yet emerging how IBM Plan to officially support technical conferences based around XPages technology.
So we really should ask how long it might be before XPages joins Notes and any other Lotus products on the Lotus Scrap heap.
Next there is the issue of design. One of the reasons we have such a large technical debt is the structure of the code that was used to develop many of these applications by Citizen developers. The combination of developers without formal training in software engineering and a platform that encourages the development of spaghetti code means we already have 50 billion lines of badly written code to address. And Xpages has been written in a way more consistent with Notes client development than non-Notes development.
Out of the box Pages does not support modern software engineering such as SOA, MV Something. Yes you can do it, but it requires a lot of additional effort and is often beyond the reach of many Notes client developers. Continued XPages development without these modern practices in place is merely adding to the level of technical debt being accumulated by your company.
With XPages it seems like we are always playing catch up with technology.
Xpages is based around dojo, while most of has have been trying to focus on jQuery and Bootstrap. It is often hard to navigate around dojo to use jQuery.
Domino Designer is based on Eclipse 3.4 when we are now up to Eclipse 4.4. The fact that XPages continues to be tied to Notes Classic development we are prevented from using the many enahcement made to Eclipse since the Batman movie was released.
We are using the HTTP Servlet that was around at the time YouTube first appeared.
Java 6 was released back when Germany hosted the world cup. Developers in many other platforms are now using Java 8.
And SSJS is based upon a version of ECMAScript that is as old as the Euro. It is difficult to image that SSJS will ever be enhanced to support the feature being made available this years with ES6.
Finally, the biggest draw-back for XPages is the fact that is it’s a proprietary language. And not even one that is used across the IBM platform. It is a language that is only used as part of Notes/Domino
A recent search of Stackoverflow found nearly 1 million questions tagged with java.
Over half a million with jQuery
But only 3,000 for XPages
And 500 for SSJS. The pool of available talent to develop on these platforms, the amount fo resources for training & support, third part products, and the transferability of your own skills will be influenced by the relatively small market share that XPages has in the development markeplace.
The third choice to address the technical debt is to move existing Notes applications to a modern Web architecture.
Here is the client-server architecture we are all familiar with for Notes/Domino. A thick client running a combination of @Formula and LotusScript programmed into forms, views and agents. The user has a single desktop screen for which a general assumption is made about the screen resolution. On the Domino servers we run scheduled Agents, index our views and store the data in NSFS (either directly or via replication).
But our world has changed and we now have a much wider range of devices to work with and some (or all) of our data may be now hosted in the cloud.
Services Oriented architecture is not a new concept to the programming world, but it is still fairly new in the Notes/Domini world.
One of the more recent changes has been a move away from SOAP services to REST services. In doing this we effectively abstract away the fact that our underlying data is Notes. And because the data is still the same Notes databases as before we can continue to operate our existing Notes client, Xpages, or Domino web interfaces to these applications, significantly reducing the risks associated with our modernization.
Notice we do leave a small gap because there are often good )performance) reasons for allowing direct access to the data.
We no longer live in a Windows only world (if we ever did!). To address the range of platforms on which our devices now run we develop our applications for Web browsers because the devices almost always are capable of running one of the leading Web browsers. These all support HTML 5, CSS3, and javaScript, providing us with a single consistent, ion-proprietary set of standards upon which to build our applications.
We will be using HTML 5 to define the structure of our applications, CSS3 to style it, and JavaScript to control its behavior.
Of all the JavaScript framework available in the market, jQuery is the best known. To many JavaScript developers it is considered to be part of the JavaScript programming language. All the modern JavaScript frameworks we are going to consider build on the Jquery framework.
We add jQuery, where its primary role is to extend the capabilities of Javascript to make changes to the page (HTML) structure. This was once known as DHTML.
We have addressed the divergence in platforms by building software for modern Web browsers. To address the need of different resolutions on our devices we adopt responsive design.
There are many responsive design frameworks in the market. Twitter bootstrap is perhaps the best known of these. But there are newer alternatives such as Google Polymer that are appearing ion the horizon that have the potential to redefine the customer expectation for Web applications.
SO we add our responsive layoer on top of Jquery.
To ensure we build Web applications with a look and feel of native client applications it is important to implement the design patters associated with Single Page Applications.
Supporting these Single Page applications we are going to require the adoption of MVC/MVP (MV something) Code software
And ti do this we will need to select from a growing range of Javascript Framework. Angular.jsis a popular within the Notes/Domino community. We at Red Pill Development favor Backbone.js supplemented with Marionette and Select
This completes our Modern Web architecture. So in summary… In order to stop our users looking like
This, we need to ensure we are not looking like
This!
We need to be constantly looking at the road ahead of us.