This document proposes developing domain-specific testing languages for software-as-a-service to raise the level of abstraction in software testing. It describes Ultimate Software's challenges testing their cloud-based human capital management solution and their approach to address these. This involves creating domain models, test commands, and configuration models. A prototype was developed as a Visual Studio extension called Legend to author tests using these abstractions. Lessons learned include the importance of configurable testing frameworks and cross-functional collaboration to build domain models.

1. TOWARDS DOMAIN-SPECIFIC TESTING LANGUAGES
FOR SOFTWARE-AS-A-SERVICE
Dionny Santiago, Adam Cando, Cody Mack, Gabriel Nunez,
Troy Thomas and Tariq M. King
MDHPCL 2013
Workshop on Model-Driven Engineering for High Performance and Cloud
Computing, September 29, 2013, Miami, Florida

2. ©2012-2013 Ultimate Software
AGENDA
 Introduction
 Motivation
 Approach
 Prototype
 Lessons Learned
 Related Work
 Conclusion 2

3. ©2012-2013 Ultimate Software
Introduction: Cloud Computing
3
Introspection
 Cloud Computing delivers “services” over the Internet
 Software, Platform and Infrastructure
 Cloud Characteristics:
 Computational Power
 Massive Data Storage
 Virtualization Technologies
 Scalability and Elasticity
Enables us to tackle more complex problem domains

4. ©2012-2013 Ultimate Software
 Seeks to raise the level of abstraction
used in software development
 Domain Modeling
 Ubiquitous Language
 MDE Tools and Approaches:
 Meta-Modeling
 Model Transformations
 Platform Independent Models into Code
Few works leverage MDE to improve Software Testing
Model-Driven Engineering (MDE)
4

5. ©2012-2013 Ultimate Software
 Ultimate Software is the developer of a cloud-based
Human Capital Management Solution – UltiPro
Testing Cloud-Based HCM Software
5
Testing Challenges
Large Problem Domain
Multiple Tester Roles
Cloud Platform Configuration
Browser Compatibility
Desktop and Mobile Products
Multi-Language Support

6. ©2012-2013 Ultimate Software
Approach
6

7. ©2012-2013 Ultimate Software
 Actions
 Set
 Blur
 Click
 Assertions
 Exists
 Is
Abstract Test Commands
7

8. ©2012-2013 Ultimate Software
Application Domain Models
8

9. ©2012-2013 Ultimate Software
Application Domain Models
9
 Domain Macro: Payroll Setup

10. ©2012-2013 Ultimate Software
Test Harness Configuration
Timing
Logging
Client-side Platform
Desktop vs. Mobile
Web Browser Configuration
Server-side Environment
Web Servers
Database Servers
Configuration Models
10

11. ©2012-2013 Ultimate Software
Example Test Specification
11

12. ©2012-2013 Ultimate Software
 Developed as a Visual Studio Extension written in C#
 Syntax Coloring
 Error Highlighting
 Intelli-Sense
 Block Outlining
 Tooltips
 Debugging
Prototype: Legend
12

13. ©2012-2013 Ultimate Software
 Developed as a Visual Studio Extension written in C#
 Syntax Coloring
 Error Highlighting
 Intelli-Sense
 Block Outlining
 Tooltips
 Debugging
Prototype: Legend
13

14. ©2012-2013 Ultimate Software
Prototype: Design
14

15. ©2012-2013 Ultimate Software
Prototype: Configuration
Configuration prototype
Abstractions for client and server configurations
Two distinct XML files

16. ©2012-2013 Ultimate Software
Configurable testing frameworks are key
Building application domain models requires
cross-functional collaboration
Test automation highly simplified
Ability to debug tests in a domain language
Using Legend may eliminate need for separate test
inventory
Lessons Learned
16

17. ©2012-2013 Ultimate Software
 MDE Approach to Platform Independent Tests
Hernandez et al.
Technique for designing platform independent tests
Platform-specific tests are generated when combined
with application models
 Behavioral-Driven Development Frameworks
RSpec / Cucumber / SpecFlow
Aid in tying acceptance tests to business requirements
Typically create and link two sets of files – specifications
and step definitions*
Related Work
17

18. ©2012-2013 Ultimate Software
Domain-specific test case specification language
Our approach, while focused on web-based applications,
could be applied to any domain
Prototyped proposed approach for a cloud-based human
capital management solution
Next steps:
Develop a full implementation of Legend
Perform a case study using data from UltiPro
Conclusion
18

19. ©2012-2013 Ultimate Software
Jorge Martinez
Michael Mattera
Virtual Team at Ultimate Software
Judges and Participants at Summer 2012 Ultimate
Software 48 Hours Project
Acknowledgements
19

20. ©2012-2013 Ultimate Software
Open Discussion
20
Introspection
Questions?
¿Preguntas?
問題
Sawwal
вопросы
質問
domande
ερωτήσεις