2. filemaker.com Advisor.com
Who am I?
! Author
" Special Edition Using FileMaker Pro 7
" FileMaker Professional Training Foundation Series
II Courseware
! Consultant
" Manage a team in Silicon Valley for Soliant
Consulting
" Specialist in commercial FM solutions
" FileMaker Pro 7 Certified
! Trainer
" Authorized instructor for FileMaker Professional
Training Foundation Series II
www.soliantconsulting.com
4. filemaker.com Advisor.com
Today s Goal
! Review the elements of QA
! Talk about the QA process in general
! Discuss how to successfully apply it to FileMaker
Pro
! Mention some great tools you should be using
5. filemaker.com Advisor.com
What re the Challenges for
FileMaker?
! Easy to use ≠ No brainer
! Often treated like a productivity application
! When is done done?
! Small teams, often home-grown
! RAD tool = smaller budget & NEVER enough time
! QA teams don t grow on trees
! Flexibility can be a curse
! Sometimes abstraction isn t worth the cost
! Users always want more
6. filemaker.com Advisor.com
Why is this so difficult?!?
! Still a nascent science.
! We lack a common vocabulary.
! Users may not be able grasp implications.
! No truly pre-fabricated parts.
! No universal standards.
7. filemaker.com Advisor.com
The path out of the woods…
! Bugs are a fact of life… the trick is what to do
about them.
! Developers: learn to debug and follow standards.
! Managers: make sure testing cycles and
expectations are clear.
! Users: be tolerant and learn about the process.
Rapid Application Development means we re
often not building software that can afford
months in QA.
! COMMUNICATE!
8. filemaker.com Advisor.com
So… What s a Bug?
! A bug occurs when a feature in your system
conflicts with your requirements and causes your
system to malfunction.
" A script that goes to the wrong layout?
" A speeling mistake?
" Is a missing feature a bug?
" What about a sloppy layout design?
" How about a script that fails if you pass it a
parameter that your buttons would never send?
" An uncommented script?
" A file that won t open?
" A bad NIC card?
9. filemaker.com Advisor.com
We re Really Asking: What s Quality?
! Quality can mean a lot of things…
" Visual consistency & usability
" Data integrity
" Proper relational modeling
" Accurate calculations
" Correct Functions
! Those are easy to measure… but what about...?
" Meets business goals
" Easy to learn
" Requires little IT support
! Or worse yet…
" The users like it
10. filemaker.com Advisor.com
Quality is Relative.
! Quality is what the clients or users say it is.
! Quality is defined differently for each project,
and each workgroup.
" Establish the metrics by which you ll measure and
gauge quality.
" Represent and evaluate the appropriate level of
quality assurance and testing.
" Define user-acceptance vs. quality-assurance
processes.
11. filemaker.com Advisor.com
Once Defined, How is Quality
Reached?
! Well-trained Developers
! Solid Development Practices
! Well-defined Requirements OR Well-established
Mutual Expectations
! Use Case Scenarios
! In-Dev QA Testing
! Deployment Testing
! User Acceptance Testing
12. filemaker.com Advisor.com
Tools of the Trade
! Requirements Document
" set the goal
! Test Plan
" define quality and methodology
! Use Case Database
" step-by-step tests used to measure results
! Bug Database
" process and data tracking of bugs
! Development & Testing tools
" Developer 7, Analyzer, Brushfire, MetaDataMagic
! Create Templates… make this second nature
13. filemaker.com Advisor.com
Tools: Requirements Doc
! Overall goals
! Strategic
! Scope
! Covers:
" Schedule & Budget
" Business Goals
" Base Architecture
" Entities Relationship Diagram
" Field Specification
" Screen layout designs/descriptions
" Business logic
" Functional work flows
" System & hardware requirements
# Load, Capacity, Deployment
15. filemaker.com Advisor.com
Tools: Use Case Database
! For FileMaker Pro projects, it s got to be
evolutionary and scalable
! Each tells a specific story
! What goes in? What comes out?
! Step-by-step when necessary
16. filemaker.com Advisor.com
Tools: Bug Database
! This is process tool more than anything else.
! Who owns what when.
" Tester enters a new bug
" QA Lead reviews and assigns or rejects
# Also assign severity/priority
" Developer(s) pass the bug along the chain
" Tester verifies and closes
! Do you give users & clients access?
19. filemaker.com Advisor.com
A few helpful terms:
! Unit Testing
" Testing a discrete function, like a custom
function, often done by the developer.
! System Testing
" Soup-to-nuts test of a solution.
! Regression Testing
" Retesting of either units or system after a version
or fix has been iterated.
! Black Box Testing
" Testing/reviewing the code itself.
21. filemaker.com Advisor.com
Process: QA Starts with Design
! In a complex system, it s not enough to just get
the basic relational model correct.
" Performance
" Security
" Validation & data integrity
" Multi-user record locking
" User Interface standards
" Coding standards
! Requirements Documents
! Use Case Scenarios
22. filemaker.com Advisor.com
Process: QA During Development
! By far, the most efficient QA gains occur when developers
correct and prevent their own bugs.
" Coding Standards
# Naming Conventions
# Script & Calc Comments
# Documentation Standards
" Debugging Practices
" Testing tools
# MetaDataMagic, Analyzer, Brushfire, etc…
" Use Case Scenarios
" QA Pre-Flight Checklist
23. filemaker.com Advisor.com
Process: Alpha Testing
! Your workgroup still hasn t seen the system yet…
! Alpha = it s not done, but it s not not done.
" QA In-Flight Checklist
" Categories: FileMaker general, Project specific, etc.
" Get your QA team and your dev team talking!
" Use a Bugbase. Anything s better than nothing.
" Use Case Scenarios
24. filemaker.com Advisor.com
Process: Beta Testing
! Beta = it s functionally complete, but largely
untested in a formal sense.
" QA Pre-Release Checklist
" Start making tough choices. The goal can never be
zero bugs.
" It s all about prioritization.
" Include workgroup users/clients as you wish, but
make the decision deliberate. (I strongly advocate
an open book methodology.)
25. filemaker.com Advisor.com
Process: Deployment Testing
! Get it out of the lab. Put it in the real world.
" No kidding. The REAL world. Real network, real
server, real backup routines, real users.
" Don t leave this to an hour before the client
expects final delivery.
" Use Case Scenarios yet again!
26. filemaker.com Advisor.com
Process: Acceptance Testing
! Not so much a does it work but rather a does it
meet our goals?
" This is more open ended, or more structured,
depending on your workgroup.
" The goal is to reach a point in time when the users
say, Yes, you re done.
" By definition, once acceptance testing is complete,
1.0 is a closed box.
27. filemaker.com Advisor.com
…Yeah, but what if…?
! Some users will never accept their system.
! Some will accept it and still find bugs… legitimate,
high-priority bugs.
! Times change.
! People change.
! Systems change.
! S%&! happens.
! The goal is to have a process you can rely on. In
a structured environment, you can all know what
to expect when life gets unpredictable.
