Tec314

filemaker.com Advisor.comfilemaker.com Advisor.com
Software QASoftware QA
and FileMaker Proand FileMaker Pro
Scott LoveScott Love
Soliant ConsultingSoliant Consulting
TEC314

Who am I?Who am I?

AuthorAuthor

Special Edition Using FileMaker Pro 7Special Edition Using FileMaker Pro 7

FileMaker Professional Training Foundation SeriesFileMaker Professional Training Foundation Series
II CoursewareII Courseware

ConsultantConsultant

Manage a team in Silicon Valley for SoliantManage a team in Silicon Valley for Soliant
ConsultingConsulting

Specialist in commercial FM solutionsSpecialist in commercial FM solutions

FileMaker Pro 7 CertifiedFileMaker Pro 7 Certified

TrainerTrainer

Authorized instructor for FileMaker ProfessionalAuthorized instructor for FileMaker Professional
Training Foundation Series IITraining Foundation Series II
www.soliantconsulting.com

QAQA

Well… at least I can spell it.Well… at least I can spell it.

Today’s GoalToday’s Goal

Review the elements of QAReview the elements of QA

Talk about the QA process in generalTalk about the QA process in general

Discuss how to successfully apply it to FileMakerDiscuss how to successfully apply it to FileMaker
ProPro

Mention some great tools you should be usingMention some great tools you should be using

What’re the Challenges forWhat’re the Challenges for
FileMaker?FileMaker?

Easy to use ≠ No brainerEasy to use ≠ No brainer

Often treated like a productivity applicationOften treated like a productivity application

When is “done” done?When is “done” done?

Small teams, often home-grownSmall teams, often home-grown

RAD tool = smaller budget & NEVER enough timeRAD tool = smaller budget & NEVER enough time

QA teams don’t grow on treesQA teams don’t grow on trees

Flexibility can be a curseFlexibility can be a curse

Sometimes abstraction isn’t worth the costSometimes abstraction isn’t worth the cost

Users always want moreUsers always want more

Why is thisWhy is this so difficult?!?so difficult?!?

Still a nascent science.Still a nascent science.

We lack a common vocabulary.We lack a common vocabulary.

Users may not be able grasp implications.Users may not be able grasp implications.

No truly pre-fabricated parts.No truly pre-fabricated parts.

No universal standards.No universal standards.

The path out of the woods…The path out of the woods…

Bugs are a fact of life… the trick is what to doBugs are a fact of life… the trick is what to do
about them.about them.

Developers: learn to debug and follow standards.Developers: learn to debug and follow standards.

Managers: make sure testing cycles andManagers: make sure testing cycles and
expectations are clear.expectations are clear.

Users: be tolerant and learn about the process.Users: be tolerant and learn about the process.
Rapid Application Development means we’reRapid Application Development means we’re
often not building software that can affordoften not building software that can afford
months in QA.months in QA.

COMMUNICATE!COMMUNICATE!

So… What’s a Bug?So… What’s a Bug?

A bug occurs when a feature in your systemA bug occurs when a feature in your system
conflicts with your requirements and causes yourconflicts with your requirements and causes your
system to malfunction.system to malfunction.

A script that goes to the wrong layout?A script that goes to the wrong layout?

A speeling mistake?A speeling mistake?

Is a missing feature a bug?Is a missing feature a bug?

What about a sloppy layout design?What about a sloppy layout design?

How about a script that fails if you pass it aHow about a script that fails if you pass it a
parameter that your buttons would never send?parameter that your buttons would never send?

An uncommented script?An uncommented script?

A file that won’t open?A file that won’t open?

A bad NIC card?A bad NIC card?

We’re Really Asking: What’s Quality?We’re Really Asking: What’s Quality?

Quality can mean a lot of things…Quality can mean a lot of things…

Visual consistency & usabilityVisual consistency & usability

Data integrityData integrity

Proper relational modelingProper relational modeling

Accurate calculationsAccurate calculations

Correct FunctionsCorrect Functions

Those are easy to measure… but what about...?Those are easy to measure… but what about...?

Meets business goalsMeets business goals

Easy to learnEasy to learn

Requires little IT supportRequires little IT support

Or worse yet…Or worse yet…

The users like itThe users like it

Quality is Relative.Quality is Relative.

Quality is what the clients or users say it is.Quality is what the clients or users say it is.

Quality is defined differently for each project,Quality is defined differently for each project,
and each workgroup.and each workgroup.

Establish the metrics by which you’ll measure andEstablish the metrics by which you’ll measure and
gauge quality.gauge quality.

Represent and evaluate theRepresent and evaluate the appropriate levelappropriate level ofof
quality assurance and testing.quality assurance and testing.

DefineDefine user-acceptanceuser-acceptance vs.vs. quality-assurancequality-assurance
processes.processes.

Once Defined, How is QualityOnce Defined, How is Quality
Reached?Reached?

Well-trained DevelopersWell-trained Developers

Solid Development PracticesSolid Development Practices

Well-defined Requirements OR Well-establishedWell-defined Requirements OR Well-established
Mutual ExpectationsMutual Expectations

Use Case ScenariosUse Case Scenarios

In-Dev QA TestingIn-Dev QA Testing

Deployment TestingDeployment Testing

User Acceptance TestingUser Acceptance Testing

Tools of the TradeTools of the Trade

Requirements DocumentRequirements Document

set the goalset the goal

Test PlanTest Plan

define quality and methodologydefine quality and methodology

Use Case DatabaseUse Case Database

step-by-step tests used tostep-by-step tests used to measuremeasure resultsresults

Bug DatabaseBug Database

process and data tracking of bugsprocess and data tracking of bugs

Development & Testing toolsDevelopment & Testing tools

Developer 7, Analyzer, Brushfire, MetaDataMagicDeveloper 7, Analyzer, Brushfire, MetaDataMagic

Create Templates… make this second natureCreate Templates… make this second nature

Tools: Requirements DocTools: Requirements Doc

Overall goalsOverall goals

StrategicStrategic

ScopeScope

Covers:Covers:

Schedule & BudgetSchedule & Budget

Business GoalsBusiness Goals

Base ArchitectureBase Architecture

Entities Relationship DiagramEntities Relationship Diagram

Field SpecificationField Specification

Screen layout designs/descriptionsScreen layout designs/descriptions

Business logicBusiness logic

Functional work flowsFunctional work flows

System & hardware requirementsSystem & hardware requirements

Load, Capacity, DeploymentLoad, Capacity, Deployment

Tools: Test PlanTools: Test Plan

Quality DefinitionQuality Definition

MethodologyMethodology

Risks & AssumptionsRisks & Assumptions

Exit/Suspension CriteriaExit/Suspension Criteria

Tools: Use Case DatabaseTools: Use Case Database

For FileMaker Pro projects, it’s got to beFor FileMaker Pro projects, it’s got to be
evolutionary and scalableevolutionary and scalable

Each tells aEach tells a specificspecific storystory

What goes in? What comes out?What goes in? What comes out?

Step-by-step when necessaryStep-by-step when necessary

Tools: Bug DatabaseTools: Bug Database

This isThis is processprocess tool more than anything else.tool more than anything else.

Who owns what when.Who owns what when.

Tester enters a new bugTester enters a new bug

QA Lead reviews and assigns or rejectsQA Lead reviews and assigns or rejects

Also assign severity/priorityAlso assign severity/priority

Developer(s) pass the bug along the chainDeveloper(s) pass the bug along the chain

Tester verifies and closesTester verifies and closes

Do you give users & clients access?Do you give users & clients access?

Tools: Testing ToolsTools: Testing Tools

Code Abstraction & OOP Development PracticesCode Abstraction & OOP Development Practices

Error RoutinesError Routines

Coding StandardsCoding Standards

Sample DataSample Data

FileMaker Developer 7FileMaker Developer 7

DDRDDR

DebuggerDebugger

Custom FunctionsCustom Functions

Tools: Testing ToolsTools: Testing Tools

AnalyzerAnalyzer

Waves in MotionWaves in Motion

www.wavesinmotion.comwww.wavesinmotion.com

BrushfireBrushfire

ChaparralChaparral

www.chapsoft.comwww.chapsoft.com

MetaDataMagicMetaDataMagic

New MillenniumNew Millennium

www.newmillennium.comwww.newmillennium.com

A few helpful terms:A few helpful terms:

Unit TestingUnit Testing

Testing a discrete function, like a customTesting a discrete function, like a custom
function, often done by the developer.function, often done by the developer.

System TestingSystem Testing

Soup-to-nuts test of a solution.Soup-to-nuts test of a solution.

Regression TestingRegression Testing

Retesting of either units or system after a versionRetesting of either units or system after a version
or fix has been iterated.or fix has been iterated.

Black Box TestingBlack Box Testing

Testing/reviewing the code itself.Testing/reviewing the code itself.

Some FileMaker IdeasSome FileMaker Ideas

# COMMENTS!# COMMENTS!

Custom FunctionsCustom Functions

Automated testingAutomated testing

““Debug Mode”Debug Mode”

““Breadcrumb Scripting”Breadcrumb Scripting”

Test ScriptsTest Scripts

Evaluate() widgetEvaluate() widget

Process: QA Starts with DesignProcess: QA Starts with Design

In a complex system, it’s not enough to just getIn a complex system, it’s not enough to just get
the basic relational model correct.the basic relational model correct.

PerformancePerformance

SecuritySecurity

Validation & data integrityValidation & data integrity

Multi-user record lockingMulti-user record locking

User Interface standardsUser Interface standards

Coding standardsCoding standards

Requirements DocumentsRequirements Documents


Process: QA During DevelopmentProcess: QA During Development

By farBy far, the most efficient QA gains occur when developers, the most efficient QA gains occur when developers
correct and prevent their own bugs.correct and prevent their own bugs.

Coding StandardsCoding Standards

Naming ConventionsNaming Conventions

Script & Calc CommentsScript & Calc Comments

Documentation StandardsDocumentation Standards

Debugging PracticesDebugging Practices

Testing toolsTesting tools

MetaDataMagic, Analyzer, Brushfire, etc…MetaDataMagic, Analyzer, Brushfire, etc…


QA “Pre-Flight Checklist”QA “Pre-Flight Checklist”

Process: Alpha TestingProcess: Alpha Testing

Your workgroup still hasn’t seen the system yet…Your workgroup still hasn’t seen the system yet…

Alpha = it’s not done, but it’s not not done.Alpha = it’s not done, but it’s not not done.

QA “In-Flight Checklist”QA “In-Flight Checklist”

Categories: FileMaker general, Project specific,Categories: FileMaker general, Project specific,
etc.etc.

Get your QA team and your dev team talking!Get your QA team and your dev team talking!

Use a Bugbase. Anything’s better than nothing.Use a Bugbase. Anything’s better than nothing.


Process: Beta TestingProcess: Beta Testing

Beta = it’s functionally complete, but largelyBeta = it’s functionally complete, but largely
untested in a formal sense.untested in a formal sense.

QA “Pre-Release Checklist”QA “Pre-Release Checklist”

Start making tough choices. The goal can never beStart making tough choices. The goal can never be
“zero bugs.”“zero bugs.”

It’s all about prioritization.It’s all about prioritization.

Include workgroup users/clients as you wish, butInclude workgroup users/clients as you wish, but
make the decision deliberate. (I strongly advocatemake the decision deliberate. (I strongly advocate
an open book methodology.)an open book methodology.)

Process: Deployment TestingProcess: Deployment Testing

Get it out of the lab. Put it in the real world.Get it out of the lab. Put it in the real world.

No kidding. The REAL world. Real network, realNo kidding. The REAL world. Real network, real
server, real backup routines, real users.server, real backup routines, real users.

Don’t leave this to an hour before the clientDon’t leave this to an hour before the client
expects final delivery.expects final delivery.

Use Case Scenarios yet again!Use Case Scenarios yet again!

Process: Acceptance TestingProcess: Acceptance Testing

Not so much a “does it work” but rather a “does itNot so much a “does it work” but rather a “does it
meet our goals?”meet our goals?”

This is more open ended, or more structured,This is more open ended, or more structured,
depending on your workgroup.depending on your workgroup.

The goal is to reach a point in time when the usersThe goal is to reach a point in time when the users
say, “Yes, you’re done.”say, “Yes, you’re done.”

By definition, once acceptance testing is complete,By definition, once acceptance testing is complete,
1.0 is a closed box.1.0 is a closed box.

……Yeah, but what if…?Yeah, but what if…?

Some users will never accept their system.Some users will never accept their system.

Some will accept it and still find bugs… legitimate,Some will accept it and still find bugs… legitimate,
high-priority bugs.high-priority bugs.

Times change.Times change.

People change.People change.

Systems change.Systems change.

S%&! happens.S%&! happens.

The goal is to have a process you can rely on. InThe goal is to have a process you can rely on. In
a structured environment, you can all know whata structured environment, you can all know what
to expect when life gets unpredictable.to expect when life gets unpredictable.

FileMaker Developer ConferenceFileMaker Developer Conference
Web Update PageWeb Update Page
AdvisorEvents.com/CFF0408p.nsf/w/CFF0408udAdvisorEvents.com/CFF0408p.nsf/w/CFF0408ud
This session WILL have updates.This session WILL have updates.

Thank you!Thank you!
Please remember to fill out your evaluation.Please remember to fill out your evaluation.

Tec314

More Related Content

What's hot

Viewers also liked

Similar to Tec314

Tec314