We wanted to be able to certify users of our software. Now there are several methods to do that already
We could have used a written test, but does that really give us a good idea of how well they use the software.
Step 1: Get the software to make statements
So we see here in the software that the user is asked to enter name
We narrowed this down to two verbs that we would start using. Interacted for entering a panel
Interacted was one o f the defined verbs from the ADL so we were able to use that out of the box.
The second verb we used was created for when the user clicked the button in the software to create the object. Our software writes these out as a * command so they are easily distinguishable from the output.
We made a web page that has the URI for the verb, we could also add this to the verb registry
So lets say I entered my name when I ran the tcl.file
When I enter a panel a statement is written
SO the statement would look like this: Sean Putman interacted with panel name. This lets us know that user has entered the panel. This will also become important later as we try to set context
Here is what our output file looked like
Once in the panel, the user then creates some type of entity or performs a task
and we use the Create verb for this task. Essentially we are running a star command
So that gives us a statement that looks like this Sean Putman created *command
And here is the output for that
There is one final piece to these statements, a timestamp.
Step 2: building the LRS
Which led us to this, real time display of statements being generated from the software. SO this is great, but what does it mean? WE now have data for data’s sake. We can see data being generated by the clicks, but there is no meaning behind it, yet
Step 3: Making the data useable
Here is the initial timeline. Show image of initial timeline, we can mouse over each line and get information on the click that took place. This was great we could compare two runs and see what took place and what the differences were, but we were still missing something
Step 4: Context
We started by running through a known exercise that we could document. We then figured out the beginning and ending statements for each step. This gave us the beginning and end of an activity. The nice thing is that the beginning and end are always the same. The person interacted with a panel and a star command is run, the same star command. What happens in between is where it can be different and where the real comparison takes place.
Did they struggle in certain areas?
Now that we have added the context we can also make suggestions for courses they might want to take based on where they took longer to complete tasks or for task they did not complete. So we are able to give them the pointed training that they need to improve their skills. We are adding keywords to our online modules that map to the context to make that much easier. Now does this currently tell us everything?