Implementation
and Testing
Part Two -VR Software Test Implementation
VR Test Data Analysis
To help analyse the Voice Recognition Test data, a Microsoft Excel
spreadsheet was again assembled to list the data from all tests with
each student, and then calculate the findings. These test results
should help to give a further indication as to why the software does
not seem to be as suitable or effective as it potentially could be.
Graph 10 shows the dictation times over the three tasks,
showing the students from the Original Study in Blue, and the New
Study in Red. It can be seen that the students from the new study
were faster across all three tasks, not just the voice dictation.
It could have been expected that there would be a more significant
increase in the speed of the typed task for the New students due to
the general increase in popularity of computers, and the fact that
all of the New students have access to computers at home as well as
in the classroom. In comparison, however, it was documented that the
Original students took part in a one-hourly session learning keyboard
skills over a 14-week period, prior to the study, whereas, although
the New students have had weekly sessions in general computer studies,
none had been given formal instruction specifically on keyboards skills.
Closer examination of the voice dictation times, as shown in
Graph 11, shows just how much faster dictation times are now with the
new version of the software. Average times recorded across all students
show that dictation with the latest version was almost twice as fast
as the older version.
This improvement must be credited to the ability of the software to
now be able to recognise 'natural' speech, as opposed to the 'discreet'
speech method previously required. Discreet speech in the Original
study required the students to speak in a slow, unnatural way, pausing
between words. To enable them to do this, they had several practice
sessions in isolated speech. With the New study this was not necessary,
as the software now asks the user to speak clearly, but naturally,
at their usual speed. This enabled the New students to average input
at 79 words per minute, whereas it had previously only been 47 words
per minute. This method of input is also considerably faster than
the other two methods. Though slightly faster overall at all three
tasks than their predecessors, the New students still only managed
to produce an average of 15 words per minute using the handwritten
method, whilst achieving a painfully slow 9 words per minute using
the typing method.
As far as accuracy is concerned, though, it still has quite a way
to go. Across all readings the new version only averaged 55% accuracy,
whereas in the Original study it produced 79% accuracy. These figures
must also be attributed to the differences between discreet and natural
speech input. Practicing isolated speech gave the Original students
a chance to focus on the pronunciation of each word, encouraging them
to speak more clearly. This would undoubtedly produce more accurate
results. The New students, being asked to speak in their usual voice
produced less accurate results as the software substitute words it
does not recognise with other words, which in most cases have completely
different meanings. This did however lead to much hilarity and was
a great source of entertainment for most of the students.
Complete tables of results from each study, and some handwritten,
typed and voice dictated samples of the text can be viewed in the
VR Test Results section of this
report, but from the overall results in
Graph 12, it can be seen that handwritten work still produces
the most accurate - though not necessarily the neatest - results.
Even though Voice Dictation appears to be considerably
less accurate, it does become much more accurate over time. Taking
into that into consideration along with the extremely slow input times
for typed and handwritten text, even allowing additional time for
correction of voice dictated text, it would still be possible to produce
an accurate, word processed document in less than half the time.
By analysing each student's results individually and observation during
the tests, it is clear that voice dictation would be extremely productive
for some students; but would not necessarily work for others.
Graph 13 shows the productivity achieved by each student for
both typed and voice dictated tasks. This shows just how significantly
faster voice dictation is. However, on examining
Graph 14 for
levels of accuracy for individual students, we see quite different
results.
The three students who produced the lowest levels of accuracy for
the voice dicatation (Paul, John B and John M) were also three of
the slowest to perform the typing task. This had a substantial bearing
on the length of time left during the session to train their voice
model. Both Paul and John B were unable to finish the voice model
training exercise, therefore this would at first appear to be why
their sample text was so much less accurate than the others. On the
other hand, Shannon was also unable to finish training her voice model
- she only managed to record ½ of the Homework Story - but
still managed to produce better results from the voice dication than
the typing task.
This final piece of information is the most surprising of all, since
Shannon has the lowest reading age (see Appendix Three for a full
list of reading ages) of all the students tested but managed to produce
one of the highest accuracy levels with the voice dictation (73%).
This student had also indicated that her performance may be affected
by the fact that the session took place just after a vigourous PE
session and lunch. She was also one of the most reluctant to read
aloud. Taking all of these factors into consideration, the fact that
she achieved such a high level of performance and accuracy after a
one short session, is quite remarkable.
Read
the rest of this report
