Equation tutor for sighted and visually impaired children

AAFG 2005

Math-Puzzle:
Equation Tutor for Sighted and
Visually-Impaired Children
Jarno Jokinen
Department of Computer Sciences
University of Tampere Finland

April, 2005


Math-Puzzle
introduction
• communication of mathematics is usually visual
• formulas, diagrams, graphs etc.
• it is very difficult for blind and partially sighted people /
students to do mathematics and is one of the biggest
obstacles for them in school and at the university


Math-Puzzle

most of the work reported in developing techniques that deal with
mathematics can be presented through next categories:
• tactile as in Braille and other raised representations
• haptic or forced feedback devices represent shapes and curves
• tonal representing equations and graphs (sonification / audification)
• audio aids that read equations with tools to help in reading process
• integrated /multimodal approaches

Math-Puzzle
Introduction into the games

/>
/>

Math-Puzzle

/>

Math-Puzzle
project impetus
• created a game instead of making tests with an existing game
• mathematical game for blind children
• after different ideas, Math-Puzzle was chosen
• training in logics, memory and equations’ manipulation strategy


Math-Puzzle
the problems in question
• is it possible to solve equations using only short speech cues?
• what are the limitations?
• what is the easy way for blind interaction to edit the equations in
static or dynamical puzzle? (memory capacity or external memory aid
see next slide)
• how long does it take to solve a puzzle?
• what are the parameters of the gameplay progress and player
performance?
• how much does it make difference when player has visual feedback

or all the tasks (including navigation within the game-field) are
presented through sounds?
• how and in which order the equations could be solved? (strategies)


Math-Puzzle
different matrix approaches
• static puzzle on the left
• dynamic puzzle on the right


Math-Puzzle
game concept
• 5x5 matrix with one equation in each row
• equations are predefined and randomly picked for the matrix
• the goal of the game is to change the places of the equation members
so that all of the equations are true
• the figures (only in the same column) can be swapped by clicking one
number and then the other one
• operators cannot be swapped as well
other controls are:
• alt+R -> new game (reset)
• alt+M -> minimize / maximize browser window
• alt+S -> show / hide figures and operators
• the puzzle completion time and the number of moves are calculated
• the game is currently implemented only in www with limitations


Math-Puzzle
testing procedure

• 5 technically aware adults (age ranged from 26 to 34)
• 10 games per player with all three playing modes:
1st visual, 2nd blind (hidden labels with sound cues), 3rd blindfolded
• laptop pc with external mouse and headphones


Math-Puzzle
problems
• sound feedback is not supported with
mozilla browsers
•

completing

the

fourth

equation

usually completes also the fifth
• if not, the answers are crossing each
other in a way that makes it very
difficult to solve the puzzle
• the program sometimes gives 1 or
more correct equations at the start
which may also result in an error on
the browser

Math-Puzzle
results and discussions
• the game (blind mode) is hard for adults that can see
• what about kids that can’t? They have better spatial understanding
• some matrixes can be a lot faster solved than others – repetitions
needed for a good estimate of skills
• if you wan’t to hear the sound cues you need to be patient
• sound is heard when the mouse is moved over the square, so if you
want to hear it again you must move the mouse away and over again
• completing the fourth equation is crucial as explained on next slide


Math-Puzzle
about the fourth equation
OK
OK

OK
OK


Math-Puzzle

conclusions
• game requires a lot of memorizing, but there are strategies that help
• completing equations in some order
• using mathematical rules (division and multiplication)
• when playing the game blindfolded, the increase in speed was bigger
than when playing the visual version

Math-Puzzle
References
/> /> />Children’s math project /> />Lambda-project: Linear Access to Mathematic for Braille Device and Audio-synthesis
/>Karshmer, A.I., Gupta, G., Gillan, D. Architecting an Auditory Browser for Navigating Mathematical
Expressions, ICCHP 2002, LNCS 2398, p. 477
/>Gaura, P. REMathEx - Reader and Editor of the Mathematical Expressions for Blind Students, 2002,
LNCS 2398, p. 486, />Fitzpatrick D. Speaking Technical Documents: Using Prosody to Convey Textual and Mathematical
Material, ICCHP 2002, LNCS 2398, p. 494,
/> />Math project, />Prosody in Mathtalk />Mathematical Access for Technology and Science,
/>Edwards, A. D. N., Stevens, R. D. and Pitt, I. J. Représentation non visuelle des mathématiques,
(translated by A. Assimacopoulos) in A. B. Safran and A. Assimacopoulos (editors) Le Déficit Visuel,
Éditions Masson, pp. 169–178 (1995),
/>Karshmer, A.I., Gupta, G., Geiger, S., and Weaver, C.: Reading and Writing Mathematics: The
MAVIS Project, BIT (Behaviour & Information Technology), January 1999