Generating and Using Accessible Mathematics on the Web

Generating and Using
Accessible Mathematics on the
Web
Sam Dooley
Pearson Assessment



Dynamic Online Accessible Math
— Online Braille Math
/>

Braille math is hard!



Blind students need high-quality braille math



Advanced preparation is costly and takes time



Math teachers are often unfamiliar with
braille



3

TVIs are often unfamiliar with math notation

Generating and Using Accessible Mathematics on the Web

16 Nov 2016


Braille math software is
harder!

4



Software translation can be problematic



Forward translation takes many steps



Real-time back translation is non-existent



No support for online or classroom use

Generating and Using Accessible Mathematics on the Web

16 Nov 2016


Braille math needs to be
online!
Blind students need:
A

5

level playing field for STEM instruction

To

read and write online braille math

To

interact with sighted instructors and peers

To

participate in online learning

To

participate in online assessment

Generating and Using Accessible Mathematics on the Web

16 Nov 2016


Braille math should be math!



Math concepts are independent of notation



Braille math codes capture all math notation



Math software can be independent of notation

Blind students only have full access to math
if their math is treated the same as printed
math.
6

Generating and Using Accessible Mathematics on the Web

16 Nov 2016


Dynamic Online Accessible Math
— Accessible Equation
Editor

/>

Accessible Equation Editor

8



Sighted user can create math for a blind user



Blind user can create math for a sighted user



Real-time, two-way braille math translation



Instantaneous interactions with math content



Discoverable braille math encoding rules

Generating and Using Accessible Mathematics on the Web

16 Nov 2016

QWERTY Keyboard Input

9



Math symbols on the keyboard



Other symbols on the buttons



Keyboard cursor navigation



Implicit and explicit selection



Backspace, delete, clear

Generating and Using Accessible Mathematics on the Web

16 Nov 2016

Braille Terminal Output



Math output on the screen



Braille output on the screen



Braille output on the terminal



Screen reader support

10

Generating and Using Accessible Mathematics on the Web

16 Nov 2016


Braille Terminal Input



Each braille cell is like a key



Sequences of keys create the math



Tooltips with text and braille



Cursor position and routing

11

Generating and Using Accessible Mathematics on the Web

16 Nov 2016


QWERTY Terminal Output



Braille input creates math content



Math content becomes math output



Math output becomes braille output



The math markup is exactly the same

12

Generating and Using Accessible Mathematics on the Web

16 Nov 2016


Research Studies


Two research studies in Fall 2015
(KY/AZ - Sep, TX - Oct)



Goal - to collect feedback from multiple
populations on AEE functionality



Populations - blind, low vision, regular
print readers, learning disabled



Criteria - high school students who had
completed Algebra I

13

Generating and Using Accessible Mathematics on the Web

16 Nov 2016


Research Outcomes



Students had limited knowledge of Nemeth



Erasing math content was unpredictable



Working with grouping symbols was difficult



Entering and closing fractions was

unexpected


14

Ending trigonometric expressions was unusual

Generating and Using Accessible Mathematics on the Web

16 Nov 2016


Ongoing Research


Additional research studies in 2016
(NFB – Jun/Jul, TBA – Sep/Oct)



Goal - to collect feedback on new and more
advanced AEE functionality



Math expression navigation



Math expression modification



Nemeth Braille discoverability

15

Generating and Using Accessible Mathematics on the Web

16 Nov 2016


Updated Outcomes


Cursor tracking improved position awareness



Dot 8 helped identify closed expressions



Erasing content is still unpredictable



Nemeth discoverability is helpful, but the
keyboard navigation is overwhelming



16

Nemeth symbol search was requested

Generating and Using Accessible Mathematics on the Web

16 Nov 2016


Technical objectives



Real-time translation from math into braille



Real-time translation from braille into math



Accessible to both sighted and blind users



Online equation editor software component

17

Generating and Using Accessible Mathematics on the Web

16 Nov 2016


Accessible Online Math!

Blind students can have:
A

level playing field for STEM instruction

Tools

to read and write online braille math

Interaction

with sighted instructors and

peers
Active
18

participation in online activities

Generating and Using Accessible Mathematics on the Web

16 Nov 2016

Braille math is math!



Blind students can read the same math



Blind students can write the same math



The math can be shared the same way



The math can be scored the same way

Blind students can now have full access to
math since their math is the same as printed
math.
19

Generating and Using Accessible Mathematics on the Web

16 Nov 2016

Dynamic Online Accessible Math
— Online Spoken Math


Speech output for static math



Browser



MathJax



Presentation MathML



Screen reader



MathPlayer

21

Generating and Using Accessible Mathematics on the Web

16 Nov 2016


Speech output for dynamic math



Speech output for the entire content



Speech output for the input context



Speech output for the input actions



Speech output for keyboard events

22

Generating and Using Accessible Mathematics on the Web

16 Nov 2016


Natural spoken math output



Dynamic math produces more math output



User intervention is even more essential



More output means more math speech modes



Spoken output can reinforce braille output

Blind students can have math read for them,
and they can read their math for themselves.
23

Generating and Using Accessible Mathematics on the Web

16 Nov 2016


Speech input for dynamic math



Dragon Naturally Speaking



Recognizes words from spoken input



Transmits characters to applications



Supports popular software packages

24

Generating and Using Accessible Mathematics on the Web

16 Nov 2016


Custom speech input rules



Custom rules support specific software



Grammar rules recognize spoken phrases



Spoken phrases become character sequences



Character sequences become input tokens



Input tokens invoke editor template rules

25

Generating and Using Accessible Mathematics on the Web

16 Nov 2016