Purpose and Background

For blind students seeking education and a career in science, engineering and mathematics, the calculus has presented a formidable barrier. This is not due to the intrinsic difficulty of the subject, which is obstacle enough for most students. The additional hurdle for blind students is the substantial graphical component of the typical calculus course. There are two major aspects of that graphical component: primarily, there is the representation of geometrical objects, especially the graphs of functions: then, there is the presentation of mathematical formulas as a graphical display.

Under a grant from the National Science Foundation, the  Computer Science Department (CSD) of the College of Staten  Island and the Computer Center for Visually Impaired People (CCVIP) of Baruch College are developing text materials and  providing an environment to offer blind and visually impaired  students technologically assisted access to the graphical  content of calculus. The goal of the project is to equal or  exceed the quality of courses for students with unimpaired  vision. We are installing facilities for reading mathematical  text and graphics directly without the help of sighted readers.  It is only quite recently that any such technology has become  practical and affordable for institutions. We can expect that  it will become so for individuals before long.

We wish to bring such a course into being for the students  who need it right now. For that reason, we are trying to base  our system, insofar as possible, on "off-the-shelf" technology  and courseware, rather than try to invent much of it.

For basic course content, we are adapting the successful self- paced mastery course in calculus developed at Carnegie-Mellon  University for students of science, engineering and mathematics,  developed by Albert Blank assisted by Raymond E. Artz and Erik Hardy (1).

This is being supplemented by essential prerequisite materials that visually impaired students may lack. The original text was oblivious  to the special needs of these students and many small adaptations  will be made. For example, the original text stated in its second  paragraph, "The axes divide the coordinate plane into four quadrants which are traditionally labeled with Roman numerals as shown in  Figure1-1." No further clarification was given.

The use of a course designed for self-pacing is basic to the program.  Our students will work with new multisensory media that present the  course materials and will augment their skills with new language and  new techniques of expression for presenting their own work. They will  generally need extra time, effort and training. Furthermore, thorough  mastery is vital because the calculus is fundamental to most of science  and engineering. Moreover, mastery is important to give the students confidence that they can become proficient in areas that were  hitherto largely inaccessible to them.

Many of the problems of presentation to students with visual  impairments can now be addressed with existing technology in a  multimedia multisensory environment:

• audio-tactile tablets can be prepared and programmed beforehand to  present graphics.
  
• scanners with optical character resolution (OCR) software can be  used to read conventional printed text into ASCII files. Braille  printers with appropriate translation software can render those  files in Braille.
  
• for those who do not read Braille or even those who do, screen  reading systems provide access to ASCII encoded text files.
  
• enlarged display screens are available for those with lesser  degrees of impairment.
  
• Hypermedia techniques can be used to provide easy access at  will to information in the courseware.  
  
   
  

We have been using the touch-sensitive Nomad audio tactile tablet (2) to  present graphic images. Consider the example cited  above as Figure 1.1 in the calculus textbook. The tactile version  of this purely verbal and visual statement is produced on a soft  plastic sheet 16.5 inches wide by 12 inches high (3). A tactile  grid, each cell measuring 1 inch by 1 inch, is produced in relief  on this sheet. A visual display of the figure is presented here.  

The more important details of the figure are presented on the graphic  as heavier and in higher relief than the less important ones in order  to give tactile expression to their varying importance. At the bottom  margin of the graphic, at the base of each vertical line, there is a  round button. When any of these buttons are pressed, Nomad voices  the x-coordinate of that line. Similarly, along the left margin there is  a column of buttons that voice the y-coordinates of the horizontal  gridlines attached to them. The axes are the two special saw tooth  lines that feel bumpy to the touch. The x-axis is the horizontal  saw tooth line attached to the button on the left margin that voices, "y =0;" it feels smoother to the touch when stroked in the positive  direction, left to right, than in the opposite or negative direction.  The y-axis is the vertical saw tooth line attached to the button in the  bottom margin that voices, "x =0." For this axis, the positive direction  and the smoother to the touch is upward. Pressure at the intersection  of the axes will voice, "origin." Pressure at any other point on the  x- axis will voice, "x-axis," etc. Along the upper right margin of the  graphic there is a row of diamond-shaped buttons. A press on one of  these will voice an associated keyword from the text.

Any feature of a graphic can be programmed to voice three levels of  information in succession. This is especially useful when applied to the  ID box, which is a rectangle consistently placed in the lower right corner  of the figure. At that location, the first level of information gives the  figure number and name. The next level lists features of the graphic that  can then be located through Nomad's search capabilities. The third  level can give any information about the graphic or the lesson that the  instructor desires. Nomad lacks high level editing capabilities but it is  easy, though sometimes tedious, to program.

The grid spacing and button placement on the graphic plates is  maintained for all the graphs the student will use in the course. The  keywords will differ depending on the lesson. The origin and axes may  lie anywhere or even be located outside the picture frame. The  coordinates of successive grid points on the axes can differ by some  other constant than one. At the same time, the constant structure of  graphics will offer a consistent, familiar environment in which the  student can operate securely.

The hope for the future is that the process of making a graphic  will be automated so that a blind person can operate interactively  at a work station to create and analyze such a graphic without  requiring the assistance of a sighted person.

2. Presentation of Formulas

a. Optical Character Recognition 
OCR programs offer great promise. However, they are not yet  completely reliable readers even of straightforward literary text.  Moreover, technical print containing formulas is still far beyond  their capabilities.

b. Braille  
Braille systems for rendering formulas exist and others are in  development. The Nemeth code was developed specifically for the  purpose of rendering mathematical expressions in Braille. It uses  standard six dot Braille and, by virtue of adroitly constructed  combinations of Braille characters, is able to represent very  complex expressions. The Nemeth system needs to use compound  characters to represent many of the symbols that are single  characters on the keyboard. A more significant difficulty is that  the code represents the graphical elements  of complex mathematical expressions that enable the learner to  develop an intuitive grasp on the material entirely by means of six-dot Braille. This has the advantage that it can be embossed by a conventional printer that produces only six dot cells. Since  our technology can produce continuous output, we are experimenting  with embossing print grouping symbols combined with Nemeth  Braille for the grouped elements to find out whether this makes for  a better tactile display.

Computer Braille is a six dot system which represents letters,  numerals and punctuation (including parenthesis, brackets and braces).  It is most useful for communicating between ASCII based and Braille  based devices without the need for a great deal of translation.  Computer Braille can be used for mathematical formulas but its use  doesn't make it easier or faster to understand them.

A hybrid system is being developed by Prof. John A. Gardner at  Oregon State University under an NSF grant in collaboration with Prof. Norberto Salinas, University of Kansas. This is the Dots+ system  which combines eight dot Braille with tactile display of some of the  graphical elements in technical formulas. With eight dots per Braille  cell. a true one-to-one match could be made between Braille cells and  ASCII's eight bit bytes. This could be even more useful for blind  programmers than computer Braille. If the Dots+ system receives  general acceptance, we plan to incorporate it  in our program. It is important to realize that our graphics are  audio captioned as well as Braille captioned. For now, since we are  only using terse captions Grade 1 Braille is all we need.

c. Voice Presentation  
Until we are able to install more advanced methods of presenting  technical text, we are simply using the time-honored method of  preparing audio cassettes made by a trained reader. We plan to install  this also on CD-ROM or DVD in the form of .wav files. By use of a  large touch screen we should, within a few years, also produce audio  feedback for the tactile graphics using .wav files. This approach will  enable us to present the entirety of our material as a package of tactile  plates and a laser disk.

The application of voice synthesis to read ASCII text files  is a most promising method for the future. In our  multimedia laboratory, we propose to install ASTER, T.V.  Raman's program (3) when it becomes available for use on a PC.  ASTER has great parsing and expressive  capabilities. It provides an auditory rendering of the structure  and content of mathematical formula in ways analogous to a  graphical display. ASTER has excellent hypertext facilities that  permit sophisticated random search for information. These  capabilities exceed those of a trained mathematical reader, as  demonstrated by cassette tapes prepared by Recording for the  Blind and Dyslexic, Princeton, N.J. ASTER reads technical ASCII files  written with the LATEX macro package for the mathematical  typesetting language, TEX. The combination of LATEX and ASTER  has a special advantage: it is possible to use a command set  that expresses the semantic content of a symbol as well as  its typographical form. For example, the symbol (a, b) for  an ordered pair of entities is used in mathematics in many  different contexts and interpretations. If it were used to  represent the coordinates of a point in a plane, say, it  would be possible to use a special LATEX macro that would  cause the symbol to be printed as usual but cause ASTER to  speak, "point a comma b".

As yet, ASTER requires substantial hardware and software  resources that would be available only in an institutional  setting. Until ASTER is installed, mathematical formulas  can be written out in English for synthesis by screen  reading programs. For example, the formula

(a+b) / (c+d)

could be voiced as "fraction a plus b over c plus d",  where the extra spaces are to be read as pauses.

3. Presentation of Student Work

Work executed in LATEX could be printed in typeset form by the  student for submission to the instructor or the LATEX source file  could be viewed in that form on the instructor's screen. With  appropriate software and a Braille printer, the student's work  could be saved in hard copy for future use. Whenever ASTER  becomes available, the student will be able to review his work with  the assistance of ASTER's search facilities. Without ASTER, we  would expect that audible review could be done by standard screen  reading systems "trained" to read the LATEX commands. For  example, "$ \sin x $," would be voiced as, "sine of x."

Conclusion

Notes

2. Available from Humanware, Inc. Loomis, California.

3. Information Technology and Disabilities, v.1, no.4, November 1994.  www.rit.edu/~easi/itd/itdv01n4/article2.html.

Acknowledgements

We are grateful to Julio C. Perez for reviewing the section on Braille  and contributing his knowledge. In addition, we'd like to thank Jen Ciaccio for her hard work in helping to publicize our efforts, and Dr. Bernie Domanski for helping with the web site design and development.

This work was supported by the National Science Foundation,  Experimental Projects in Human Resources and Education,  Grant No. HRD-9450166 (1994), "Multisensory Calculus for Teaching  Students with Visual Impairments," and HRD-9906115 (1999),  "Multisensory Calculus for Visually Impaired People".  

Authors

Initial Web Site Development by Dr. Bernie Domanski. 
Questions, comments and constructive suggestions are welcome and encouraged.
Last Update: Saturday, September 22, 2001