Interactive Visual Supports for Children with Autism

“Visual schedules and choice boards are tools used in current best practices for helping children with autism and other special needs. These non-verbal kids need help communicating their choices, understanding time and activities, and so on. We are working with Windows SmartPhone, Linux small displays, and large touchscreen-enabled platforms to develop solutions that ease these burdens, provide logging and visualizations of data, and help automate some of the features of using these communication techniques.” – read more from the official STAR project page.

vSked is a system that was designed and developed by STAR for use in autistic classrooms. At a basic level, it consists of a Java application running on a laptop that serves as the “host” to the classroom. This laptop is connected to an HDTV, and the TV has a USB touch panel attached to it that is connected to the laptop providing input. With this configuration, any touch on the HDTV served as a mouse click to the laptop, and effectively turned the HDTV into a large touchscreen display. The TV is placed at front of the classroom so that all of the students can see it. vSked development was started before the explosion of the mobile market, so although it seems archaic now given the prevalence of tablets and smartphones, each student has their own tablet-like Windows UMPC running Windows XP. The client version of vSked is a Java application running on each of the UMPCs. Each UMPC is able to communicate to the host laptop over LAN, as they are all connected to the same router via wi-fi.

vSked was a tool designed to help the children stay on task, be productive, focused, and reinforce success. Given the nature of the system being focused on the TV at the center of the classroom, everyone was drawn together. The system outlined the day’s schedule, showing the current activity on the TV screen at a time. Students then would complete the activity on independently. Sometimes tasks were virtual and completed on the UMPC, like choosing the correct weather on screen, and sometimes completed in the physical world, like exercising. When a student finishes an activity they are rewarded by a random congratulatory animation and the host TV would reflect those students completed status.  Now, the teacher would know who was done, and also who possibly needed help.

I was asked to work on this project in the Winter and Spring quarters of 2011 – well after it had been developed and used in classrooms. In the first run of its use, initial feedback was very positive. The teacher of the autistic classroom that we were working with requested a couple of enhancements to the project, and that is what I was brought in to help achieve. She wanted to figure out a way for the students to be rewarded with more than just the animation shown on the UMPC. What we came up with was a “market” where students can purchase “rewards” for good behavior and correct work. I assisted in adding this virtual market to vSked. We added functionality to the host (teacher) UI on the TV to where she could select a student and award (and remove) “money” in $1 increments. Each student could view their “bank” on their own device and browse for rewards to purchase, like snacks, free time, stuffed animals, etc. All of the items that students were allowed to purchase were customize able from a web interface per student by the teacher. All of this combined made for a very rewarding experience for the students, and it made the teacher’s life easier by having a digital system to help manage the activities in the classroom all while helping the students stay engaged.

My favorite part about working on vSked was that it was my first time developing a real “solution” based on requests from a customer. It was very fun and interesting to hear specific feature requests by the teacher, and to also get her feedback on UI changes and other usability concerns. I found myself very dedicated to making a system that did what she asked, but that also was easy to use and robust.

Below are some photos from my day at the school once we launched the newest version of vSked complete with the new digital store for the students.