Demo
View Flash Demo (Flash Player 8+ Required)
My idea to help Sensory Substitution to evolve, expand and achieve wider usage in the future, is to develop a functional piece of software which will bring several different technologies together, to create a more fulfilling Sensory Substitution experience.
The software is called SWIC software- this stands for Sensory Substitution, Wayfinder Internet Community software.
The software will be available for use on mobile phones as well as PC’s. Both versions will be the same and have the same functions, the only difference being that they will be tailored for the difference in screen sizes.
The working software will be able to be voice activated, or, if used with a mouse or manual pointing device such as a tablet pen, the buttons will explain their functions when rolled-over. These navigational options are essential for blind users of the software.
To begin with I’ve aimed the software at blind users, but the software could be expanded to be used with other Sensory Substitution technologies. Sensory Substitution for the deaf is already being developed using a plate of electrodes to stimulate the skin corresponding to the vibrations made by sounds.
My hope is that companies who are developing sensory substitution devices, wayfinder devices for the blind and also obstacle avoidance devices will make their hardware compatible with the software.
These companies could use the Wiki on my website to inform people of emerging technologies which they are developing and even contact others to collaborate on projects. They could also provide links to driver updates for their existing hardware products which would already be in use with the SWIC software.
How does it work?
I have created a rough initial design for the “look and feel “ of the SWIC software including some of the essential navigational elements which I would like to be included in the final working design.
Initially the users are presented with a screen, allowing them to select an area to enter into and control.
Please click here to view the demo, to get an idea of the “look and feel” of the software.
The 3 main components inside the SWIC software package:
Sensory Substitution
This section of the software allows users to control their Sensory Substitution devices. At the moment the main input device is a camera. The image from this camera can be clearly seen in the centre of the screen. The users can start, stop and control the volume and settings of their Sensory Substitution device.
The output of the interpretation of the camera image could be sounds, using the vOICe Software or it could output to a device such as the Brainport which stimulates a series of electrodes placed on the tongue. All settings for the hardware can be controlled through the software.
The camera image itself can also be controlled using the software. The GUI can be rendered, allowing users to experience web graphics displayed on their screen. Images can also be opened and experienced through Sensory Substitution.
Wayfinder
This section of the software uses new GPS technology which is accurate to within 5 meters to allow users to accurately plan routes, get directions and find information about their current location. They can even look up real-time weather forecasts for specific areas.
Users can also download and upload audio maps to and from their groups (controlled within the community section of the software). Favourite locations can be stored and retrieved easily and quickly.
Directions sought will be spoken, similar to GPS car navigation kits; which are currently on the market.
Community
The community part of the software is heavily dependent upon the development of wireless mesh networks which are currently emerging and will soon cover most developed areas. When “out and about” the user can connect through the network to the SWIC community and search for friends in their area. They can also manage and chat to their friends who are also using the SWIC software. Uploading and sharing experiences through this software could also be a way to improve the development of Sensory Substitution Technologies.
Users can easily navigate from one area to another, switching between applications and functions with relative ease. Many more features could be added to this software so I’ve created this demo as a starting block for ideas to be generated.
Please visit the Wiki area linking from this site to comment and make suggestions on the way the software could be further developed or extra features which could be added.
You can also contact me to express your thoughts or suggestions. Thank you for visiting sensorysubstitution.co.uk.