Voice Control and Eye Tracking First Impressions

In my previous post I talked about ergonomics and hands. In this post I'll focus on actually not using hands at all! There's a lot of content online such as this video and this podcast which demonstrate things like this. Those links actually inspired me to try out hands free computer use. In fact this whole blog post was made by voice control and eye tracking.

There are a lot of moving parts with voice control and eye tracking, which makes them really interesting. I hope to do a future post on how it actually works! This post will focus just on first impressions.

Voice Control

Finding Talon Voice, Windows and Android

From the first moment I used talon voice control I had confidence in it. It's the one that's used in the above links, and was made for programmers. I did the tutorials and they really are the things that you need to get up and running.

But months before I did that, I used the windows operating system voice control and I just didn't find it powerful enough for example for selection of text. It left me thinking that voice control isn't that powerful and that I didn't really have the time to try several voice control programs. I sure am glad I tried talon voice.

Note that voice control is different to dictation. Voice control allows you to control the operating system, and includes dictation as part of that. Dictation converts words that are spoken into text.

On the note of trying out different voice control programs, I did also try the android voice control on my mobile phone at the same time as I was trying out the windows voice control. I found the android one to be really good actually.

Comparisons

It's interesting how each voice control software is almost like its own language - for example the windows and android ones will allow you to click a word just by reading it out.

Each also has its own version of

• a mouse grid which allows you to click into a labelled grid.
• showing labels which allows you to click onto the labelled things on the screen.

Limitations

There are not many limitations to this - as per the above video one of the users simply casts her computer to her 4k TV, and then does voice control. How cool is that! But if I had to list some limitations at this early stage, there are just the two below.

Sometimes focusing on an area either in a window or in a browser can be difficult by just using tab, so that is limitation, but it can be overcome either by macros for voice or by eye control or other things.

Another limitation I found was not being able to find what you need on google with a few search words. You'll be able to find these eventually by digging into the code but it would be nice if they came up on google.

Minor limitations I found perhaps particularly with my voice, which can be overcome simply by using different words, is that the word enter is misunderstood for "end ten" a lot! I find that if I speak clearly, the recognition is much better. Having said that the recognition is amazing - much better than I thought it would be.

Reflections on Voice Control

It's interesting that you begin to take voice control for granted, for example it becomes strange that the microwave is not voice controlled!

It's interesting how quickly you pick up the alphabet, as there is a word for each letter in the talon alphabet, and this pretty much has to be memorized.

In fact, a lot of things have to be memorized and remembered - so I experienced spending time sometime just thinking about what command to use. When I switched back to using the keyboard for a very short while, it was clear that I almost had like more time for thinking. But I think that as you use voice control more, that thinking time becomes much much less. It could be compared to changing where each key on a keyboard is and then asking someone to use the keyboard - it would take them a while to find each key!

This also got me thinking about concurrency in computer science, and how the mind just concurrently also needs to process stuff!

Eye Tracking

While the voice control software talon voice is free, hardware for eye tracking is not. The cost maybe even put me off eye tracking for a bit. But the ability to focus into different parts of the screen really appealed to me, to complement the voice control and not just to use eye tracking independently.

Finding an Eye Tracker

Another thing that put me off is the approved list of eye trackers for windows. This list is really short, which is a shame because in this day and age with such cheap technology you would think that anything is possible.

It's when I realized that talon voice could work with a non approved eye tracker that I got really interested in it. So I got the tobii 5 eye tracker. There is a small hack that you do have to do, which is to end any tobii services in the windows task manager.

Reflections on Eye Tracking

Impressions so far have actually been really good! I say actually, because I was ready for disappointment which would reflect the disappointment with the lack of products in the market.

I don't think I could use just eye control, it's definitely something that complements the voice control for me, but is a really big advantage. When I've tried to do everything using eye control I find it too draining both mentally and on the eyes. On the other hand I think I could do everything with voice control, but the learning curve is much steeper. Having both definitely listens the learning curve, in fact I would say with both there is not much of a learning curve.

I will say that say in the first week it was annoying that a cursor was following where I was looking, and I thought this annoyance would never go away, but it literally went away in like the second week of use. I actually expect a cursor where I look now, for example on the TV!

I also think that in the first week I really struggled to get the cursor where I wanted. In the second week it was way easier, and I was convinced that something had changed like the calibration even though I did not recalibrate. It must just be me getting used to using the eye tracker!

One limitation which is a bit annoying, but can be overcome by turning eye tracking on and off, is when you want to hover over something but look at something else that is close by. The thing you are hovering over might change, not allowing you to look at what you want to look at!

Using the computer used to cause pain, but now it is the main time to heal as hours are spent at the computer, healing! It's a good and strange feeling after having associated a keyboard and mouse with pain.