Let your Google Glass know what exactly your eyes are looking at!
What is Gaze Glass?
GlassGaze is an android app designed for Google Glass that makes it possible to do gaze tracking on the Glass. It allows the Google Glass to know where exactly you are looking at. You can interact with the head-mounted display with you eyes. With GlassGaze, your google Glass not only knows where and what point in the display you are looking at but also knows where you are looking in your field view. Now, not only you can take picture or video with your Glass but also you can record your gaze point inside the image and video and share them with your friends. It would be very interesting for your friends to know what you where you were watching in front of you!
GlassGaze is actually a client for the Haytham gaze tracker. Both the Haytham gaze tracker and the GlassGaze client are freeware and open-source. You just need to attach a small camera to the Google Glass frame to capture the eye image. The camera send the image to a remote computer that the Haytham gaze tracker is runnign on it. All the eye tracking processing will be done in the computer and the gaze data will be sent to the Glass through Wifi. The need for a remote computer for the main processing in our first prototype is due to the low processing power and limited battery life of the current Google Glasses.
GlassGaze actually provides a basic framework for experiencing the eye and gaze tracking on the Google Glass. The gaze data streamed by the Haytham gaze tracker will be received by the GlassGaze app. The app facilitate the user to calibrate the system for both gaze tracking while watching the display and while viewing the scene. Besides the demo apps inside the application that show the main features of the app, as an developer you have access to the gaze tracking data from other applications running on the Glass!
What Does It Do?
- Gaze estimation inside the head-mounted display that can for example be used for interaction with the Glass
- Gaze estimation inside the image captured by the front view camera of the device.
How to Use GlassGaze?
Figure below shows the main components of the system. Following on this page, is more detailed information you need to know about setting up the system.
Components of the GlassGaze system.
Software
In terms of software you basically just need to download and install the Haytham_server (currently only available for windows) on your computer, and download the GlassGaze apk version and install it on your Glass. Both computer and Glass need to be in the same local network. All the communications are handled through TCP sockets.
Haytham_server
Installing the Haytham_server on the windows is very straightforward. Once you have installed the software on your machine launch the application and select the Google Glass mode from the first window that pops up. After you see the main window of the Haytham, you need to select the eye camera in the camera tab and start the capturing. You can then adjust the pupil tracking to get a relatively robust pupil detection in the image.
GlassGaze app
Download the apk version of the GlassGaze app and install it on your Glass using adb (steps are described below). Alternatively, if you are familiar with android programming, get the source code from GitHub and compile the code to your Glass using Android studio.
Installing the GlassGaze.apk on Google Glass. An easy way of doing this (for beginners) has been described in the following steps:
- Connect your Google Glass to the usb port of your computer;
- Download the latest usb driver for Google Glass from here;
- Extract the folder. Before installing the driver open the android_winusb.inf which is inside the folder because you probably need to modify this file as follows:
- Go to device manager and copy the VID and PID of you Glass: (Control Panel/System/Device Manager/Portable Devices/Glass 1/Properties/Details/Hardware Ids).
- Look inside the android_winusb.inf file and see if you can find similar lines as below. The VID and PID values should match what you have for your Glass. If it does not match copy corrected version of these two lines to these two sections: [Google.NTx86] and [Google.NTamd64];
- ;Google Glass
- %SingleAdbInterface% = USB_Install, USB\VID_18D1&PID_4E11&REV_0216
- %CompositeAdbInterface% = USB_Install, USB\VID_18D1&PID_4E11&MI_01
- Make sure that Driver Signature Enforcement has been disabled in Windows;
- Install the modified version of the driver for you Glass. (Control Panel/System/Device Manager/Other Devices/Glass 1/ Update Driver Software/Browse my computer for driver software/);
- Install the Android Debug Bridge (ADB) on your computer. (There is a good guide in this page)
- Open CMD (Command Prompt) or Terminal (for Macs) and type adb.exe devices;
- You should see the Google Glass device listed. If it is listed as an unauthorized device make sure that you have Debug Mode Enabled on your Glass;
- Navigate the command prompt to where you have the file GlassGaze.apk;
- Install the GlassGaze by typing adb install GlassGaze.apk
Hardware
- Google Glass;
- A night-vision camera (you can just remove the infrared filter of a regular webcam. It usually sits behind the lens);
- An infrared LED; and
- A windows machine!
You can use any kind of camera as long as it’s night vision, as shown in the figures below. You need to attach the camera to the frame of the Glass. Figure below shows a wired version using Genius "islim 321R" webcam attached to the Glass and also a wireless version which is more convenient.
We have tried a wireless camera which allows us to make a fully mobile and wireless system. You can get this camera from a Chinese company called 3RD Eye Electronics. You just send an email to sales8@3rdee.com and order the camera model "EyeITU_SetB". The camera comes with a rechargeable battery, an adapter and a USB receiver. In the video bellow we show an easy way of attaching this camera to the Glass using a piece of aluminum sculpting wire and some shrink tubes.
*NOTE: You don’t actually get a clean image from this wireless cameras and the image sometimes gets very noisy, however, this is a very cheap and small camera that works and it can easily be mounted on a Google Glass.
Questions
Please post your questions to the discussion forum of the Haytham project: