WISEglass: Smart eyeglasses

WISEglass aims to realise smart eyewear with the perception, performance, and handling of regular eyeglasses. Our approach is based on the observation that most of our senses, vitals, and action involve the head. Smart eyeglasses with regular look thus offer unique and continuous insight into human physiology in the context of individual behaviour and environmental exposure. We pioneered in developing concepts and technical demonstrators for smart eyeglasses, addressing various applications related to health and well-being.

Technically, WISEglass is based on our embedded computing systems family (WISEnode, WISEnxt) which is integrated with sensors and actuators into 3D-printed eyeglasses frames. We intend to maintain the eyeglasses designers creative freedom in shaping the glasses, while maximising sensor and actuator performance through frame personalisation. Our smart eyeglasses can be used for various monitoring and interaction applications due to the diverse set of sensors and actuators that can be integrated.

Key features

• Multi-modal wearable sensor platform with frame-integrated computing.
• Stream data via Bluetooth Smart or store up to 512MB locally.
• Battery runtime of 32+ hours.
• Modalities: EMG, skull vibration, proximity, RGB light, photoplesmography (PPG), motion, orientation, etc.

Example applications

Activity recognition: WISEglass can be used to distinguish different activities of daily life. In a study with 9 participants we evaluated the recognition performance for 20 different activities which were grouped into 9 activity clusters. Using leave-one-person-out cross-validation, we achieved an average recognition rate of 77%. For more details, please see related publications.

Screen-use detection: Screen-use and its timing are important factors for circadian phase shift. Especially when exposed to at night, the light from e.g. a computer screen can induce a phase delay and hence hamper falling asleep. Using a Gaussian Mixture Model classifier we obtained 80% accuracy in the work presented in Wahl et al. (2017).

Heart beat estimation: Using an optical pulse sensor based on the photoplesmography (PPG) in the WISEglass frame, we can determine heart rate and heart rate variability. In our evaluation study we found the mean root mean square error (RMSE) during stationary activities to be below 9 bpm. Further, we integrated the heart rate into the controls of the popular Pac-Man game.

Wireless game controller: To demonstrate the wide range of WISEglass applications we modified the popular Pac-Man game such that it can be controlled using WISEglass as presented in Wahl et al. (2015). The user controls the moving direction of Pac-Man through head gestures (e.g. shaking head to the left for left) which are sensed by the gyroscope. In addition we integrated a heart rate display into the game so the user can see the effect of playing on the heart rate.

Nutrition/automated dietary monitoring: By integrating Electromyography (EMG) electrodes and skull vibration sensors into WISEglass, we turn eyeglasses into “diet eyeglasses”, which can count chewing cycles, classify food categories in daily intake, and determine intake timing. Our bottom-up eating detection algorithm can retrieve eating events in free-living with an average F1 score of 99.2% and eating timing errors of 2.4 ± 0.4 s and 4.3 ± 0.4 s for the start and end of eating events, respectively.

Videos

https://www.youtube.com/watch?v=Pf2peIBZuak

Selected publications

Contact

Rui Zhang

• Job title: Researcher
• Address:
  
  Henkestr. 91, Geb. 7
  91052 Erlangen
• Phone number: +49 9131 85-23604
• Email: rui.rui.zhang@fau.de

Category: Ubiquitous Technology