WISEglass: Smart eyeglasses

Symbolbild zum Artikel. Der Link öffnet das Bild in einer großen Anzeige.

WISEglass aims to realise smart eyewear with the perception, performance, and handling of regular eyeglasses. Our approach is based on the observation that most of our senses, vitals, and action involve the head. Smart eyeglasses with regular look thus offer unique and continuous insight into human physiology in the context of individual behaviour and environmental exposure. We pioneered in developing concepts and technical demonstrators for smart eyeglasses, addressing various applications related to health and well-being.

Technically, WISEglass is based on our embedded computing systems family (WISEnode, WISEnxt) which is integrated with sensors and actuators into 3D-printed eyeglasses frames. We intend to maintain the eyeglasses designers creative freedom in shaping the glasses, while maximising sensor and actuator performance through frame personalisation. Our smart eyeglasses can be used for various monitoring and interaction applications due to the diverse set of sensors and actuators that can be integrated.

Key features

  • Multi-modal wearable sensor platform with frame-integrated computing.
  • Stream data via Bluetooth Smart or store up to 512MB locally.
  • Battery runtime of 32+ hours.
  • Modalities: EMG, skull vibration, proximity, RGB light, photoplesmography (PPG), motion, orientation, etc.

Example applications

Activity recognition: WISEglass can be used to distinguish different activities of daily life. In a study with 9 participants we evaluated the recognition performance for 20 different activities which were grouped into 9 activity clusters. Using leave-one-person-out cross-validation, we achieved an average recognition rate of 77%. For more details, please see related publications.

Screen-use detection: Screen-use and its timing are important factors for circadian phase shift. Especially when exposed to at night, the light from e.g. a computer screen can induce a phase delay and hence hamper falling asleep. Using a Gaussian Mixture Model classifier we obtained 80% accuracy in the work presented in Wahl et al. (2017).

Heart beat estimation: Using an optical pulse sensor based on the photoplesmography (PPG) in the WISEglass frame, we can determine heart rate and heart rate variability. In our evaluation study we found the mean root mean square error (RMSE) during stationary activities to be below 9 bpm. Further, we integrated the heart rate into the controls of the popular Pac-Man game.

Wireless game controller: To demonstrate the wide range of WISEglass applications we modified the popular Pac-Man game such that it can be controlled using WISEglass as presented in Wahl et al. (2015). The user controls the moving direction of Pac-Man through head gestures (e.g. shaking head to the left for left) which are sensed by the gyroscope. In addition we integrated a heart rate display into the game so the user can see the effect of playing on the heart rate.

Nutrition/automated dietary monitoring: By integrating Electromyography (EMG) electrodes and skull vibration sensors into WISEglass, we turn eyeglasses into “diet eyeglasses”, which can count chewing cycles, classify food categories in daily intake, and determine intake timing. Our bottom-up eating detection algorithm can retrieve eating events in free-living with an average F1 score of 99.2% and eating timing errors of 2.4 ± 0.4 s and 4.3 ± 0.4 s for the start and end of eating events, respectively.

Videos

https://www.youtube.com/watch?v=Pf2peIBZuak

Selected publications

Rui Zhang, Oliver Amft, "Retrieval and Timing Performance of Chewing-Based Eating Event Detection in Wearable Sensors", Sensors, www.mdpi.com, 2020.

Florian Wahl, "Methods for monitoring the human circadian rhythm in free-living", PhD Thesis, opus4.kobv.de, 2019.

Rui Zhang, Oliver Amft, "Free-living eating event spotting using EMG-monitoring eyeglasses", Proceedings of the 2018 IEEE EMBS International Conference on Biomedical Health Informatics (BHI '18), IEEE, March 2018.

Rui Zhang, Oliver Amft, "Monitoring chewing and eating in free-living using smart eyeglasses", IEEE Journal of Biomedical and Health Informatics, IEEE Xplore, 2018.

Rui Zhang, Volodymyr Kolbin, Mirko Süttenbach, Martin Hedges, Oliver Amft, "Evaluation of 3D-printed Conductive Lines and EMG Electrodes on Smart Eyeglasses Frames", Proceedings of the 2018 ACM International Symposium on Wearable Computers, ACM, 2018.

Florian Wahl, Rui Zhang, Martin Freund, Oliver Amft, "Personalizing 3D-printed smart eyeglasses to augment daily life", IEEE Computer, 2017.

Florian Wahl, Jakob Kasbauer, Oliver Amft, "Computer Screen Use Detection Using Smart Eyeglasses", Frontiers in ICT, Frontiers, 2017.

Rui Zhang, Oliver Amft, "Bite Glasses: Measuring Chewing Using EMG and Bone Vibration in Smart Eyeglasses", Proceedings of the 2016 ACM International Symposium on Wearable Computers (ISWC '16), ACM, September 2016.

Rui Zhang, Oliver Amft, "Regular-look eyeglasses can monitor chewing", Proceedings of the 2016 ACM International Symposium on Wearable Computers (ISWC '16), ACM, September 2016.

Rui Zhang, Severin Bernhart, Oliver Amft, "Diet eyeglasses: Recognising food chewing using EMG and smart eyeglasses", Proceedings of the International Conference on Wearable and Implantable Body Sensor Networks (BSN' 16), IEEE, June 2016.

Oliver Amft, Florian Wahl, Shoya Ishimaru, Kai Kunze, "Making regular eyeglasses smart", IEEE Pervasive Computing, July-September 2015.

Florian Wahl, Martin Freund, Oliver Amft, "WISEglass: Smart eyeglasses recognising context", EAI Endorsed Transactions on Pervasive Health and Technology, eudl.eu, 2015.

Florian Wahl, Martin Freund, Oliver Amft, "WISEglass – Multi-purpose context-aware smart eyeglasses", Proceedings of the 2015 ACM International Symposium on Wearable Computers (ISWC '15), ACM press, 2015.

Florian Wahl, Martin Freund, Oliver Amft, "Using Smart Eyeglasses as a Wearable Game Controller", Adjunct Proceedings of the 2015 ACM Conference on Pervasive and Ubiquitous Computing, ACM, 2015.

Pages

      Contact

      Rui Zhang

      • Job title: Researcher
      • Address:
        Henkestr. 91, Geb. 7
        91052 Erlangen
      • Phone number: +49 9131 85-23604
      • Email: rui.rui.zhang@fau.de

       

      Friedrich-Alexander-Universität Erlangen-Nürnberg