WISEglass: Smart eyeglasses
WISEglass aims to realise smart eyewear with the perception, performance, and handling of regular eyeglasses. Our approach is based on the observation that most of our senses, vitals, and action involve the head. Smart eyeglasses with regular look thus offer unique and continuous insight into human physiology in the context of individual behaviour and environmental exposure. We pioneered in developing concepts and technical demonstrators for smart eyeglasses, addressing various applications related to health and well-being.
Technically, WISEglass is based on our embedded computing systems family (WISEnode, WISEnxt) which is integrated with sensors and actuators into 3D-printed eyeglasses frames. We intend to maintain the eyeglasses designers creative freedom in shaping the glasses, while maximising sensor and actuator performance through frame personalisation. Our smart eyeglasses can be used for various monitoring and interaction applications due to the diverse set of sensors and actuators that can be integrated.
Key features
- Multi-modal wearable sensor platform with frame-integrated computing.
- Stream data via Bluetooth Smart or store up to 512MB locally.
- Battery runtime of 32+ hours.
- Modalities: EMG, skull vibration, proximity, RGB light, photoplesmography (PPG), motion, orientation, etc.
Example applications
Activity recognition: WISEglass can be used to distinguish different activities of daily life. In a study with 9 participants we evaluated the recognition performance for 20 different activities which were grouped into 9 activity clusters. Using leave-one-person-out cross-validation, we achieved an average recognition rate of 77%. For more details, please see related publications.
Screen-use detection: Screen-use and its timing are important factors for circadian phase shift. Especially when exposed to at night, the light from e.g. a computer screen can induce a phase delay and hence hamper falling asleep. Using a Gaussian Mixture Model classifier we obtained 80% accuracy in the work presented in Wahl et al. (2017).
Heart beat estimation: Using an optical pulse sensor based on the photoplesmography (PPG) in the WISEglass frame, we can determine heart rate and heart rate variability. In our evaluation study we found the mean root mean square error (RMSE) during stationary activities to be below 9 bpm. Further, we integrated the heart rate into the controls of the popular Pac-Man game.
Wireless game controller: To demonstrate the wide range of WISEglass applications we modified the popular Pac-Man game such that it can be controlled using WISEglass as presented in Wahl et al. (2015). The user controls the moving direction of Pac-Man through head gestures (e.g. shaking head to the left for left) which are sensed by the gyroscope. In addition we integrated a heart rate display into the game so the user can see the effect of playing on the heart rate.
Nutrition/automated dietary monitoring: By integrating Electromyography (EMG) electrodes and skull vibration sensors into WISEglass, we turn eyeglasses into “diet eyeglasses”, which can count chewing cycles, classify food categories in daily intake, and determine intake timing. Our bottom-up eating detection algorithm can retrieve eating events in free-living with an average F1 score of 99.2% and eating timing errors of 2.4 ± 0.4 s and 4.3 ± 0.4 s for the start and end of eating events, respectively.
Videos
https://www.youtube.com/watch?v=Pf2peIBZuak
Selected publications
"Retrieval and Timing Performance of Chewing-Based Eating Event Detection in Wearable Sensors", Sensors, www.mdpi.com, 2020.
,"Methods for monitoring the human circadian rhythm in free-living", PhD Thesis, opus4.kobv.de, 2019.
,"Free-living eating event spotting using EMG-monitoring eyeglasses", Proceedings of the 2018 IEEE EMBS International Conference on Biomedical Health Informatics (BHI '18), IEEE, March 2018.
,"Monitoring chewing and eating in free-living using smart eyeglasses", IEEE Journal of Biomedical and Health Informatics, IEEE Xplore, 2018.
,"Evaluation of 3D-printed Conductive Lines and EMG Electrodes on Smart Eyeglasses Frames", Proceedings of the 2018 ACM International Symposium on Wearable Computers, ACM, 2018.
,"Personalizing 3D-Printed Smart Eyeglasses to Augment Daily Life", Computer, IEEE Xplore, February 2017.
,"Computer Screen Use Detection Using Smart Eyeglasses", Frontiers in ICT, Frontiers, 2017.
,"Bite Glasses: Measuring Chewing Using EMG and Bone Vibration in Smart Eyeglasses", Proceedings of the 2016 ACM International Symposium on Wearable Computers (ISWC '16), ACM, September 2016.
,"Regular-look eyeglasses can monitor chewing", Proceedings of the 2016 ACM International Symposium on Wearable Computers (ISWC '16), ACM, September 2016.
,"Diet eyeglasses: Recognising food chewing using EMG and smart eyeglasses", Proceedings of the International Conference on Wearable and Implantable Body Sensor Networks (BSN' 16), IEEE, June 2016.
,"Making regular eyeglasses smart", IEEE Pervasive Computing, July-September 2015.
,"WISEglass: Smart eyeglasses recognising context", EAI Endorsed Transactions on Pervasive Health and Technology, eudl.eu, 2015.
,"WISEglass – Multi-purpose context-aware smart eyeglasses", Proceedings of the 2015 ACM International Symposium on Wearable Computers (ISWC '15), ACM press, 2015.
,"Using Smart Eyeglasses as a Wearable Game Controller", Adjunct Proceedings of the 2015 ACM Conference on Pervasive and Ubiquitous Computing, ACM, 2015.
,Pages
Contact
Rui Zhang
- Job title: Researcher
- Address:
Henkestr. 91, Geb. 7
91052 Erlangen - Phone number: +49 9131 85-23604
- Email: rui.rui.zhang@fau.de