Daily Worn Snake-Shaped Multipurpose Robotic Appendages

Robotic appendages have long been envisioned in popular media as a mean to augment users to do more things or have superpowers. For example, villains like Dr. Octopus from Spiderman comics or Screaming Mantis from Metal Gear video games had various types of wearable-robotic arms that they used to fight the protagonists.

Recent advancements in robotics have enabled researchers to prototype various robotic appendages. However, the majority of the prototypes focused on specific tasks or domains, such as physical interactions or industrial and rehabilitation. These robots are often huge, bulky, and difficult to wear. All investigates works presented robots without any considerations of societal impact when these robots are worn publicly.

This project takes the first steps to realize wearable robotic appendages within our daily lives by using an iterative user-centric design approach. Accordingly, we conducted various evaluations and developed a number of prototypes, and the extracted insights from these studies are used as basis to form the main design considerations for designing and implementing multipurpose robotic appendages within the context of our daily life.

Our vision for the future of daily worn robotic appendages.

Vision

We envision an omni-purpose robotic appendage that can be worn and used throughout the day. The illustration shows appendages can be used in different contexts, starting from the morning and at home privately (1-2), public wearability (3), work environment (4-5) and usability in public (6). The appendage is context adaptive by partaking different roles; a smart assistant, an extension of the user’s body, or providing digital interaction capabilities.

Robotic Appendages

 Orochi

Orochi is a serpentine-shaped wearable robot that fulfils our vision of daily worn appendages. its malleable body and unobtrusive design enables it to be worn and used in various daily contexts and tasks. The robot comprises 22-24 degrees of freedom (Dof) with two types of end-effectors. Orochi was used to investigate robotic appendages could be used in our daily lives. Therefore, we conducted a series of workshops to explore the requirements, expectations and usability contexts of these emerging robotic appendages.

 
 

HapticSnakes

The HapticSnakes is a system that comprises two serpentine-shaped appendages main created to explore Haptic feedback. The malleable body of the appendages and the variety of attached end-effectors enable numerous feedback types, which offer a rich feedback medium that is applicable in various systems (e.g. VR, AR ..etc). Accordingly, we provided a design space for creating Haptic feedback using the HapticSnakes. We also discussed the HapticSnakes main advantages and limitations with respect to existing feedback systems.

ARSRL

This project explores using augmented reality (AR) to interact with a wrist-worn serpentine-shaped appendage. The robot consists of 5 Dof, where each is covered with fiducial markers to localize the AR contents. Within the AR environment, the appendage can play three interactive roles: 1) as an extension to the user to carry out physical interactions; 2) as an always available interface to digital contents; 3) as an agent that the user can talk and interact with that is physically embodied by the appendage. AR is used here as the canvas to deliver these roles in one cohesive user experience.

 
 

T2Snaker

This project explores multimodal skill-transfer and training environments using immersive VR and appendages. We developed T2Snaker to explore virtual table-tennis training using both VR and a robot. The robot comprises a serpentine-shaped appendage with 9 DoF and a table-tennis racket as an end-effector. The initial evaluation of T2Snaker focuses its usability and effectiveness as a replacement to standard training sessions for beginners.

Orochi is a snake-like wearable robotic appendage that can be used to interact with various objects. For example, Orochi can be used to interact touch-free with a vending.
ARSR fuses Augmented-Reality and a wearable robot, so users can interact with both digital contents and the physical world. The image shows a chat message localized to the end-effector of the robot. The view is taken through the Augmented Reality Hea
HapticSnakes is a wearable multi-haptic feedback system for VR. It is comprised of an appendage equiped with a rotary end-effector. The robot can be worn on the front or back of the user, where it can provide various types of haptic feedback, includ
A novel interaction of the HapticSnakes is that it can be used to feed the user within VR/AR environments.
ARSRL is a wrist-worn wearable robot that can be controlled from augmented reality through a head-mounted display.
ARSRL can be controlled using AR in a variety of methods. In the image, the robot can be programmed using AR to record and play back various movements which can be used for physical interactions.

 Project Publications

  • Designing Serpentine-Shaped Robotic Appendages for Augmenting Daily Interactions, Mohammed Al-Sada (Doctoral Dissertation, Supervised by Prof. Tatsuo Nakajima, 2019. (Download).

  • Kodai Fuchino, Mohammed Al-Sada, Tamon Miyake, Tatsuo Nakajima. (2022). T2Snaker: a Robotic Coach for Table Tennis. Augmented Human International Conference 2022, Kashiwanoha, Chiba, Japan

    DOI: 10.1145/3519391.3524029 , (Download)

  • Kodai, F., Iskandar, A., Al-Sada, M., & Nakajima, T. Exploring Wearable Robotic Appendages for Daily use. 情報処理学会 第84回全国大会 2022, Matsuyama, Japan

  • Mohammed Al-Sada, Keren Jiang, Shubhankar Ranade, Mohammed Kalkattawi, and Tatsuo Nakajima. 2020. HapticSnakes: multi-haptic feedback wearable robots for immersive virtual reality. Virtual Real. 24, 2 (Jun 2020), 191–209.

    DOI:https://doi.org/10.1007/s10055-019-00404-x , (Download)

  • Mohammed Al-Sada and Tatsuo Nakajima. (2019). Multipurpose Daily Worn Robotic Appendages. 1ST INTERNATIONAL CONFERENCE ON VISUALIZATION AND COMPUTER-HUMAN INTERACTION (2019), Doha, Qatar.

  • Mohammed Al-Sada, Thomas Höglund, Mohamed Khamis, Jaryd Urbani, and Tatsuo Nakajima. 2019. Orochi: Investigating Requirements and Expectations for Multipurpose Daily Used Supernumerary Robotic Limbs. In Proceedings of the 10th Augmented Human International Conference 2019 (AH2019). Association for Computing Machinery, New York, NY, USA, Article 37, 1–9.

    DOI:https://doi.org/10.1145/3311823.3311850 , (Download)

  • Mohammed Al-Sada, "Design Space of Multipurpose Daily Worn Snake-Shaped Robotic Appendages," 2019 IEEE International Conference on Pervasive Computing and Communications Workshops (PerCom Workshops), 2019, pp. 455-456.

    DOI:https://doi.ieeecomputersociety.org/10.1109/PERCOMW.2019.8730745

  • Jiang, K., Piao, X., Al-Sada, M., Höglund, T., Ranade, S., Nakajima, T. (2019). A Robotic Haptic Feedback Device for Immersive Virtual Reality Applications. In: , et al. Ambient Intelligence – Software and Applications –, 9th International Symposium on Ambient Intelligence. ISAmI2018 2018. Advances in Intelligent Systems and Computing, vol 806. Springer, Cham.

    DOI:https://doi.org/10.1007/978-3-030-01746-0_17

  • Mohammed Al-Sada, Keren Jiang, Shubhankar Ranade, Xinlei Piao, Thomas Höglund, and Tatsuo Nakajima. 2018. HapticSerpent: A Wearable Haptic Feedback Robot for VR. In Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems (CHI EA '18). Association for Computing Machinery, New York, NY, USA, Paper LBW624, 1–6. 

    DOI:https://doi.org/10.1145/3170427.3188518

  • J. Urbani, M. Al-Sada, T. Nakajima and T. Höglund, "Exploring Augmented Reality Interaction for Everyday Multipurpose Wearable Robots," 2018 IEEE 24th International Conference on Embedded and Real-Time Computing Systems and Applications (RTCSA), 2018, pp. 209-216,

    DOI:10.1109/RTCSA.2018.00033

    Mohammed Al-Sada, Mohammed Khamis, Akira Kato, Shigeki Sugano, Tatsuo Nakajima and Florian Alt. “Challenges and Opportunities of Supernumerary Robotic Limbs.” (2017). In Extended Abstracts of the 2017 CHI Conference on Human Factors in Computing Systems (CHI EA '17). (Download)

  • Resources

  • To inspire future works in novel robotic appendages, we made all gathered use cases of robotic appendages, our classification and analysis available here: (Use Case Analysis).

Credits

This project would not be successful without the tremendous support from the collaborators, co-authors and user study participants. Thank you very much for all your efforts put into this project.

  • Ms. Xinley Piao, Distributed and Ubiquitous Computing Lab, Waseda University, Tokyo, Japan

  • Ms. Keren Jiang, Distributed and Ubiquitous Computing Lab, Waseda University, Tokyo, Japan

  • Mr. Shubhankar Ranade, Distributed and Ubiquitous Computing Lab, Waseda University, Tokyo, Japan

  • Jaryd Urbani, Distributed and Ubiquitous Computing Lab, Waseda University, Tokyo Japan

  • Mohammed Al-Sada, Distributed and Ubiquitous Computing Lab, Waseda University, Tokyo, Japan. Department of Computer Science and Engineering, Qatar University, Doha, Qatar

  • Mr. Thomas Hoglund, University of Vaasaa, Finland

  • Dr. Tamon Miyake, Sugano Lab, Waseda University, Tokyo, Japan

  • Kodai Fuchino, Distributed and Ubiquitous Computing Lab, Waseda University, Tokyo, Japan.

  • Prof. Mohammed Khamis, University of Glasgow, Scotland

  • Dr. Akira Kato, Waseda University, Tokyo, Japan

  • Prof. Shigeki Sugano, Waseda University, Tokyo, Japan

  • Prof. Floarian Alt, Ludwig Maximilian University of Munich, Germany

  • Prof. Tatsuo Nakajima, Distributed and Ubiquitous Computing Lab, Waseda University, Tokyo, Japan

Next

Cutieroid Project (Hatsuki)