Human-Robot Interaction Through Extended Reality: Sustainable Computing and Optimization

Rajashekhar Vachiravelu Saminathan is a mechanical engineer, robotics researcher, and inventor pursuing his PhD at the HIVE Lab in the Department of Design at the Indian Institute of Technology Kanpur, Uttar Pradesh, India. He has published more than 50 peer-reviewed papers and articles in journals, conferences, and magazines and holds Indian and American patents. He conducts research in the areas of mechanism design for robots, physical AI, extended reality, and human-robot interaction.

Gowdham Prabhakar, PhD is an Assistant Professor in the Department of Design at the Indian Institute of Technology Kanpur, Uttar Pradesh, India and the founder of HIVE Lab. A creative technologist, his research spans human-computer interaction, music technology, and extended reality, with a focus on multimodal and multisensory interaction. At HIVE Lab, his group develops tangible musical interfaces, XR systems, soft robots, generative AI artworks, and assistive communication technologies.

List of Figures xix
List of Tables xxvii
Preface xxix
Acknowledgments xxxi

Part I: Foundations of Human-Robot Interaction and Extended Reality 1
1 Human-Robot Interaction 3
2 Humans 19
3 Robotics 49
4 Interactions 67
5 Extended Reality 87
6 XR-Enabled Human-Robot Interaction Systems 101

Part II: Evolution and Vision for HRI through XR 113
7 The Evolution of Human-Robot Interaction 115
8 Understanding Extended Reality (XR) as a Medium for Robotics 127

Part III: Mathematical Foundations 137
9 Role of Mathematics in Human-Robot Interaction through Extended Reality 139
10 Spatial Representation and Coordinate Transformations 153
11 Kinematic Modeling and Motion Mapping 163
12 Dynamics and Physical Interaction Modeling 173
13 Control Theory and Shared Autonomy 185
14 Probabilistic Modeling and Decision-Making 195
15 Geometric, Algebraic, and Learning-Based Methods 205

Part IV: Designing XR-Robot Systems 217
16 Principles of Designing XR-Enabled Human-Robot Systems 219
17 Tools and Technologies for XR-HRI Development 233
18 The Extended Reality Pipeline Design 247

Part V: Human-Robot Interaction through Extended Reality Systems 257
19 Human-Robot Interaction through Virtual Reality 259
20 Human-Robot Interaction through Augmented Reality 267
21 Human-Robot Interaction through Diminished Reality 273
22 Human-Robot Interaction through Augmented Virtuality 281
23 Human-Robot Interaction through Mixed Reality 287

Part VI: The Dual-Robot Case Study 293
24 Conceptualizing a Dual-Robot Framework 295
25 Building the XR Environment for Dual Control 307
26 Evaluating the XR-HRI System 321

Part VII: Future Research Directions in Human-Robot Interaction through Extended Reality 329
27 Future Research Directions in XR Hardware for Human-Robot Interaction 331
28 Future Research Directions in XR Software for Human-Robot Interaction 345

Part VIII: Broader Perspectives and Future Landscapes 357
29 AI in HRI through XR 359
30 Challenges in Real-World Deployment 373
31 Applications across Domains 381
32 Ethics, Empathy, and Human Factors in XR-HRI 395
33 Future Horizons-Toward Symbiotic XR-Robot Ecosystems 407

References 417
Index 457