Welcome to the MI Lab

We focus on fundamental and applied research on the future of interactive
technology, emphasizing haptics in VR. We create innovative multisensory
user interfaces and interaction techniques for immersive computing to
enhance user experiences and realism.

News

  • (March 21, 2024) Dr. Kim will serve on the program committee for UIST 2024.
  • (March 21, 2024) Dr. Kim will serve on the technical program committee for Ubicomp/ISWC 2024.
  • (February 3, 2024) Dr. Kim will serve on the program committee for EuroHaptics 2024.
  • (January 27, 2024) One poster paper has been accepted to IEEE VR 2024! Another great news!
  • (January 21, 2024) Two demo papers have been accepted to IEEE VR 2024! Great news!
  • (January 19, 2024) Our submission to ACM CHI 2024 has been conditionally accepted! Congratulations, Haokun, Yatharth, and Hyunjae!
  • (December 15, 2023) We received the Best Student Demo Award at the SIGGRAPH Asia Emerging Technologies held in Sydney, Australia! Congrats, Yatharth and Haokun! 🏆
  • (October 17, 2023) Haokun presented “Fabric Thermal Display using Ultrasound Waves” at IEEE ISMAR 2023 in Sydney, Australia!
  • (September 29, 2023) Dr. Kim gave a talk at the University of Chicago (Pedro Lopes’ Lab).
  • (September 6, 2023) Dr. Kim will serve as a WIP Paper Co-Chair for the IEEE World Haptics Conference 2025.
  • (August 9, 2023) Our submission to IEEE ISMAR 2023 has been accepted! Congratulations, Haokun and Yatharth!
  • (August 9, 2023) Dr. Kim will serve as an associate editor of Frontiers in Virtual Reality (Haptics Section).
  • (July 31, 2023) Our submission to SIGGRAPH Asia 2023 Emerging Technologies has been conditionally accepted! Congratulations, Yatharth and Haokun!
  • (July 20, 2023) Dr. Kim will serve as a program committee member for the IEEE Haptics Symposium 2024.
  • (July 19, 2023) Dr. Kim gave a talk on Multimodal Interfaces for Immersive Virtual Reality at CEA, Palaiseau, France.
  • (July 18, 2023) Dr. Kim gave a talk on Multimodal Interfaces for Immersive Virtual Reality at CNRS/INRIA, Rennes, France.
  • (July 10, 2023) Dr. Kim presented “Interplaying Thermal and Tactile Sensations for Immersive Virtual Reality” at the WHC 2023 Workshop on Touching the Future: Science and Technology of Multisensory Cutaneous Displays” in Delft, the Netherlands. The workshop was organized by Lynette Jones and Hsin-Ni Ho, and Dr Kim joined as a panelist, along with Pedro Lopes, Allison Okamura, and Yasemin Vardar.
  • (July 4, 2023) Dr. Kim gave a talk on Multimodal Interfaces for Immersive Virtual Reality at the BK21 Workshop in Cheonan, Korea.
  • (March 30, 2023) Our paper “Recent Advances and Opportunities of Active Materials for Haptic Technologies in Virtual and Augmented Reality” has been selected as one of the top downloaded articles during its first 12 months of publication in Advanced Functional Materials. 🏆
  • (March 29, 2023) Our paper “Upper Body Thermal Referral and Tactile Masking for Localized Feedback” has been nominated for the Best Paper at IEEE VR 2023. 🏆
  • (January 30, 2023) Dr. Kim will serve as a program committee member for the IEEE World Haptics Conference 2023.

Open Position

We are always looking for new team members who bring a diverse set of skills and motivations. We offer positions for research assistants, independent study projects, internships, and postdocs. If you are interested in joining the group, please get in touch by sending an email to jin dot kim at utdallas.edu. Research areas include:

  • Human-Computer Interaction (UI/UX, interaction techniques, texture entry, gestures, mobile, wearables)
  • Tactile and Haptic Interaction (haptics interfaces, mid-air haptics, thermal, multimodal interaction)
  • Virtual Reality (Metaverse, 3D user interfaces, interactive computing)

Projects

Fabric Thermal Display using Ultrasonic Waves

This paper presents a fabric-based thermal display of a polyester fabric material combined with thermally-conductive materials using an ultrasound haptic display. We first empirically test the thermal generation process in five fabric materials by applying 40 kHz ultrasonic waves to the fabric materials. We also examine their thermal characteristics by applying different frequencies and amplitudes of ultrasonic cues. We show that polyester demonstrates the best thermal performance. We then combine it with thermally-conductive materials, including copper and aluminum, and compare them with the fabric-only condition. We integrate polyester with aluminum into a glove to explore the use cases in VR and share our findings, insights, limitations, and future works. [Paper | Video]


Upper Body Thermal Referral and Tactile Masking for Localized Feedback

This paper investigates the effects of thermal referral and tactile masking illusions to achieve localized thermal feedback on the upper body. We use a 2D array of sixteen vibrotactile actuators (4×4) with four thermal actuators to explore the thermal distribution on the user’s back. A combination of thermal and tactile sensations is delivered to establish the distributions of thermal referral illusions with different numbers of vibrotactile cues. The result confirms that localized thermal feedback can be achieved through cross-modal thermo-tactile interaction on the user’s back of the body. [Paper | Video]


MetaTwin: Synchronizing Physical and Virtual Spaces for Seamless World

MetaTwin is a collaborative Metaverse platform that supports one-to-one spatiotemporal synchrony between physical and virtual spaces. The users can interact with other users and surrounding IoT devices without being tied to physical spaces. Resource sharing is implemented to allow users to share media, including presentation slides and music. We deploy MetaTwin in two different network environments (i.e., within the US and Korea-US international) and summarize users’ feedback about the experience. [Paper | Video 1 | Video 2]


FIRE: Mid-Air Thermo-Tactile Display in VR

We design a proof-of-concept thermo-tactile feedback system with an open-top chamber, heat modules, and an ultrasound display. Our approach is to provide heated airflow along the path to the focused pressure point created from the ultrasound display to generate thermal and vibrotactile cues in mid-air simultaneously. [Paper | Video]


TangibleData: Interactive Data Visualization with Mid-Air Haptics

We build an interactive 3D data visualization tool that adapts hand gestures and mid-air haptics to provide tangible interaction in VR using ultrasound haptic feedback on 3Ddata visualization. We consider two types of 3D visualization datasets and provide different data encoding methods for haptic representations. [Paper 1 | Paper 2 | Video 1 | Video 2]


In-Air Text Input Technique

We empirically explore fundamental requirements for achieving VR in-air typing by observing the unconstrained eyes-free in-air typing of touch typists. We examine properties of finger kinematics, correlated movement of fingers, interrelation in consecutive key-strokes, and 3D distribution of key-stroke movements. We further test finger kinematic features, including 3D position, velocity, acceleration, and temporal features, including previous fingers and keys. Based on this analysis, we assess the performance of various classifiers, including Naive Bayes, Random Forest, Support Vector Machines, and Deep Neural Networks, in terms of the accuracy for correctly classifying the keystroke. [Paper 1 | Paper 2 | Video]


Refinity: An Interactive Holographic Signage for New Retail Shopping Experience

Refinity is an interactive holographic signage for the new retail shopping experience. In this project, we show a concept of futuristic shopping experience with a tangible 3D mid-air interface that allows customers to directly select and explore realistic virtual products using autostereoscopic 3D display combined with mid-air haptics and finger tracker. We also present an example of in-store shopping scenario for natural interactions with 3D. This shopping experience will engage users in producing a memorable in-store experience with the merging of digital and physical interactions. [Paper | Video]


RealWalk: Haptic Shoes Using Actuated MR Fluid for Walking in VR

RealWalk is a pair of haptic shoes for HMD-based VR, designed to create realistic sensations of ground surface deformation and texture through MR fluid actuators. RealWalk offers a novel interaction scheme through the physical interaction between the shoes and the ground surfaces while walking in VR. Each shoe consists of two MR fluid actuators, an insole pressure sensor, and a foot position tracker. When a user steps on the ground with the shoes, the two MR fluid actuators are depressed, creating a variety of ground material deformation such as snow, mud, and dry sand by changing its viscosity. We build an interactive VR application and compare RealWalk with vibrotactile-based haptic shoes to investigate its effectiveness. [Paper 1 | Paper 2 | Paper 3 | Video]


Touch3D: Touchscreen Interaction on Multiscopic 3D with Electrovibration Haptics

Touch3D is an interactive mobile platform that provides realistic viewing and touching experiences through glasses-free 3D visualization with electrovibration. Touch3D is designed to take advantage of both visual and tactile illusions to maximize multimodal experience in touchscreen interaction. We seamlessly integrate two technologies: Automultiscopic 3D Display and Electrovibration Display; and weave both hardware and software into one fluid interface. Our museum application using Touch3D demonstrates important implications for the improvement of 3D perception in both visual and tactile modalities for enhanced touchscreen interaction. [Paper | Video]


AirPiano: Enhancing Music Playing Experience in Virtual Reality with Mid-Air Haptic Feedback

AirPiano is a music playing system to provide touchable experiences in HMD-based virtual reality with mid-air haptic feedback. AirPiano allows users to enjoy enriched virtual piano-playing experiences with touchable keys in the air. We implement two haptic rendering schemes to mimic the resisting force of piano keys using ultrasonic vibrations. Constant Feedback is designed to provide short and intense feedback whereas Adaptive Feedback is designed to follow the changes in feedback from a real keypress. [Paper | Video]


Interactive 3D Painting with Haptic Brush in Immersive Room

We propose an interactive artwork system with automultiscopic 3D and haptic paint brush in an immersive room. Our system consists of a 81-view automultiscopic display, a handheld haptic paint brush, and a large-scale color palette station in a CAVE-like cubic room filled with the visual scene of the artwork. The 81-view rendering and multiplexing technology is applied by setting up the virtual cameras in the off-axis layout. The haptic paint brush is designed and implemented using a 2D array of multiple piezoelectric actuators. It provides the tactile feedback of spatial distance information between a virtual brush and a distal 3D object displayed on the automultiscopic display for the precise control of the brush when it is interacting with automultiscopic 3D. [Paper]

en_USEnglish
Scroll to Top