Welcome to the MI Lab

We focus on fundamental and applied research on the future of interactive
technology, emphasizing haptics in VR. We create innovative multisensory
user interfaces and interaction techniques for immersive computing to
enhance user experiences and realism.


  • (July 16, 2024) One paper is conditionally accepted to IEEE ISMAR 2024! Congratulations, Yatharth and Daniel!
  • (July 12, 2024) Received an NSF grant “CIRC: Dev: SMILE: Scan to Multi-sensorial Interactive Learning Environment” (Award CNS-2346528, 07/2024-06/2026).
  • (June 30, 2024) Two papers are conditionally accepted to ACM UIST 2024! Congratulations, Yatharth, Haokun, Hyunjae, and Daniel!
  • (May 1, 2024) Two Emerging Technologies Installation submissions have been accepted to present at SIGGRAPH 2024 in Denver, Colorado! Great work, Yatharth, Haokun, and Ayush!
  • (April 16, 2024) One paper was accepted to ACM IMWUT. Congratulations, Haokun and Yatharth!
  • (April 11, 2024) We won the Best Demo Honorable Mention Award at the IEEE Haptics Symposium 2024. Congratulations, Haokun and Yatharth! 🏆
  • (March 21, 2024) Dr. Kim will serve on the program committee for UIST 2024.
  • (March 21, 2024) Dr. Kim will serve on the technical program committee for Ubicomp/ISWC 2024.
  • (February 3, 2024) Dr. Kim will serve on the program committee for EuroHaptics 2024.
  • (January 27, 2024) One poster paper has been accepted to IEEE VR 2024! Another great news!
  • (January 21, 2024) Two demo papers have been accepted to IEEE VR 2024! Great news!
  • (January 19, 2024) Our submission to ACM CHI 2024 has been accepted! Congratulations, Haokun, Yatharth, and Hyunjae!
  • (December 15, 2023) We received the Best Student Demo Award at the SIGGRAPH Asia Emerging Technologies held in Sydney, Australia! Congrats, Yatharth and Haokun! 🏆
  • (October 17, 2023) Haokun presented “Fabric Thermal Display using Ultrasound Waves” at IEEE ISMAR 2023 in Sydney, Australia!
  • (September 29, 2023) Dr. Kim gave a talk at the University of Chicago (Pedro Lopes’ Lab).
  • (September 6, 2023) Dr. Kim will serve as a WIP Paper Co-Chair for the IEEE World Haptics Conference 2025.
  • (August 9, 2023) Our submission to IEEE ISMAR 2023 has been accepted! Congratulations, Haokun and Yatharth!
  • (August 9, 2023) Dr. Kim will serve as an associate editor of Frontiers in Virtual Reality (Haptics Section).
  • (July 31, 2023) Our submission to SIGGRAPH Asia 2023 Emerging Technologies has been conditionally accepted! Congratulations, Yatharth and Haokun!
  • (July 20, 2023) Dr. Kim will serve as a program committee member for the IEEE Haptics Symposium 2024.
  • (July 19, 2023) Dr. Kim gave a talk on Multimodal Interfaces for Immersive Virtual Reality at CEA, Palaiseau, France.
  • (July 18, 2023) Dr. Kim gave a talk on Multimodal Interfaces for Immersive Virtual Reality at CNRS/INRIA, Rennes, France.
  • (July 10, 2023) Dr. Kim presented “Interplaying Thermal and Tactile Sensations for Immersive Virtual Reality” at the WHC 2023 Workshop on Touching the Future: Science and Technology of Multisensory Cutaneous Displays” in Delft, the Netherlands. The workshop was organized by Lynette Jones and Hsin-Ni Ho, and Dr Kim joined as a panelist, along with Pedro Lopes, Allison Okamura, and Yasemin Vardar.
  • (July 4, 2023) Dr. Kim gave a talk on Multimodal Interfaces for Immersive Virtual Reality at the BK21 Workshop in Cheonan, Korea.
  • (March 30, 2023) Our paper “Recent Advances and Opportunities of Active Materials for Haptic Technologies in Virtual and Augmented Reality” has been selected as one of the top downloaded articles during its first 12 months of publication in Advanced Functional Materials. 🏆
  • (March 29, 2023) Our paper “Upper Body Thermal Referral and Tactile Masking for Localized Feedback” has been nominated for the Best Paper at IEEE VR 2023. 🏆
  • (January 30, 2023) Dr. Kim will serve as a program committee member for the IEEE World Haptics Conference 2023.
  • (December 19, 2022) Our paper “Upper Body Thermal Referral and Tactile Masking for Localized Feedback” has been conditionally accepted for presentation at IEEE VR 2023 and for publication in a special issue of the IEEE Transactions on Visualization and Computer Graphics (IEEE TVCG)! Congrats, Haokun and Yatharth!
  • (December 5, 2022) Dr. Kim will serve as a Guest Editor for IEEE Transactions on Haptics (ToH to WHC track).
  • (December 1, 2022) We received the Best Poster Award from VRST 2022! Congrats, Ayush! 🏆
  • (October 4, 2022) The book chapter entitled “Multimodal Interaction with Mid-Air Haptics” by Dr. Kim was published. This chapter is one of the contributions in Ultrasound Mid-Air Haptics for Touchless Interfaces published by Springer.
  • (September 6, 2022) Two poster papers were accepted to VRST 2022! Congratulations, Ayush, Henry, and Brandon!
  • (June 20, 2022) Dr. Jin Ryong Kim received an Outstanding Associate Editor Award from the IEEE Robotics and Automation Letters for his distinguished service (https://www.ieee-ras.org/publications/ra-l/ra-l-distinguished-service-awards)🏆

Open Position

We are always looking for new team members who bring a diverse set of skills and motivations. We offer positions for research assistants, independent study projects, internships, and postdocs. If you are interested in joining the group, please get in touch by sending an email to jin dot kim at utdallas.edu. Research areas include:

  • Human-Computer Interaction (UI/UX, interaction techniques, texture entry, gestures, mobile, wearables)
  • Tactile and Haptic Interaction (haptics interfaces, mid-air haptics, thermal, multimodal interaction)
  • Virtual Reality (Metaverse, 3D user interfaces, interactive computing)


Thermal Masking: When the Illusion Takes Over the Real (CHI2024)

This paper reports on a thermal illusion called thermal masking. Thermal masking is a phenomenon induced by thermal referral to completely mask the original thermal sensation, providing thermal sensation only at the tactile site. Three experiments are conducted using thermal and vibrotactile actuators to investigate the nature of thermal masking. The results show a higher percentage of thermal masking occurs in warm than hot or cold conditions. The second experiment examines how far the thermal masking can be perceived. The results show that masking can reach up to 24 cm from the thermal site. The third experiment explores the interaction space by placing the tactile actuators on the opposite side of the thermal actuator. The results confirm that thermal masking can reach the other side of the arm, and the performance was higher in warm conditions. [Paper | Video]

Let It Snow: Designing Snowfall Experience in VR (IMWUT/Ubicomp 2024)

We present Snow, a cross-modal interface that integrates cold and tactile stimuli in mid-air to create snowflakes and raindrops for VR experiences. Snow uses six Peltier packs and an ultrasound haptic display to create unique cold-tactile sensations for users to experience catching snowflakes and getting rained on their bare hands. Our approach considers humans’ ability to identify tactile and cold stimuli without masking each other when projected onto the same location on their skin, making illusions of snowflakes and raindrops. We design both visual and haptic renderings to be tightly coupled to present snow melting and rain droplets for realistic visuo-tactile experiences. For multiple snowflakes and raindrops rendering, we propose an aggregated haptic scheme to simulate heavy snowfall and rainfall environments with many visual particles. The results show that the aggregated haptic rendering scheme demonstrates a more realistic experience than other schemes. We also confirm that our approach of providing cold-tactile cues enhances the user experiences in both conditions compared to other modality conditions. [Paper | Video]

Fabric Thermal Display using Ultrasonic Waves (ISMAR2023)

This paper presents a fabric-based thermal display of polyester fabric material combined with thermally-conductive materials using an ultrasound haptic display. We first empirically test the thermal generation process in five fabric materials by applying 40 kHz ultrasonic waves to the fabric materials. We also examine their thermal characteristics by applying different frequencies and amplitudes of ultrasonic cues. We show that polyester demonstrates the best thermal performance. We then combine it with thermally-conductive materials, including copper and aluminum, and compare them with the fabric-only condition. We integrate polyester with aluminum into a glove to explore the use cases in VR and share our findings, insights, limitations, and future works. [Paper | Video]

Upper Body Thermal Referral and Tactile Masking for Localized Feedback (TVCG/VR2023)

This paper investigates the effects of thermal referral and tactile masking illusions to achieve localized thermal feedback on the upper body. We use a 2D array of sixteen vibrotactile actuators (4×4) with four thermal actuators to explore the thermal distribution on the user’s back. A combination of thermal and tactile sensations is delivered to establish the distributions of thermal referral illusions with different numbers of vibrotactile cues. The result confirms that localized thermal feedback can be achieved through cross-modal thermo-tactile interaction on the user’s back of the body. [Paper | Video]

MetaTwin: Synchronizing Physical and Virtual Spaces for Seamless World (VRST2023)

MetaTwin is a collaborative Metaverse platform that supports one-to-one spatiotemporal synchrony between physical and virtual spaces. The users can interact with other users and surrounding IoT devices without being tied to physical spaces. Resource sharing is implemented to allow users to share media, including presentation slides and music. We deploy MetaTwin in two different network environments (i.e., within the US and Korea-US international) and summarize users’ feedback about the experience. [Paper | Video 1 | Video 2]

FIRE: Mid-Air Thermo-Tactile Display in VR (VRST2022/SIGGRAPH Asia Emerging Technologies)

We design a proof-of-concept thermo-tactile feedback system with an open-top chamber, heat modules, and an ultrasound display. Our approach is to provide heated airflow along the path to the focused pressure point created from the ultrasound display to generate thermal and vibrotactile cues in mid-air simultaneously. [Paper | Video]

TangibleData: Interactive Data Visualization with Mid-Air Haptics (VRST2022/SIGGRAPH Emerging Technologies)

We build an interactive 3D data visualization tool that adapts hand gestures and mid-air haptics to provide tangible interaction in VR using ultrasound haptic feedback on 3Ddata visualization. We consider two types of 3D visualization datasets and provide different data encoding methods for haptic representations. [Paper 1 | Paper 2 | Video 1 | Video 2]

In-Air Text Input Technique (CHI2022/VRST2021)

We empirically explore fundamental requirements for achieving VR in-air typing by observing the unconstrained eyes-free in-air typing of touch typists. We examine properties of finger kinematics, correlated movement of fingers, interrelation in consecutive key-strokes, and 3D distribution of key-stroke movements. We further test finger kinematic features, including 3D position, velocity, acceleration, and temporal features, including previous fingers and keys. Based on this analysis, we assess the performance of various classifiers, including Naive Bayes, Random Forest, Support Vector Machines, and Deep Neural Networks, in terms of the accuracy for correctly classifying the keystroke. [Paper 1 | Paper 2 | Video]

Refinity: An Interactive Holographic Signage for New Retail Shopping Experience

Refinity is an interactive holographic signage for the new retail shopping experience. In this project, we show a concept of futuristic shopping experience with a tangible 3D mid-air interface that allows customers to directly select and explore realistic virtual products using autostereoscopic 3D display combined with mid-air haptics and finger tracker. We also present an example of in-store shopping scenario for natural interactions with 3D. This shopping experience will engage users in producing a memorable in-store experience with the merging of digital and physical interactions. [Paper | Video]

RealWalk: Haptic Shoes Using Actuated MR Fluid for Walking in VR (TOH/IEEE Haptics 2019)

RealWalk is a pair of haptic shoes for HMD-based VR, designed to create realistic sensations of ground surface deformation and texture through MR fluid actuators. RealWalk offers a novel interaction scheme through the physical interaction between the shoes and the ground surfaces while walking in VR. Each shoe consists of two MR fluid actuators, an insole pressure sensor, and a foot position tracker. When a user steps on the ground with the shoes, the two MR fluid actuators are depressed, creating a variety of ground material deformation such as snow, mud, and dry sand by changing its viscosity. We build an interactive VR application and compare RealWalk with vibrotactile-based haptic shoes to investigate its effectiveness. [Paper 1 | Paper 2 | Paper 3 | Video]

Touch3D: Touchscreen Interaction on Multiscopic 3D with Electrovibration Haptics (SIGGRAPH 2017)

Touch3D is an interactive mobile platform that provides realistic viewing and touching experiences through glasses-free 3D visualization with electrovibration. Touch3D is designed to take advantage of both visual and tactile illusions to maximize multimodal experience in touchscreen interaction. We seamlessly integrate two technologies: Automultiscopic 3D Display and Electrovibration Display; and weave both hardware and software into one fluid interface. Our museum application using Touch3D demonstrates important implications for the improvement of 3D perception in both visual and tactile modalities for enhanced touchscreen interaction. [Paper | Video]

AirPiano: Enhancing Music Playing Experience in Virtual Reality with Mid-Air Haptic Feedback (WHC2017)

AirPiano is a music playing system to provide touchable experiences in HMD-based virtual reality with mid-air haptic feedback. AirPiano allows users to enjoy enriched virtual piano-playing experiences with touchable keys in the air. We implement two haptic rendering schemes to mimic the resisting force of piano keys using ultrasonic vibrations. Constant Feedback is designed to provide short and intense feedback whereas Adaptive Feedback is designed to follow the changes in feedback from a real keypress. [Paper | Video]

Interactive 3D Painting with Haptic Brush in Immersive Room

We propose an interactive artwork system with automultiscopic 3D and haptic paint brush in an immersive room. Our system consists of an 81-view automultiscopic display, a handheld haptic paint brush, and a large-scale color palette station in a CAVE-like cubic room filled with the visual scene of the artwork. The 81-view rendering and multiplexing technology is applied by setting up the virtual cameras in the off-axis layout. The haptic paint brush is designed and implemented using a 2D array of multiple piezoelectric actuators. It provides the tactile feedback of spatial distance information between a virtual brush and a distal 3D object displayed on the automultiscopic display for the precise control of the brush when it is interacting with automultiscopic 3D. [Paper]

Scroll to Top