Erwin Wu

Abstract　Accurate target detection in low-light environments is crucial for unmanned aerial vehicles (UAVs) and autonomous driving applications. In this study, the authors introduce a real-time multimodal fusion for enhanced detection (RMF-ED), a novel framework designed to overcome the limitations of low-light target detection. By leveraging the complementary capabilities of near-infrared (NIR) cameras and light detection and ranging (LiDAR) sensors, RMF-ED enhances detection performance. An advanced NIR generative adversarial network (NIR-GAN) model was developed to address the lack of annotated NIR datasets, integrating structural similarity index measure (SSIM) loss and L1 loss functions. This approach enables the generation of high-quality NIR images from RGB datasets, bridging a critical gap in training data. Furthermore, the multimodal fusion algorithm integrates RGB images, NIR images, and LiDAR point clouds, ensuring consistency and accuracy in proposal fusion. Experimental results on the KITTI dataset demonstrate that RMF-ED achieves performance comparable to or exceeding state-of-the-art fusion algorithms, with a computational time of only 21 ms. These features make RMF-ED an efficient and versatile solution for real-time applications in low-light environments.

Related workRMF-ED

Abstract　Existing research showed that unpleasant haptic feedback, such as pain, could enhance the user experience and performance in various scenarios (e.g. entertainment and training). This paper introduces ThermOuch, a wearable thermo-haptic device that leverages the thermal grill illusion (TGI) to simulate pain sensations in virtual reality (VR) without causing actual invasive/non-invasive harm. Our results of the user-perception experiments revealed that higher temperature-changing rates, particularly with increased warming, were associated with more intense pain perceived by the participants through our system. Furthermore, a higher ratio of warm-to-cool temperature transitions reduced the sensation of coldness prior to pain. Our experiments also showed that introducing an additional stimulus unit potentially heightened pain perception, and altering the spacing between stimulus units modified the perceived pain area. Lastly, the user study in VR demonstrated that ThermOuch significantly enhanced the sense of presence and body ownership for the participants, as well as elevated their biosignal-indicated arousal levels.

Abstract　As an extension of physical space, VR and MR technologies offer convenient solutions for spatial and product design by providing immersive user experiences. Nevertheless, these technologies are limited in their ability to provide physical feedback, which can lead to a perceived disconnection between visual, auditory, and bodily sensations. This paper introduces PhytualBlend, an innovative real-time interaction system designed to enhance the physical feedback experience within virtual environments. The system employs perceptual and executive hardware driven by development boards as its hardware components. User’s operation on physical entities can be synchronized seamlessly to server-based virtual world through a local information hub, including position, orientation, and haptic interaction. Diverse interaction modes within the virtual world trigger corresponding feedback in the physical space, encompassing tactile vibrations, thermal sensations, visual illumination, and object deformations. Our prototypical demonstrates remarkable performance, achieving a frame rate exceeding 60 frames per second with imperceptible network latency. The modular design of PhytualBlend enables efficient and scalable expansion of various physical feedback, interactive modes, and new virtual environments. PhytualBlend bridges the gap between the virtual and physical worlds, enabling remote collaboration participants to experience and interact with virtual contents in a more tangible and realistic manner.

Abstract　As construction complexity increases in the architecture, engineering and construction (AEC) industry, traditional paper plans struggle to clearly and conveniently provide adequate installation information, and they suffer from poor interactivity and limited feedback guidance. This paper proposes a Phygital-Aided Construction System (PACS), which combines AR and embedded sensing technologies to help unskilled workers build complex structures. The system is based on an AR user interface and spatially anchored virtual models to help participants review information before construction and provide real-time feedback to unskilled workers on assembly location, assembly method and assembly results during various steps of construction. We designed modular structures called "capsules" to house embedded hardware. The hardware combination (pressure sensors, magnetic sensors, etc.) and placement can be selected according to different sensing needs. The capsules merge with the structural components into a unified whole, which provides long-term condition tracking to meet building maintenance needs. We use HoloLens as the display device and Unity as the customized AR environment carrier, allowing the experiencer to assemble a wooden Dougong model composed of 16 components. The experiment shows that participants can master the use of the system after simple and quick training. This system reduces assembly time and assembly errors compared to paper instructions. In addition, the results of the questionnaire indicate that the system helps improve users' sense of participation and concentration in the assembly process. PACS has the potential to reduce the construction threshold of complex buildings, shorten construction time and reduce construction errors.