Development of Network-Based Multi-Modal Surgery Training System
In general, minimally invasive surgery is the most difficult surgery because a field of view of an endoscope is limited and force sensation from surgical tools such as forceps is poor. Especially in early clinical education for medical students, a virtual reality surgical simulator will be an effective tool.
In this research, we proposed a network-based visuohaptic surgery training system for laparoscopical techniques. To realize this system, we developed a novel volume-based haptic communication approach, which allows participants at remote sites on the network to simultaneously interact with the same target object in virtual environments presented by multi-level computer performance systems, by only exchanging a small set of manipulation parameters for the target object and additional packet for synchronization of status of binary tree and deformation of shared volume model. In addition, we developed a multi-modal surgery training approach. In this approach, we recorded a video from the first person perspective and operation information of surgical tools of an instructor. And then, we displayed the recorded video and the guidance force to trainees. We constructed a prototype surgery training system, and effectiveness of our approach was confirmed.
A journal paper (submitted).
Kazuyoshi Tagawa, Hiromi T. Tanaka, Yoshimasa Kurumi, Masaru Komori and Sigehiro Morikawa, "A Laparoscopic Surgery Simulator Using First Person View and Guidance Force", Stud Health Technol Inform (MMVR2013), 2013.
Kazuyoshi Tagawa, Hiromi T. Tanaka, Yoshimasa Kurumi, Masaru Komori and Sigehiro Morikawa, "Development of a VR Training System for Laparoscopic Cholecystectomy", Journal of Japan Society of Computer Aided Surgery, Vol.14, No.3, pp.340-341, 2012.
Kazuyoshi Tagawa, Hiromi T. Tanaka, Masaru Komori, Yoshimasa Kurumi and Sigehiro Morikawa, "A Visuohaptic Surgery Training System for Laparoscopical Techniques", The Japanese Journal for Medical Virtual Reality, Vol.10, No.1, pp.11-18, 2012.
Kazuyoshi Tagawa, Tatsuro Bito and Hiromi T. Tanaka, "A Method of Synchronization for Haptic Collaborative Virtual Environments in Multipoint and Multi-level Computer Performance Systems", Stud Health Technol Inform (MMVR2011), pp.638-644, 2011.
Tatsuro Bito, Kazuyoshi Tagawa and Hiromi T. Tanaka, "A Synchronization Method for Online Remesh Deformation Model in Haptic Collaborative Virtual Environments of Multipoint", Proc. The 15th Annual Conference of Virtual Reality Society of Japan, pp. CD, 2010. (Award for Young Scientists)
Expression of Cystohepatic Duct Anomaly using Modular Structured Organ Model in a Laparoscopic Surgery Simulator
Cystohepatic duct anomaly is often found in a laparoscopic cholecystectomy and these cause certain risks in this surgical procedure. Although anomalies can be detected by preoperative CT or MRI examinations, it is hard to operate smoothly under the actual surgical view on a 2D monitor. Surgical training for such special cases is one of the advantages of a VR surgical simulator. This is especially true in early clinical education for medical students; the simulator can be an effective tool and teaching resource both visually and haptically.
In this research, we have developed a novel laparoscopic surgical simulator. In this simulator, each part of varied objects is maintained by modularized multi-resolution models, and any models which consist of arbitrary combination of these modules can be produced with low computational cost.
Kazuyoshi Tagawa, Naoko Omi, Hiromi T. Tanaka, Masaru Komori, Yoshimasa Kurumi and Shigehiro Morikawa, "A Semi-Automatic Modeling Approach for Serosa and Adhesion", In Proc. of IEEE EMBC (IEEE-EMBC2013), 2013. (accepted)
Masaru Komori, Kazuyoshi Tagawa, Hiromi T. Tanaka, Yoshimasa Kurumi and Shigehiro Morikawa, "Current Status of Organ Variation 3D Model Library Construction", In Proc. of IEEE EMBC (IEEE-EMBC2013), 2013. (accepted)
Kazuyoshi Tagawa, Hiromi T. Tanaka, Yoshimasa Kurumi, Masaru Komori and Sigehiro Morikawa, "A Constructive Multi-Resolution Modeling of Anomalies of Organs", IEICE Trans. on Information and Systems, Vol.J96-D, No.5, pp.1365-1373, May. 2013.
Kazuyoshi Tagawa, Hiromi T. Tanaka, Yoshimasa Kurumi, Masaru Komori and Sigehiro Morikawa, "Expression of Cystohepatic Duct Anomaly Using Modular Structured Organ Model in a Laparoscopic Surgery Simulator", International Journal of Computer Assisted Radiology and Surgery (CARS2012), 2012.
Acceleration of Deformation Simulation by Multi-Rate Online Re-Mesh Approach and GPU
In this research, we proposed an approach to generate high quality force and deformation using a novel automatic space and time adaptive level of detail technique in combination with a parallel computation using a graphics processing unit (GPU). The elastic object is represented by a multi-resolution hierarchy of tetrahedral adaptive volume mesh. The tetrahedral adaptive volume mesh and the time step of the deformation simulation are locally refined by online re-mesh and multi-rate simulation to concentrate the computational load into the regions that deform the most.
In order to compute the online re-mesh and multi-rate deformation simulation on a GPU efficiently, we propose a novel data structure which consists of an extended and transposed connection table, a node list and a separated mass list. This effective computation is achieved by the relocatability of the connection table and optimized memory access at the computation of both deformation and re-meshing.
Kazuyoshi Tagawa, Takahiro Yamada and Hiromi T. Tanaka, "A Rectangular Tetrahedral Adaptive Mesh Based Corotational Deformation Model for Interactive Soft Tissue Simulation", In Proc. of IEEE EMBC (IEEE-EMBC2013), 2013. (accepted)
Kazuyoshi Tagawa, Satoshi Nakagawa, Hiromi T. Tanaka, "An Efficient Deformation and Ablation Simulation Using a Rectangular Tetrahedra Based Online Re-Mesh and a Non-Linear Finite Element Model", Transaction of Virtual Reality Society of Japan, Vol.18, No.1, 2013.
Kazuyoshi Tagawa, Yasuyuki Sasaki and Hiromi T. Tanaka, "Online Re-Mesh and Multi-Rate Deformation Simulation by GPU for Haptic Interaction with Large Scale Elastic Objects", In Proc. of IEEE Haptics, pp.531-538, 2012.
Kazuyoshi Tagawa, Yasuyuki Sasaki, Hiromi T. Tanaka, "Acceleration of Online Remesh Deformation Simulation by GPU for Haptic Interaction with Large Scale Deformable Objects", Transaction of Virtual Reality Society of Japan, Vol.16, No.3, pp.479-487, 2011.
Yasuyuki Sasaki, Kazuyoshi Tagawa and Hiromi T. Tanaka, "Stable Force Presentation by Fast Deformation Computation Using Online-Remesh and GPU", In Proc. of ACM Haptic and Audio Interaction Design, pp. CD, 2011.
Kazuyoshi Tagawa and Hiromi T. Tanaka, "A Hybrid Dynamic Deformation Model for Surgery Simulation", In Proc. of Medicine Meets Virtual Reality, pp.645-649, 2011.
Kazuyoshi Tagawa, Takeshi Fujii, Satoshi Yamaguchi and Hiromi T. Tanaka, "A Multi-rate Online Remesh Method for Haptic Needle Insertion Simulation into Nonuniform Soft Tissue", The Japanese Journal for Medical Virtual Reality, Vol. 9, No. 1, pp.7-16, 2011.
Kazuyoshi Tagawa, Satoshi Nakagawa, Hiromi T. Tanaka, Yoshimasa Kurumi, Sigehiro Morikawa and Masaru Komori, "Efficient Detachment Simulation Using Online-Remesh Deformation Models of Rectangular Tetrahedral Volume Mesh", International Journal of Computer Assisted Radiology and Surgery (CARS2011), Volume 6, Supplement 1, pp.118-120, 2011.
Adaptive and Embedded Deformation Model for Complex Inhomogeneous Elastic Objects
In this research, we proposed a novel approach to haptic interact with complex inhomogeneous elastic objects like organs. In surgery simulations, representation of complex inhomogeneous elastic objects which have various physical properties and geometries is required. Also, consideration of geometric nonlinearity is required.
To solve this problem, we propose an adaptive and embedded deformation model. This approach uses both an online re-mesh deformation model and a corotational FEM model together. Our approach can update an inverse of a global stiffness matrix of a corotational FEM model quickly during the online re-meshing. In addition, an efficient computation algorithm by GPU is also proposed.
Kazuyoshi Tagawa, Tatsuya Oishi and Hiromi T. Tanaka, "Adaptive and Embedded Deformation Model: An Approach to Haptic Interaction with Complex Inhomogeneous Elastic Objects", In Proc. of IEEE Haptics, 2013.
Tatsuya Oishi, Kazuyoshi Tagawa and Hiromi T. Tanaka, "An Adaptive and Embedded Deformation Simulation for Inhomogeneous Soft Objects", Proc. 第150回グラフィクスとCAD研究会, 2013.
Tatsuya Oishi, Kazuyoshi Tagawa and Hiromi T. Tanaka, "非一様柔軟物との力覚インタラクションのための適応的埋込み変形モデル", 第9回力触覚の提示と計算研究会抄録集, pp.CD, 2012.
Acquisition of Elastically Deformable Object Model Based on Measurement
In this research, we proposed an approach to generating impulse response deformation model (IRDM) based on the measurement of real object. In this approach, the external force on the surface of the object is applied by air jet, and the deformation of the object is measured using markers and stereo cameras. Through implementation of the measurement system and experiments, it was proved that models that behave similarly to the real objects can be obtained. Also, the evaluation of the model using subjects was performed, and the similarity of the model to the real object from the aspect of perception was confirmed.
Koichi Hirota, Kazuyoshi Tagawa, "Acquisition of Impulse Response Deformation Model based on Measurement", Transaction of Virtual Reality Society of Japan, Vol.18, No.2, 2013. (accepted)
Koichi Hirota, Kazuyoshi Tagawa, "Acquisition of Elastically Deformable Object Model Based on Measurement", Haptics: Perception, Devices and Scenarios, LNCS 5024 (Eurohaptics2012), 2012.
Koichi Hirota and Kazuyoshi Tagawa, "計測に基づく弾性変形モデルの構築, 力触覚の提示と計算研究会抄録集, pp.CD, 2012.
Impulse Response Deformation Model
In this research, we proposed an approach to haptic interaction with dynamically deformable object. In this approach, the behavior of an object is defined by a set of temporal deformation patterns after impulse force is applied to each degree of freedom, which we call impulse response deformation model. Deformation resulting from interaction is obtained by computing convolution of the model and the history of interaction force. The time complexity of computing interaction
force is independent of the complexity of the model. This feature is advantageous for time-critical applications.
A journal paper (submitted).
Kazuyoshi Tagawa, Koichi Hirota and Michitaka Hirose, "Manipulation of Dynamically Deformable Object using Impulse-Based Approach", in Book of "Advances in Haptics" (ISBN 978-953-307-093-3), 2010.
Kazuyoshi Tagawa, Koichi Hirota, and Michitaka Hirose, "A Data Compression Method for Impulse Response Deformation Model",
In Proc. of Third Joint EuroHaptics Conference and Symposium on Haptic Interfaces for Virtual Enviroments and Teleoperator Systems (WorldHaptics09), Mar. 2009.
Kazuyoshi Tagawa, Koichi Hirora, and Michitaka Hirose, "Surface Contact Interaction with Dynamically Deformable Object", In Proc. of 6th International Conference of EuroHaptics 2008, pp.675-680, Jun. 2008.
Kazuyoshi Tagawa, Koichi Hirora, and Michitaka Hirose, "Manipulation of Dynamically Deformable Object", In Proc. of 16th Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems (IEEE Haptics08), pp.327-333, Mar. 2008.
Kazuyoshi Tagawa, Koichi Hirota, and Michitaka Hirose, "Dynamic
Deformation Model for Haptic Interaction", IEICE Trans. on Information
and Systems, Vol.J90-D, No.9, pp.2615-2623, Sep. 2007.
Kazuyoshi Tagawa, Koichi Hirota, and Michitaka Hirose, "Acquiring &
Representing Elasticity of Deformable Objects using Impulse Response
Deformation Model", NASA Ames Research Center, Moffett Field, CA, USA,
Mar. 2007.
Kazuyoshi Tagawa, Koichi Hirota, and Michitaka Hirose, "Impulse
Response Model: an Approach to Haptic Interaction with Dynamically
Deformable Object", In Proc. of 14th Symposium
on Haptic Interfaces for Virtual Environment and Teleoperator Systems
(IEEE Haptics06), Mar. 2006.
Automultiscopic Display
In this research, we developed two types of automultiscopic display. In the display, different and independent views are presented to different viewpoints. Our first display system was realized by rotating a flat-panel display whose viewing angle is strictly limited, or has sharp directivity. Latter display system was realized by rotating a mirror on two axes and switching images directionally at high speed.
Yusuke Doyama, Tomohiro Tanikawa, Kazuyoshi Tagawa, Koichi Hirota and Michitaka Hirose, "CAGRA: Optimal Parameter Setting and Rendering Method for Occlusion-capable Automultiscopic Three-Dimensional Display", In Proc. of IEEE Virtual Realiry 2010 (IEEE-VR2010), Mar. 2010.
Yusuke Doyama, Tomohiro Tanikawa, Kazuyoshi Tagawa, Koichi Hirota and Michitaka Hirose, "CAGRA: Occlusion-capable Automultiscopic 3D Display with Spherical Coverage", In Proc. of 18th International Conference on Artificial Reality and Telexistence (ICAT08), pp.36-42, Dec. 2008.
Koichi Hirora, Kazuyoshi Tagawa, and Yasuhiro Suzuki, "Automultiscopic Display by Revolving Flat-panel Displays", In Proc. of IEEE Virtual Realiry 2008 (IEEE-VR08), pp.161-168, Mar. 2008.
Improvement of Cloth Behavior Simulation Using Evaluation Experiments in Vacuum
In spite of many dynamic cloth simulation methods have been proposed in computer graphics and textile engineering fields, there are no quantitative evaluation method for comparing real and virtual with various dynamic cloth deformations that can be seen our daily life.
In this paper, the evaluation experiments in vacuum is proposed and several results of experiments are evaluated and discussed.
Kazuyoshi Tagawa, Hirotaku Hayashi, and Ryugo Kijima, "Empirical
Evaluation of Cloth Behavior Simulation", In Proc. of Int. Conf. on
VSMM '04, Int. Soc. On VSMM, 2004.
Kazuyoshi Tagawa, Hirotaku Hayashi, and Ryugo Kijima, "Evaluation
of Cloth Behavior Simulation", In Proc. of Symposium on Computer
Graphics and Animation, ACM Siggraph / Eurographics, 2004.
Kazuyoshi Tagawa, Hirotaku Hayashi, and Ryugo Kijima, "Dynamic Cloth
Simulation and its Evaluation Experiments in Vacuum", Transaction
of Virtual Reality Society of Japan, Vol.8, No.3, pp.263-270,
2003. [PDF]
競争的外部資金: 研究代表者 田川和義, the Sasakawa Scientific Research Grant from the Japan Science Society, 2003年度 1年間（終了）．
Construction of a Virtual Fashion System
We constructed a virtual fashion system which enables virtual fitting of clothes. To realize this system, we proposed several novel approaches: a fast and stable cloth behavior simulation, fast collision detection approach between the cloth and human, a motion capture system using gyro sensors, a large screen, and a virtual mirror.
Kazuyoshi Tagawa, Tomokazu Kakimi, Ryugo Kijima, and Takeo Ojika, "Dynamic Cloth Simulation Method using an Elastic Model on the Virtual Fashion System", In Research reports of the Faculty of Engineering, Gifu University, pp.93-100, 1999.
Kazuyoshi Tagawa, Tomokazu Kakimi, Ryugo Kijima, and Takeo Ojika, "A Method for Dynamic Cloth Simulation and Application in a Virtual Fashion System", In Proc. of Int. Conf. on
VSMM '98, Int. Soc. On VSMM, 1998.
