Computational Ophthalmology

Project

Computational Ophthalmology: Adaptive Sight Enhancement for Vision Deficiency Using Augmented-Reality.

Grants

JSPS Grants-in-Aid for Scientific Research (A) Grant No. 17H00738, 2017-2021., No. 20K20408, 2019-2024.

Abstract

In this research, we develop a technology called “computational ophthalmology” as an approach to dramatically improve the quality of social life of individuals with vision deficiency by making full use of the latest augmented reality technology. Various types of sensing devices are combined for capturing the objective configuration of a real environment scene or the subjective intentions of individual patient and we develop a method to compensate for the characteristic color perception and vision characteristic of the disease through an optical see-through / video see-through head-mounted display. And, it is clinically verified that the effect of adaptive visual acuity enhancement, which was difficult to use with conventional optical instruments and simple image conversion, can be achieved by the proposed technology.

Invited Talks

  1. Xiaoyang Mao, “Computational Ophthalmology: Digital Approaches to Visually Impaired Support”, IEEE International Conference on Sensors and Nanotechnology, Pulau Pinang, Malaysia, Keynote Speaker, July 2019.
  2. Xiaoyang Mao, “Arriving Light Control for Color Vision Deficiency Support Using Optical See-Through HeadMounted Display,” Computer Graphics International 2019, Calgary, Canada, Invited Speaker, June 2019.
  3. Xiaoyang Mao, “AR Daltonization for Supporting People with Color Vision Deficiency,” Computer Graphics International 2018, Bintan, Indonesia, Invited Speaker, June 2018.

Color Vision Deficiency

  1. Zhenyang Zhu, Xiaoyang Mao, “Image Recoloring for Color Vision Deficiency Compensation: A Survey,” The Visual Computer, First online 2021-07-14. (https://doi.org/10.1007/s00371-021-02240-0)
  2. Xinyi Wang, Zhenyang Zhu, Xiaodiao Chen, Kentaro Go, Masahiro Toyoura, Xiaoyang Mao, “Fast Contrast and Naturalness Preserving Image Recolouring for Dichromats,” Computers & Graphics, Vol.98, pp.19-28, 2021-5. (https://doi.org/10.1016/j.cag.2021.04.027)
  3. Zhenyang Zhu, Masahiro Toyoura, Kentaro Go, Kenji Kashiwagi, Issei Fujishiro, Tien-Tsin Wong, Xiaoyang Mao, “Personalized Image Recoloring for Color Vision Deficiency Compensation,” IEEE Transactions on Multimedia, First online 2021-03-31. (https://doi.org/10.1109/TMM.2021.3070108)
  4. Zhenyang Zhu, Xinyi Wang, Xiaodiao Chen, Masahiro Toyoura, Xiaoyang Mao, “最適化を利用した二色覚補償のための高速色変換技術,” Visual Computing シンポジウム, Article 20, 2020-12.
  5. Xinyi Wang, Zhenyang Zhu, Xiaodiao Chen, Masahiro Toyoura, and Xiaoyang Mao, “Evaluation of Color Vision Compensation Algorithms for People with Varying Degree of Color Vision Deficiency,” Cyberworlds, pp. 149-152, 2020-9. (https://doi.org/10.1109/CW49994.2020.00032)
  6. Tang, Ying, Zhu, Zhenyang, Toyoura, Masahiro, Go, Kentaro, Kashiwagi, Kenji, Fujishiro, Issei, Mao, Xiaoyang, “ALCC-Glasses: Arriving Light Chroma Controllable Optical See-Through Head-Mounted Display System for Color Vision Deficiency Compensation,” Applied Sciences, Vol.10, No.7, pp.2381, 2020-3. (https://doi.org/10.3390/app10072381)
  7. Zhenyang Zhu, Masahiro Toyoura, Kentaro Go, Issei Fujishiro, Kenji Kashiwagi, Xiaoyang Mao, “Naturalness- and information-preserving image recoloring for red–green dichromats,” Signal Processing: Image Communication, Elsevier, vol. 76, pp. 68–80, 2019-8. (https://doi.org/10.1016/j.image.2019.04.004)
  8. Zhenyang Zhu, Masahiro Toyoura, Kentaro Go, Issei Fujishiro, Kenji Kashiwagi, Xiaoyang Mao, “Processing images for red–green dichromats compensation via naturalness and information-preservation considered recoloring,” The Visual Computer, Springer-Verlag, vol. 35, no. 6-8, pp. 1053-1066, 2019-6. (https://doi.org/10.1007/s00371-019-01689-4)
  9. Ying Tang, Zhenyang Zhu, Masahiro Toyoura, Kentaro Go, Kenji Kashiwagi, Issei Fujishiro, Xiaoyang Mao, “Arriving Light Control for Color Vision Deficiency Compensation Using Optical See-Through Head-Mounted Display,” Proceedings of the 16th ACM SIGGRAPH International Conference on Virtual-Reality Continuum and its Applications in Industry, Article 6, 2018-12. (https://doi.org/10.1145/3284398.3284407)
  10. 朱 臻陽，豊浦 正広，郷 健太郎，柏木 賢治，藤代 一成，茅 暁陽，“色覚障がい支援のための個人適応型色変換アルゴリズム”，Visual Computing シンポジウム，Article 34，2019-6．
  11. 朱 臻陽，豊浦 正広，郷 健太郎，藤代 一成，柏木 賢治，茅 暁陽，“色覚障がい支援のための個人適応型色変換アルゴリズム”，画像電子学会研究会，Article 7，2018-8．
  12. 朱 臻陽，豊浦 正広，郷 健太郎，柏木 賢治，藤代 一成，茅 暁陽，“自然さ保存とコントラスト強調を考慮した色覚補償のための色変換アルゴリズム”，Visual Computing シンポジウム，Article 34，2018-6．
  13. 朱 臻陽，豊浦 正広，茅 暁陽，柏木 賢治，郷 健太郎，藤代 一成，“画像処理による色覚障がい支援”，画像電子学会ビジュアルコンピューティングワークショップ，Article 11，2017-12．
  14. 水越 駿，豊浦 正広，茅 暁陽，柏木 賢治，郷 健太郎，藤代 一成，“光学透過型ヘッドマウントディスプレイを用いた色覚補償”，画像電子学会ビジュアルコンピューティングワークショップ，Article 10，2017-12.
  15. 水越 駿，茅 暁陽，豊浦 正広，柏木 賢治，“色覚障がい者に対する透過型HMDデバイスを用いた支援”，画像電子学会研究会，pp.17-20，Article 5，2017-9．

Distorted View

  1. Katsuhito Moritake, Zhenyang Zhu, Masahiro Toyoura, Kentaro Go, Kenji Kashiwagi, Issei Fujishiro, Xiaoyang Mao, “Eye-tracker-free compensation for metamorphopsia,” Cyberworlds, pp. 78–84, 2021-9. (https://doi.org/10.1109/CW52790.2021.00019)
  2. Zhenyang Zhu, Masahiro Toyoura, Issei Fujishiro, Kentaro Go, Kenji Kashiwagi, Xiaoyang Mao, “LineM: Assessing Metamorphopsia Symptom Using Line Manipulation Task,” The Visual Computer, First online 2021-02-25. (https://doi.org/10.1007/s00371-021-02091-9)
  3. Hiromichi Ichige, Masahiro Toyoura, Kentaro Go, Kenji Kashiwagi, Issei Fujishiro, Xiaoyang Mao, “Visual Assessment of Distorted View for Metamorphopsia Patient by Interactive Line Manipulation,” Cyberworlds, pp. 187-190, 2019-10. (https://doi.org/10.1109/CW.2019.00038)
  4. 市毛 大道，豊浦 正広，郷 健太郎，柏木 賢治，藤代 一成，茅 暁陽，“線変形操作を用いた変視症検査方法”，Visual Computing シンポジウム，Article 8，2018-6．
  5. 市毛 大道，豊浦 正広，茅 暁陽，柏木 賢治，郷 健太郎，藤代 一成，“変視症の定量的評価と可視化”，画像電子学会ビジュアルコンピューティングワークショップ，Article 12，2017-12．

Visual Field Narrowing

  1. Keisuke ICHINOSE, Xi ZHAO, Issei FUJISHIRO, Masahiro TOYOURA, Kenji KASHIWAGI, Kentaro GO, Xiaoyang MAO, “Enhancing Edge Indicator for Visual Field Loss Compensation for Homonymous Hemianopia Patients Using Edge Indicator”, The Computer Vision, First online 2021-05-09. (DOI 10.1007/s00371-021-02156-9)
  2. Changtong Mao, Kentaro Go, Yuichiro Kinoshita, Kenji Kashiwagi, Masahiro Toyoura, Issei Fujishiro, Jianjun Li, Xiaoyang Mao, “Different Eye Movement Behaviors Related to Artificial Visual Field Defects – A Pilot Study of Video-Based Perimetry,” IEEE Access, vol. 9, pp. 77649-77660, 2021-5. (https://doi.org/10.1109/ACCESS.2021.3080687)
  3. 一瀬 佳祐，趙 希，藤代 一成，豊浦 正広，柏木 賢治，郷 健太郎，茅 暁陽，“同名半盲患者のための視野外情報補償”，情報処理学会第83回全国大会，pp.85-86，6Y-09，2021-3，学生奨励賞受賞．
  4. Keisuke ICHINOSE, Xi ZHAO, Issei FUJISHIRO, Masahiro TOYOURA, Kenji KASHIWAGI, Kentaro GO, Xiaoyang MAO, “Visual Field Loss Compensation for Homonymous Hemianopia Patients Using Edge Indicator”, Cyberworlds, pp. 79-85, 2020-9. (https://doi.org/10.1109/CW49994.2020.00019)
  5. Lina Chen, Kentaro Go, Yuichiro Kinoshita, Kenji Kashiwagi, Masahiro Toyoura, Issei Fujishiro, and Xiaoyang Mao, “Using an Eye Tracking Device to Discriminate Different Symptoms in Glaucoma,” Cyberworlds, pp. 141-144, 2020-9. (https://doi.org/10.1109/CW49994.2020.00030)
  6. Changtong Mao, Kentaro Go, Yuichiro Kinoshita, Kenji Kashiwagi, Masahiro Toyoura, Issei Fujishiro, Jianjun Li, and Xiaoyang Mao, “Different Eye Movement Patterns on Simulated Visual Field Defects by Video-watching Task,” Cyberworlds, pp. 153-156, 2020-9. (https://doi.org/10.1109/CW49994.2020.00033)
  7. 登野 拓志，茅 暁陽，豊浦 正広，郷 健太郎，柏木 賢治，藤代 一成，“全景映像の適応的半透明ブレンディングによる視野狭窄の緩和と視野特性の評価”，画像電子学会研究会，Article 15，2018-8．
  8. 登野 拓志，茅 暁陽，豊浦 正広，郷 健太郎，柏木 賢治，藤代 一成，“全景映像の適応的半透明ブレンディングによる視野狭窄の緩和”，情報処理学会全国大会，pp.161-162，3Y-01，2018-3

Patents

  1. 茅 暁陽，王 欣怡，朱 臻陽，豊浦 正広，“画像処理方法、画像処理プログラム、及び表示装置”，特願2020-117905，2020-7．
  2. 茅 暁陽，朱 臻陽，豊浦 正広，柏木 賢治，“画像処理方法、画像処理プログラム、及び表示装置”，特開2019-145876，2018-2．
  3. 柏木 賢治，茅 暁陽，豊浦 正広，赤澤 和也，“眼科検査装置及び眼科検査方法”，特願2017-118382，国際PCT出願 PCT-JP2018-22492，国際公開番号WO2018230582，2017-6．

Exhibitions

  1. 茅 暁陽，“色覚障がい補償のための画像変換技術”，イノベーション・ジャパン2020，I-11，2020-10．
  2. 藤代一成，菅琢哉，内野花梨：「アンビエント可視化」，KEIO TECHNO–MALL 2019（第20回慶應科学技術展），No. 45，東京国際フォーラム（東京都・千代田区），2019 年12 月13 日．
  3. 茅 暁陽，朱 臻陽，豊浦 正広，柏木 賢治，郷 健太郎，藤代 一成，“色覚障がい支援のための色変換技術”，やまなし産学官連携研究交流事業，医6，2019-10．
  4. 藤代 一成，登野 拓志，趙 希，茅 暁陽，豊浦 正広，郷 健太郎，柏木 賢治，朱 臻陽，市毛 大道，唐 瀛，“コンピュテーショナル・オプサルモロジー ARを用いた視覚障がい者の視力の適応的増強”，KEIO TECHNO-MALL (慶應科学技術展)，93，2018-12．（KTM2018賞）
  5. 藤代一成，清水文也，早川雄登，今田明洋：「SAM：知的環境メディア—概念，方法論，応用—」，KEIO TECHNO–MALL 2017（第18 回慶應科学技術展），No. 92，東京国際フォーラム（東京都・千代田区），2017 年12 月15 日

Media Coverage

  1. 「拡張現実×視覚障害「計算眼科学」がもたらす視覚体験とは」，DG LAB HAUS (Open Innovation Media), 2020 年10 月7 日
  2. 「計算メガネ」 FM Fuji 2019. 11. 28オンエア
  3. 「視界のゆがみ簡単に推定山梨大など、矯正技術にも応用」，日経産業新聞朝刊006 面，2019 年6 月18 日
  4. 日本経済新聞, “その色は 味わいは 本物か 技術で五感を操る 仮想現実が拓く世界（4）“, （全国版朝刊，2019年5月29日011面，Web版にも掲載有）

Other

Mono-glass for Providing Distance Information

mono.s We propose mono-glass for providing distance information for people losing sight in one eye. We implemented a pilot system of mono-glass under careful consideration of precision, real-time processing, and intuitive presentation. >> More