3D Grape Bunch Model Reconstruction from 2D Images

Abstract

A crucial step in the production of table grapes is berry thinning. This is because the market value of table grape production is significantly influenced by bunch compactness, bunch form, and berry size, all of which are primarily regulated by this task. Grape farmers must count the number of berries in the working bunch and decide which berry should be eliminated during thinning, a process requiring extensive viticultural knowledge. However, the use of 2D pictures for automatic berry counting and identifying the berries to be removed has limitations, as the number of visible berries might vary greatly depending on the direction of view. In addition, it is extremely important to understand the 3D structure of a bunch when considering future automation via robotics. For the reasons stated, obtaining a field-applicable 3D grape bunch model is needed. Thus, the contribution of this study is a novel technology for reconstructing a 3D model of a grape bunch with uniquely identified berries from 2D images captured in the real grape field environment.

Highlights

  • Reconstruct a 3D model of a grape bunch with uniquely identified berries from videos captured of a real grape field.
  • New clustering and neural network-based for uniquely identifying berries in 3D point cloud and refined with video frames.
  • A 2D instance segmentation DNN preprocesses to exclude background from 3D point cloud generation.

Publications

  1. Y. S. Woo, P. Buayai, H. Nishizaki, K. Makino, L. M. Kamarudin, X. Mao, “End-to-end lightweight berry number prediction for supporting table grape cultivation,” Computers and Electronics in Agriculture, vol. 213, p. 108203, 2023, doi: https://doi.org/10.1016/j.compag.2023.108203

References

[1] P. Buayai, K. R. Saikaew, and X. Mao, “End-to-end automatic berry counting for table grape thinning,” IEEE Access, vol. 9, pp. 4829-4842, 2020, doi:  https://doi.org/10.1109/ACCESS.2020.3048374.