南湖新闻网讯（通讯员 林家豪）近日，我校柑橘全程机械化平台研究团队以“Non-destructive fruit firmness evaluation using a soft gripper and vision-based tactile sensing”为题在农业计算机与电子信息领域期刊Computers and Electronics in Agriculture发表研究论文。该研究通过视触觉传感柔性末端机械爪与深度学习方法，实现了对球形水果的稳定抓取和成熟度准确分类。
果实硬度是判断其成熟程度的重要指标之一，反映了内部淀粉物质的形态变化，可以更准确地指示内部组织的成熟度，对其准确评估对采后管理决策具有重要参考意义。团队受到鳍条效应启发设计了一种软材料制作的柔性机械爪，其具有稳定无损抓取水果的功能的同时，可联合深度学习网络评估抓取过程中的作用力大小，进而实现判断水果硬度和成熟程度的功能。
研究首先使用有限元建模和相关实验证明了柔性材料制作的机械爪具备实现稳定抓取同时、兼具非破坏性的优势；随后结合视觉技术记录机械爪与水果接触时的变形特性，提出的深度学习网络的序列式图像信息处理模型——包括卷积神经网络、长短时记忆单元，使得该网络模型能够将软机械爪在时序下的变形特征，进行作用力分析进而推断水果硬度与成熟度。研究以西红柿和油桃为试验样本，验证了柔性机械爪的抓取稳定性，并采集了相关序列图像数据集与水果硬度预测的深度学习网络。
同时为了方便现实场景中的应用，研究团队将控制算法和深度学习模型移植到了边缘计算设备上，并将该套系统搭载在机械臂上，提高了运动自由度，预期可实现生产线上的基于硬度的水果成熟度分级。同时制作成本低廉，准确度较高，与其他水果硬度与成熟度检测手段相比应用潜力巨大。
我校工学院博士研究生林家豪和本科生胡晴为本论文共同第一作者，工学院副研究员陈耀晖和澳洲联邦科学与工业研究组织博士后研究员王兴为共同通讯作者。工学院李善军教授参与了论文的指导工作，硕士杜璇和理学院硕士赵亮共同参与了论文研究。
审核人 陈耀晖
【英文摘要】
As fruit firmness is a crucial characteristic associated with the maturity level, its accurate estimation is of great importance to post-harvest processing and wholesale in the industry. Benefiting from the advances of soft robotics, a soft gripper with simultaneous compliant deformation and tactile sensing is proposed in this study for the fruit firmness classification. The gripper design inspired by the fin ray effect can achieve active deformation, which helps simplify the actuation system and improve the delicate manipulation capability. Finite element modelling, along with experimental tests, is first utilized to validate the gripper’s feasibility in compliant and safe fruit grasping, and respiratory tests are then conducted to further demonstrate the non-destructive nature. Moreover, fruit–gripper interaction is captured by visual sensors and then processed using an attention-based CNN–LSTM algorithm to predict firmness information. Tomatoes and nectarines are chosen as the sample fruit for experimental validation. R² values of their firmness prediction are 0.795 and 0.753, and the accuracy of maturity grading is 84.6% and 81.5%, respectively. In general, the soft gripper provides a promising solution for both safe grasping and non-destructive firmness evaluation, and it is expected to be integrated into automated production lines to pack fruit based on different firmness levels.
论文链接：https://www.sciencedirect.com/science/article/pii/S0168169923006440
