Visual-guided Robotic Object Grasping using Dual Neural Network Controllers

Authors Organisations
Type Article
Original languageEnglish
JournalIEEE Transactions on Industrial Informatics
Publication statusAccepted/In press - 18 May 2020
Show download statistics
View graph of relations
Citation formats


It has been a challenging task for a robotic arm to accurately reach and grasp objects, and much research attention has been attracted. This paper proposes a robotic hand-eye coordination system by simulating the human behavior pattern to achieve a fast and robust reaching ability. This is achieved by two neural-network-based controllers, including a rough reaching movement controller implemented by a pre-trained Radial Basis Function (RBF) for rough reaching movements, and a correction movement controller built from a specifically designed Brain Emotional Nesting Network (BENN) for smooth correction movements. In particular, the proposed BENN is designed with high nonlinear mapping ability, with its adaptive laws derived from the Lyapunov stability theorem; from this, the robust tracking performance and accordingly the stability of the proposed control system are guaranteed by the utilisation of the H∞ control approach. The proposed BENN is validated and evaluated by a chaos synchronisation simulation, and the overall control system by object grasping tasks through a physical robotic arm in a real-world environment. The experimental results demonstrate the superiority of the proposed control system in reference to those with single neural networks.