Deep Learning for Video Object SegmentationA Review

Authors Organisations
  • Mingqi Gao(Author)
    University of Warwick
    Southern University of Science and Technology
  • Feng Zheng(Author)
    Southern University of Science and Technology
  • James Yu(Author)
    Southern University of Science and Technology
  • Caifeng Shan(Author)
    Southern University of Science and Technology
  • Guiguang Ding(Author)
    Tsinghua University
  • Jungong Han(Author)
    University of Warwick
Type Article
Original languageEnglish
Pages (from-to)457-531
Number of pages75
JournalArtificial Intelligence Review
Issue number1
Early online date08 Apr 2022
Publication statusPublished - 01 Jan 2023
Permanent link
Show download statistics
View graph of relations
Citation formats


As one of the fundamental problems in the field of video understanding, video object segmentation aims at segmenting objects of interest throughout the given video sequence. Recently, with the advancements of deep learning techniques, deep neural networks have shown outstanding performance improvements in many computer vision applications, with video object segmentation being one of the most advocated and intensively investigated. In this paper, we present a systematic review of the deep learning-based video segmentation literature, highlighting the pros and cons of each category of approaches. Concretely, we start by introducing the definition, background concepts and basic ideas of algorithms in this field. Subsequently, we summarise the datasets for training and testing a video object segmentation algorithm, as well as common challenges and evaluation metrics. Next, previous works are grouped and reviewed based on how they extract and use spatial and temporal features, where their architectures, contributions and the differences among each other are elaborated. At last, the quantitative and qualitative results of several representative methods on a dataset with many remaining challenges are provided and analysed, followed by further discussions on future research directions. This article is expected to serve as a tutorial and source of reference for learners intended to quickly grasp the current progress in this research area and practitioners interested in applying the video object segmentation methods to their problems. A public website is built to collect and track the related works in this field:


  • Convolutional neural network, Deep learning, Video object segmentation