Adaptive Memory Management for Video Object Segmentation

The paper titled Adaptive Memory Management for Video Object Segmentation authored by Ali Pourganjalikhan and Charalambos Poullis will be presented at the 19th Conference of Robots and Vision (CRV) 2022.

TL;DR: Given a sequence of frames and the masks of the target objects in their first appearance, the goal is to segment the corresponding objects in the rest of the frame sequence. Storing the intermediate frames' predictions provides the network with richer cues for segmenting an object in the current frame. However, the size of the memory bank gradually increases with the length of the video, which slows down inference speed and makes it impractical to handle arbitrary-length videos. We propose an adaptive memory bank strategy for matching-based networks for semi-supervised video object segmentation that can handle videos of arbitrary length by discarding obsolete features. Our strategy keeps a fixed top-K bank of frames instead, which results in a fixed-sized memory bank and faster performance.