|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| We introduce a new image segmentation task, termed Entity Segmentation (ES) with the aim to segment all visual entities in an image without considering semantic category labels. It has many practical applications in image manipulation/editing where the segmentation mask quality is typically crucial but category labels are less important. In this setting, all semantically-meaningful segments are equally treated as categoryless entities and there is no thing-stuff distinction. Based on our unified entity representation, we propose a center-based entity segmentation framework with two novel modules to improve mask quality. Experimentally, both our new task and framework demonstrate superior advantages as against existing work. In particular, ES enables the following: (1) merging multiple datasets to form a large training set without the need to resolve label conflicts; (2) any model trained on one dataset can generalize exceptionally well to other datasets with unseen domains. |
 |
| The ``entity'' refers to either a thing (instance) mask or a stuff mask in the common context, which can be any semantically meaningful and coherent region in the open-world setting. |
 |
| Unlike existing panoptic segmentation methods that generally include two output branches for thing and stuff separately, the proposed framework relies on a dense one-stage detection architecture and a dynamic mask kernel branch to detect and segment all entities (regardless of thing or stuff) in a unified manner. Besides, we propose a global kernel bank module and an overlap suppression module to further improve the model's performance on the task. The global kernel bank module generates mask kernels to exploit some common properties (e.g, textures, edges) shared by many entities and the overlap suppression module encourages the masks not to overlap with each other. |
 |
Lu Qi*, Jason Kuen*, Yi Wang, Jiuxiang Gu, Hengshuang Zhao, Zhe Lin, Philip Torr, Jiaya Jia.
Open-World Entity Segmentation. 2021. (hosted on ArXiv) |
Acknowledgements |