Five experiments investigated the role of spatial connectedness between a pair of objects presented in the change detection task for the actual capacity of visual working memory (VWM) in healthy young adults (total N = 405). Three experiments yielded a surprising nonlinear relationship between the proportion of pair-wise connected objects and capacity, with the highest capacity observed for homogenous displays, when either all objects were connected or disjointed. A drop in capacity, ranging from an average of a quarter of an object out of three objects maintained in VWM, was noted when only some objects were connected while others were disjointed. As indicated by another two experiments, this effect was specific to double-feature encoding, but disappeared when single visual features had to be memorized. No existing theoretical model of VWM can directly explain this novel effect. Overall, the nonlinear effect of spatial connectedness implies that representations in VWM possess hierarchical structure defined by wholes, parts, features, and their relations, and the heterogeneity of such a structure hinders VWM performance, while homogeneity facilitates it.