One challenge for the modern recommendation systems is the Tyranny of Majority - the generated recommendations are often optimized for the mainstream trends so that the minority preference groups remain discriminated. Moreover, most modern recommendation techniques are characterized as black-box systems. Given a lack of understanding of the dataset characteristics and insufficient diversity of represented individuals, such approaches inevitably lead to amplifying hidden data biases and existing disparities. In this research, we address this problem by proposing a novel approach to detecting and describing potentially discriminated user groups for a given recommendation algorithm. We propose a Bias-Aware Hierarchical Clustering algorithm that identifies user clusters based on latent embeddings constructed by a black-box recommender to identify users whose needs are not met by the given recommendation method. Next, a post-hoc explainer model is applied to reveal the most important descriptive features that characterize these user segments. Our method is model-agnostic and does not require any a priori information about existing disparities and sensitive attributes. An experimental evaluation on a synthetic dataset and two real-world datasets from different domains shows that, compared with other clustering methods and arbitrarily selected user groups, our method is capable of identifying underperforming segments for different recommendation algorithms, and detect more severe disparities.