Simple view
Full metadata view
Authors
Statistics
Explaining self-supervised image representations with visual probing
computer vision
language and vision machine learning
Unsupervised Learning AI Ethics Trust Fairness
explainability
Recently introduced self-supervised methods for image representation learning provide on par or superior results to their fully supervised competitors, yet the corresponding efforts to explain the self-supervised approaches lag behind. Motivated by this observation, we introduce a novel visual probing framework for explaining the self-supervised models by leveraging probing tasks employed previously in natural language processing. The probing tasks require knowledge about semantic relationships between image parts. Hence, we propose a systematic approach to obtain analogs of natural language in vision, such as visual words, context, and taxonomy. We show the effectiveness and applicability of those analogs in the context of explaining self-supervised representations. Our key findings emphasize that relations between language and vision can serve as an effective yet intuitive tool for discovering how machine learning models work, independently of data modality. Our work opens a plethora of research pathways towards more explainable and transparent AI.
cris.lastimport.scopus | 2024-04-24T01:16:58Z | |
dc.abstract.en | Recently introduced self-supervised methods for image representation learning provide on par or superior results to their fully supervised competitors, yet the corresponding efforts to explain the self-supervised approaches lag behind. Motivated by this observation, we introduce a novel visual probing framework for explaining the self-supervised models by leveraging probing tasks employed previously in natural language processing. The probing tasks require knowledge about semantic relationships between image parts. Hence, we propose a systematic approach to obtain analogs of natural language in vision, such as visual words, context, and taxonomy. We show the effectiveness and applicability of those analogs in the context of explaining self-supervised representations. Our key findings emphasize that relations between language and vision can serve as an effective yet intuitive tool for discovering how machine learning models work, independently of data modality. Our work opens a plethora of research pathways towards more explainable and transparent AI. | pl |
dc.affiliation | Wydział Matematyki i Informatyki : Instytut Informatyki i Matematyki Komputerowej | pl |
dc.conference | Thirtieth International Joint Conference on Artificial Intelligence | pl |
dc.conference.city | Montreal | |
dc.conference.country | Kanada | |
dc.conference.datefinish | 2021-08-27 | |
dc.conference.datestart | 2021-08-19 | |
dc.conference.shortcut | IJCAI | |
dc.contributor.author | Basaj, Dominika | pl |
dc.contributor.author | Oleszkiewicz, Witold | pl |
dc.contributor.author | Sieradzki, Igor - 203333 | pl |
dc.contributor.author | Górszczak, Michał | pl |
dc.contributor.author | Rychalska, Barbara | pl |
dc.contributor.author | Trzciński, Tomasz - 428564 | pl |
dc.contributor.author | Zieliński, Bartosz - 106948 | pl |
dc.date.accessioned | 2021-12-06T11:30:56Z | |
dc.date.available | 2021-12-06T11:30:56Z | |
dc.date.issued | 2021 | pl |
dc.date.openaccess | 0 | |
dc.description.accesstime | w momencie opublikowania | |
dc.description.conftype | international | pl |
dc.description.physical | 592-598 | pl |
dc.description.publication | 0,6 | pl |
dc.description.version | ostateczna wersja wydawcy | |
dc.identifier.doi | 10.24963/ijcai.2021/82 | pl |
dc.identifier.eisbn | 978-0-9992411-9-6 | pl |
dc.identifier.project | POIR.04.04.00- 00-14DE/18-00 | pl |
dc.identifier.project | 2018/31/N/ST6/02273 | pl |
dc.identifier.uri | https://ruj.uj.edu.pl/xmlui/handle/item/284703 | |
dc.language | eng | pl |
dc.language.container | eng | pl |
dc.pubinfo | Freiburg, Germany : International Joint Conferences on Artificial Intelligence | pl |
dc.rights | Udzielam licencji. Uznanie autorstwa - Użycie niekomercyjne - Bez utworów zależnych 4.0 Międzynarodowa | * |
dc.rights.licence | OTHER | |
dc.rights.uri | http://creativecommons.org/licenses/by-nc-nd/4.0/legalcode.pl | * |
dc.share.type | inne | |
dc.subject.en | computer vision | pl |
dc.subject.en | language and vision machine learning | pl |
dc.subject.en | Unsupervised Learning AI Ethics Trust Fairness | pl |
dc.subject.en | explainability | pl |
dc.subtype | ConferenceProceedings | pl |
dc.title | Explaining self-supervised image representations with visual probing | pl |
dc.title.container | Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence (IJCAI-21) | pl |
dc.type | BookSection | pl |
dspace.entity.type | Publication |
* The migration of download and view statistics prior to the date of April 8, 2024 is in progress.
Views
1
Views per month
Views per city
Downloads
Open Access