Extreme entropy machines : robust information theoretic classification

doi:10.1007/s10044-015-0497-8

Extreme entropy machines : robust information theoretic classification

Communities & Collections

pcg.skipToMenu

Extreme entropy machines : robust information theoretic classification

View Usage Statistics

Bibliographic description

punktacja MEiN [2017 A]: 20

title:	Extreme entropy machines : robust information theoretic classification
author:	Czarnecki Wojciech , Tabor Jacek
journal title:	Pattern Analysis and Applications
volume:	20
issue:	2
date of publication :	2017
pages:	383-400
ISSN:	1433-7541
eISSN:	1433-755X
DOI:	10.1007/s10044-015-0497-8
language:	English
journal language:	English
abstract in English:	Most existing classification methods are aimed at minimization of empirical risk (through some simple point-based error measured with loss function) with added regularization. We propose to approach the classification problem by applying entropy measures as a model objective function. We focus on quadratic Renyi’s entropy and connected Cauchy-Schwarz Divergence which leads to the construction of extreme entropy machines (EEM). The main contribution of this paper is proposing a model based on the information theoretic concepts which on the one hand shows new, entropic perspective on known linear classifiers and on the other leads to a construction of very robust method competitive with the state of the art noninformation theoretic ones (including Support Vector Machines and Extreme Learning Machines). Evaluation on numerous problems spanning from small, simple ones from UCI repository to the large (hundreds of thousands of samples) extremely unbalanced (up to 100:1 classes’ ratios) datasets shows wide applicability of the EEM in real-life problems. Furthermore, it scales better than all considered competitive methods.
keywords in English:	rapid learning, extreme learning machines, random projections, classification, entropy
affiliation:	Wydział Matematyki i Informatyki : Instytut Informatyki i Matematyki Komputerowej
type:	journal article
subtype:	academic paper