In this paper we introduce the BIRAFFE data set which is the result of the experiment in affective computing we conducted in early 2019. The experiment is part of the work aimed at the development of computer models for emotion classification and recognition. We strongly believe that such models should be personalized by design as emotional responses of different persons are subject to individual differences due to their personality. In the experiment we assumed data fusion from both visual and audio stimuli both taken from standard public data bases (IADS and IAPS respectively). Moreover, we combined two paradigms. In the first one, subjects were exposed to stimuli, and later their bodily reactions (ECG, GSR, and face expression) were recorded. In the second one the subjects played basic computer games, with the same reactions constantly recorded. We decided to make the data set publicly available to the research community using the Zenodo platform. As such, the data set contributes to the development and replication of experiments in AfC.