VALIDATION OF SPALLATION MODELS: AN APPROACH∗

To understand and describe the proton induced spallation reactions, a large number of computer codes have been proposed. Various quantitative tests are used in literature to judge the agreement between model calculations and experimental data. The judgement is based on the magnitude of the deviation of the tests from their expected values in the case of the perfect agreement. However, the expected values of the tests and their standard deviations are usually not well known. Thus the conclusions may be ambiguous. It is proposed to calculate the expected values and standard deviations of the tests by Monte Carlo method and to use the tests in their standardized form.


Introduction
Proton induced spallation reactions cover a wide range of applications in various branches of physics, e.g., particle transport codes, accelerator based nuclear waste transmutation, neutron spallation targets, astrophysics etc. It becomes very important to find a theoretical model which can describe a complete set of reaction outcomes. The tests which are most frequently used in literature to judge the quality of data description by individual models are listed in Table I.
The expectation value of each test corresponds to the perfect agreement of model σ calc and experimental σ exp cross sections. However, it should be taken into consideration that the experimental data are biased by statistical errors, thus they are statistical variables. Therefore, the perfect agreement should be meant as E (σ exp ) = σ calc . (1.1) The expectation values of tests listed in the table are estimated assuming that the following equation holds which is strictly valid only for linear functions but actually serves as a reasonable approximation. The ranking of models is usually done by a comparison of the magnitude of the deviation of the test value from its value expected for the case of the perfect agreement with data. Such an approach allows to determine the relative success of different model descriptions but cannot answer the question whether the description is accurate enough to be treated as a proof of the model adequacy. This is caused by the lack of information on the standard deviations of most of the tests.

The proposed approach
In the present work, it is proposed: (i) to evaluate the expectation value of each test E(Test) and its standard deviation σ(Test) by the Monte Carlo method for the case of the perfect agreement of the model and data cross sections (defined by Eq. (1.1)), and (ii) to use instead the statistical tests listed in Table I, the standardized test evaluated according to the formula The Monte Carlo sampling of the test values is performed under two assumptions concerning experimental data: (i) The experimental data are independent Gaussian variables with standard deviation equal to statistical error of data. This assumption is fulfilled when the statistics of experimental events is large enough. When the statistics is poor, one has to use the Poisson distribution.  To check reliability of this method, the analytically derived probability distribution, expectation value and standard deviation of the H-test were compared with the quantities obtained by the Monte Carlo sampling. It may be shown that with the above assumptions the probability distribution of the H-test and its parameters have the following form: The comparison of histograms obtained with the Monte Carlo method and analytical formulae of the H-test are presented in Fig. 1. As can be seen, the agreement between analytic distributions and histograms is perfect.

Results
As an example of the proposed method, the standardized tests were calculated for the experimental spectrum of the Al(p, d) reaction at beam energy 1.2 GeV measured at ϑ = 100 • [2] with results of calculations performed by the means of INCL4.6 intranuclear cascade code coupled to SMM code. The values have been calculated separately for E d < 30 MeV and E d > 30 MeV because in the first energy region evaporation dominates, whereas for the second region the prompt emission from the cascade stage is prevailing (due to coalescence of escaping neutron and proton It is reasonable to conjecture that the probability distributions of standardized tests are close to the standard normal distribution N (0, 1) because of the central limit theorem. Therefore, standardized tests may be easily used for making the decision on acceptance of the hypothesis claiming the good agreement of the model with the data. In the present example, this hypothesis may be rejected on the significance level smaller than 0.0027.
The proposed approach allows to make objective, quantitative comparisons concerning the quality of data reproduction by different theoretical spallation models which is not possible using usual, non-standardized tests.
The author would like to thank the code developers and authors A. Boudard, J. Cugnon, J.C. David, S. Leray and D. Mancusi for providing the latest version of their Liege intranuclear cascade code INCL4.6, and D. Filges, F. Goldenbaum and B. Kamys for fruitful discussions. The support by the Foundation of Polish Science -MPD program co-financed by the European Union within the European Regional Development Fund is gratefully acknowledged.