Understanding the impact of nuclear-data covariances on various integral responses using adjustment

.


Introduction
Nuclear-data covariances capture the limited knowledge we have on nuclear data due to approximations made to derive nuclear models, and limited precision of differential experimental data. These covariances are a vital ingredient to quantify economical and safety bounds on various application simulations due to imprecise nuclear data. Integral responses, such as the effective neutron-multiplication factor, k eff , describe such application simulations on a small scale and validate the nuclear data for the application space the responses represent.
There has been a strong push, recently highlighted in Ref. [1], by nuclear data producer and user communities to include more integral responses than k eff into the validation and adjustment of nuclear data. The reason for that request is that including more responses in the validation enables studying how nuclear data would perform for a broader application space. Also, when one uses many integral responses for validation and adjustment, some compensating errors [2] between various nuclear-data observables can be uncovered. Compensating errors are often introduced in nuclear data by calibrating several nuclear-data observables at once by hand with respect to k eff [3][4][5]. The effective neutron multiplication factor queries nuclear-data observables in the same way. More explicitly, the ratios of sensitivities of k eff to two nuclear-data observables of the same isotope are very similar in the same energy range across different assemblies. For instance, the ratios of sensitivities to fission cross sections and average prompt fis- * e-mail: dneudecker@lanl.gov sion neutron multiplicity in the same energy range are very similar from one assembly to another if they are of the same class. Therefore, mistakes in one nuclear-data observable can offset mistakes in other observables. Some integral responses beyond k eff were, however, shown [6][7][8][9][10][11][12] to have different sensitivities to the same nuclear data compared to those of k eff . Given the different usage of nuclear data to simulate various responses, there is a smaller nuclear-data phase space where errors in one observable can compensate for others, while still predicting the experimental values of all responses within their uncertainties.
The EUCLID (Experiments Underpinned by Computational Learning for Improvements in Nuclear Data) LDRD-DR (Laboratory Directed Research & Development-Directed Research) project [13] explored how nuclear data is used when simulating nine different integral responses, measured from LLNL pulsed spheres, critical or sub-critical assemblies, by calculating sensitivity profiles of these responses to nuclear data [6][7][8][9][10][11][12]. At the same time, ENDF/B-VIII.0 mean values and covariances [14] were processed. The combination of these data-sensitivities, mean values and covariancesallowed us to quantify the bounds on the simulated integral responses due to nuclear-data covariances. We also used these data to adjust ENDF/B-VIII.0 nuclear data with respect to these integral responses.
Section 2 summarizes briefly how bounds on integral responses due to nuclear data were quantified along with the adjustment methodology. Section 3 reviews the integral responses considered (LLNL pulsed-sphere neutronleakage spectra [15], k eff , fission reaction rates and re-activity coefficients of ICSBEP critical assemblies [16]), their sensitivities and what nuclear data were used for the study. The results of assessing nuclear-data bounds on integral experiments and adjustment are discussed in Section 4 along with mentioning necessary extensions of the work. A brief summary, conclusions and outlook are provided in Section 5.

Forward-propagation of nuclear-data covariances to integral bounds
Nuclear-data covariances were forward-propagated to bounds on integral quantities ∆ ND C with the well-known sandwich formula [17]: where S C,i is the vector of sensitivities with respect to an integral values C for nuclear-data observable i. It is a vector because sensitivities are given for each nuclear-data observable i on a 51-energy grid. The variable Cov ND i,j denotes nuclear data covariances between observables i and j on the same energy grid as S C,i .
We consider the following observables i and j: The neutron-induced fission, elastic, inelastic, capture, (n,2n), (n,3n), (n,4n), (n,p), (n,α) and (n,t) cross section along with the prompt-fission neutron spectrum (PFNS) and the average total-fission neutron multiplicity, ν, if both Cov ND i,j and S C,i are available. We consider covariances for the total cross section implicitly. I.e., we do not include the total covariances explicitly in the adjustment but rather represent it by the cross-section covariances that contribute to it. Otherwise, this would amount to double-counting uncertainties due to the sum-rule applying to the total cross section and other contributing cross sections. The same reasoning applies to why we use only covariances and sensitivities for the ν, rather than the prompt one. We chose to use covariances and sensitivities for ν as often no covariances are supplied for the average delayed-fission neutron multiplicity. Also, covariances are rarely, if at all, supplied for the delayed-fission neutron spectrum, and never for the total-fission neutron spectrum. Hence, only PFNS covariances and sensitivities were used. The effect of missing the delayed component is expected to be small as the PFNS dominates the total-fission neutron spectrum.
If S C,i is defined to be relative to the nuclear data as well as C, the covariances also need to be divided by the respective nuclear data. Then the resulting ∆ ND C is given also relative to C.
Forward-propagating covariances to provide bounds on integral responses allows us to understand the uncertainties on predicted quantities due to nuclear data. It can help understand if a simulated value is within nucleardata uncertainties or far beyond. If it is far beyond, this can either point to a shortcoming in experimental data, or nuclear-data mean values and covariances.

Adjustment methodology
Adjustment of nuclear data was undertaken with the generalized linear least squares technique (GLLS) as, for instance, used by the LANL Whisper [18] code. Adjusted nuclear data mean values σ * and covariances Cov * are obtained by: with The variables E, C, and σ N D correspond to vectors of experimental and calculated integral values as well as nuclear data. The matrix Cov e is the covariance matrix of integral experimental data. An online tool was developed by M. Grosskopf as part of EUCLID to perform adjustment [19].

Integral responses
The following four integral responses were considered: • The effective neutron multiplication factor, k eff , of over 1,000 ICSBEP [16] critical assemblies. More specifically, we will study here plutonium metal fast (PU-MET-FAST), metal intermediate (PUT-MET-INT), and solution thermal (PU-SOL-THERM) assemblies. k eff experiments are usually considered to be the gold standard for nuclear-data validation; they are well-vetted and provide reasonable experimental uncertainties (in the range of 0.05-0.5%). These experiments are frequently used to validate nuclear data up to 5 MeV. In the case of ENDF/B-VIII.0, a sub-set of these integral data are used to tweak nuclear data by hand (within the limits of differential experimental data) to better predict k eff [14].
• The neutron-leakage spectra emitted from Pu LLNL pulsed spheres. The spheres were pulsed by a 14-MeV D+T neutron source in the center of the sphere [15]. Experimental data are provided as a function of time-offlight (TOF), and for different material thicknesses, angles, and neutron detectors. These experiments extend the energy range of validation of around 5 MeV reached by k eff to 15 MeV. However, these experiments are suspected to be biased [8,20]; their uncertainties (0.5-2%) are not rigorously quantified.
• Fission reaction rates measured in the Jezebel, Godiva, Flattop-HEU, and Flattop-Pu critical assemblies are employed as well [7]. These measurements were designed to study the PFNS and the reactions measured. Experimental uncertainties are in the range of 0.8-2%. Questions have been raised on understanding the exact measurement value and surroundings reported for these experiments [21,22].
• Reactivity coefficients, measured in five different locations in the Jezebel critical assembly [23] have been considered. Configuration a corresponds to a void measurement (no sample), and b with a small sample of a defined material that does not change the spectrum of the assembly. Because of the ratio type of these data, some systematic uncertainties are reduced. A blanket uncertainty of 7.5% was added nonetheless to all reported experimental uncertainties following the IRPhEP uncertainty guide [24].

Sensitivities of integral responses to nuclear data
Sensitivities for k eff were calculated with the "ksen" card in MCNP ® Code Version 6.2 1 [25]. Sensitivities for LLNL pulsed-sphere neutron-leakage spectra were also calculated with MCNP but with a brute-force methodology [6,8], while "ksen" sensitivities were used to back out sensitivities for ∆ρ [6,11]. Reaction rate sensitivities were computed via SENSMG [26]. All sensitivites were computed on the same 51-energy grid and for a subset of relevant reactions (neutron-induced fission, elastic, inelastic, capture, (n,2n), (n,3n), (n,4n), (n,p), (n,α) and (n,t) cross section along with PFNS and ν) for all isotopes that non-negligibly appear in the experiment. Sensitivities of k eff to up to the P5 Legendre moment for angular distributions (MF=34) were also calculated, but were not used as these could not be computed at this time for other responses either due to the prohibitive number of Monte Carlo runs required for brute-force methods or limitations in codes to provide those sensitivities.

Nuclear data mean values and covariances
ENDF/B-VIII.0 mean values and covariances [14] were processed with the code NJOY2016.67 [27] onto a 51energy grid. PFNS covariances were processed with separate NJOY/ERRORR runs per incident-energy grid, E inc , for those E inc given in ENDF/B-VIII.0 using the IWT=9 option (CLAW spectrum) to collapse to the 51-(outgoing) energy grid maintaining the original E inc . PFNS covariances were then expanded (assuming full correlations between the PFNS belonging originally to one E inc bin) to the E inc of the sensitivities. The data were processed into a JSON format for easier readability [6]. In the current analysis, we consider ENDF/B-VIII.0 covariances for 1 H, 9 Be, 12 C, 27 Al, 56 Fe, 235,238 U, and 239,240 Pu isotopes. In future studies, this will be extended to include covariances for all isotopes that are available in ENDF/B-VIII.0.
1 MCNP ® and Monte Carlo N-Particle ® are registered trademarks owned by Triad National Security, LLC, manager and operator of Los Alamos National Laboratory. Any third party use of such registered marks should be properly attributed to Triad National Security, LLC, including the use of the designation as appropriate. For the purposes of visual clarity, the registered trademark symbol is assumed for all references to MCNP within the remainder of this paper.

Bounds on integral data due to nuclear-data covariances
Bounds of all four studied responses are shown in Figs. 1-4 based on experimental data only as well as including in quadrature Monte-Carlo simulation and nucleardata uncertainties. The latter were obtained by forwardpropagating nuclear-data covariances via Eq. (1). One key observation is that most of the C/E values for k eff lie within the 1-sigma nuclear data uncertainties. That is more than the expected 68% if one would assume that C/E k eff values follow an independent Gaussian distribution. Previously, this was interpreted as that nuclear-data covariances were over-estimated with regards to the spread in C/E of k eff [28]. However, this good agreement reflects two facts [14]: • ENDF/B-VIII.0 mean values were tweaked by hand to better predict k eff of a sub-set of the experimental data shown in Fig. 1. This effectively breaks the assumption that the data should follow an independent Gaussian distribution as nuclear data were manipulated to enforce agreement, • Moreover, ENDF/B-VIII.0 covariances remained unchanged and only reflect the knowledge coming from differential experimental data and theory.
Due to that it is justified that the spread of C/E of k eff lies well within the uncertainties encompassing nuclear-data covariances.  Compared to k eff , a reasonable amount of C/E values fall outside of the 1-sigma range for reaction rates (Fig. 2). ENDF/B nuclear data are usually validated with respect to these measurements [14], but most of the values are trending to be low which was already noticed in Ref. [29].
Hence, studies on these data are currently ongoing [22] to understand if either an adjustment of nuclear data is needed or whether experimental biases might be at play.  The C/E agreement of the Pu LLNL pulsed spheres is imperfect below 185, from 205 to 235 and above 480 ns in Fig. 3. It is difficult to model the neutron source accurately impacting possibly C/E values below 185 ns. The discrepancy in C/E from 205 to 235 ns was resolved in Refs. [30][31][32] by improving the 239 Pu nuclear data for the continuum inelastic cross section and angular distribution. Above 480 ns, it is likely that we see the impact of background or detector efficiency biasing the experimental data. It is also noteworthy that nuclear-data uncertainties on the spectrum from 173 to 187 ns seem underestimated. The spectrum in this TOF range is mostly sensitive to elastic and inelastic scattering distribution. Neither sensitivities, nor in some cases covariances, exist for these observables leading to the underestimated nuclear-data uncertainties.
Reactivity coefficients measured in Jezebel are infrequently used for validating nuclear data. Hence, it is not surprising to see C/E values outside of the 1-sigma range in Fig. 4. Still, it is obvious that there are some experimental outliers, such as for BeO or tungsten samples, while the agreement of C and E for ∆ρ using Pu samples is distinctly better. The experimental data of ∆ρ need to be further explored.

Adjustment
If we adjust nuclear-data mean values and covariances with respect to k eff of Jezebel via Eq. (2), the evaluated 239 Pu nuclear data change minimally in Fig. 5, while the evaluated 239 Pu uncertainties decrease distinctly. These results illustrate again that ENDF/B-VIII.0 mean values were tweaked by hand to reproduce Jezebel k eff ; hence, any remaining possible adjustment of cross sections is small. ENDF/B-VIII.0 covariances, however, remained unchanged in this processing of tweaking the mean values. Therefore, adjusted uncertainties reduce significantly when the physics is constrained by Jezebel k eff .
If we adjust with respect to a sub-set of PU-MET-FAST, PU-MET-INT and PU-SOLT-THERM k eff values, the evaluated mean values change more distinctly (by more than a factor of 10) compared to adjustment with only Jezebel k eff (Fig. 6). However, the proposed changes in evaluated data are well within the evaluated nuclear-data uncertainties of the respective reactions. Hence, here we see the effect of adjusting to k eff values that were not as closely matched by hand-tuning nuclear data. The overall change of evaluated uncertainties in the fast range is comparable to adjusting to k eff of Jezebel. This (EUCLID) sub-set of assemblies was chosen by identifying those that had reliable k eff values and increasing uncertainties when suspected to be underestimated [33]. In a second step, we down-selected those benchmark series with multiple, strongly correlated configurations to consider only one k eff value per series in the adjustment to avoid double-counting these experiments as we do not consider covariances between them at this point.
The adjustment to fission reaction rates in the Jezebel critical assembly leads to comparably modest changes in evaluated nuclear data in Fig. 7. We only adjust to reaction rates in Jezebel, while bounds for fission reaction rates in Godiva, Flattop, Flattop-Pu and Jezebel are shown in Fig. 2, because adjustment was also performed for k eff and reactivity coefficients of Jezebel in Figs. 5 and 9. It is noteworthy that the fission cross section is increased by 1%, an effect seen also when including cross sections averaged over a 252 Cf spontaneous-fission spectrum [34] or similar adjustment work by R. Casperson [29]. The size of the change depends on what sub-set of reaction rates are used for adjustment. It is also noteworthy that neither ν nor the capture cross section change. The reason for that is that sensitivities of reaction rates in Jezebel to 239 Pu ν and the capture cross section are negligibly small (Fig. 8). Hence, fission reaction rates in the Jezebel critical assembly allow us to study the PFNS (adjusted, but not shown) and (n,f) cross sections independently from ν as well as scattering independent from the capture cross section. Therefore, reaction rates help in disentangling compensating errors in nuclear data introduced by k eff .
If we adjust to Pu LLNL pulsed-sphere-neutron leakage spectra or Pu reactivity coefficients in Jezebel, the changes in evaluated 239 Pu nuclear data in Fig. 9 are far beyond those seen for k eff . This goes along, for the Pu pulsed sphere, with the larger C/E values seen for the simulated and experimental spectra compared to k eff or reaction rates. A large adjustment of the inelastic cross section was expected given research in Refs. [30][31][32] that showed that the spectrum from 205 to 235 ns could be better predicted by changes in the inelastic cross sections and angular distributions. However, a change by 160% is far beyond the evaluated uncertainty of this reaction. This large adjust-   ment reflects that the inelastic cross section has to absorb, in this adjustment exercise, shortcomings in both the cross section and the angular distributions; the latter cannot be adjusted here due to missing sensitivities. Hence, adjustments of the inelastic cross sections are over-estimated.
Also, one should question such large adjustments in the nuclear data in the light of large discrepancies in C/E at late times that are likely caused by biases in the experimental data rather than the nuclear data. Therefore, the adjustment should account for possible bias in the experimental data by modeling this bias as part of the GLLS algorithm. In addition, one should consider that all pulsedsphere data are implicitly assumed to have uncorrelated uncertainties as correlations are not provided. Missing, but plausibly non-negligible, correlations could result in distinctly different adjustment results and thus conclusions. Considering correlations for experimental uncertainties, however, requires more detailed studies of these experiments. Last but not least, it is surprising to see adjustments of nuclear data (e.g., the fission cross section) down to 10 keV despite the fact that the thin Pu sphere (0.7 mean-free path) is mostly sensitive to the fission cross section from 12-15 MeV. This observation was attributed in Ref. [29] to the strongly correlated covariance matrix of the Neutron Data Standards [35] that is currently under revision.
It is surprising to see such large changes in the elastic and inelastic 239 Pu cross sections (beyond their 1-sigma evaluated uncertainties) when adjusted with Pu reactivity coefficients in Jezebel despite the C/E values of these data being in reasonable agreement. In Ref. [11], it was discussed that sensitivity profiles of reactivity coefficients to the elastic cross section are hampered by large Monte Carlo uncertainties, while fission cross section sensitivities have small uncertainties. Currently, work is ongoing within EUCLID to include uncertainties in the sensitivities in the adjustment algorithm.

Summary, conclusion, and outlook
The EUCLID (Experiments Underpinned by Computational Learning for Improvements in Nuclear Data) LDRD-DR (Laboratory Directed Research & Development-Directed Research) project [13] computed sensitivities for various integral responses. At the same time, ENDF/B-VIII.0 covariances were processed at LANL.
Here, we use sensitivities of the effective neutron multiplication factor, k eff , reaction rates, and reactivity coefficients of ICSBEP critical assemblies [16] as well as LLNL pulsed-sphere neutron-leakage spectra [15] and covariances to assess the uncertainties on simulated responses due to nuclear data. We also perform adjustment with  From these studies, one can clearly observe which integral data were used for validating ENDF/B-VIII.0: The adjustment to nuclear-data mean values with k eff and reaction rates is modest as nuclear data were changed by hand to better agree to some experimental data of these responses. Also, the calculated over experimental values, C/E, lie well within the spread of forward-propagated nuclear-data uncertainties. The evaluated covariances, however, change significantly with adjustment reflecting that ENDF/B-VIII.0 mean values are tweaked to better predict some integral data, while covariances remain unchanged and only reflect uncertainties from differential experiments and nuclear theory. It was also shown, with the example of adjusting 239 Pu nuclear data with respect to fission reaction rates in the Jezebel assembly, that one gains more insight into nuclear data by using responses beyond k eff . For instance, adjustment with Pu-assembly k eff would change all 239 Pu fission-source term along with scattering and capture nuclear data, while fission reaction rates in the Jezebel assembly lead only to adjustments in the promptfission neutron spectrum, fission, elastic and inelastic cross sections. Hence, using both responses allows to partially disentangle the Pu fission source term. We could fully disentangle it, if we had a response that would only be sensitive to one of the three fission-source-term observables.
LLNL pulsed-sphere neutron-leakage spectra and reactivity coefficients show a distinctly larger spread in C/E values compared to uncertainties stemming from nuclear data. That shows that nuclear data is not tweaked to the pulsed sphere data, but also that the experimental responses need to be better understood. In cases where the experimental data cannot be corrected, but biases are apparent, data needs to be either truncated, rejected, or these biases need to be modeled within the adjustment, to avoid undue impact on adjusted mean values and covariances. In addition to that, covariances and sensitivities are missing for elastic and inelastic angular distribution that would account for an important part of simulated pulsed-sphere neutron-leakage spectra uncertainties. It was also shown that large uncertainties on sensitivities can lead to unreasonably large adjustment of nuclear data. It was also noted that strong nuclear-data correlations, specifically those of the 239 Pu fission cross sections, can lead to adjustment of the cross section in much lower energy groups than should be affected by the measurement at hand. Lastly, no correlations are taken into account between the different experimental data as they are rarely provided. Many of the measurements could arguably have strong correlations (e.g., all data of one pulsed-sphere measurement, the different responses measured on Jezebel, k eff values for multiple, slightly differing configurations on the same critical experiment), and including these correlations has the potential to significantly change adjustment results. However, estimating reliable correlations requires a detailed uncertainty analysis that is time-intensive. As a starting point, WPEC SG-33 [36] covariances could be used.
Work is currently ongoing at LANL to include modeling the bias of experimental data and uncertainties in sensitivities in the adjustment, as well as to better understand the quality of experimental data of some responses. We will also extend the adjustment studies to include more integral responses to understand whether they yield additional understanding on the nuclear data beyond k eff . Last but not least, all studies presented here will be extended