EUCLID: A New Approach to Constrain Nuclear Data via Optimized Validation Experiments using Machine Learning

,


Introduction
Unconstrained physics spaces occur between several nuclear data observables when their values can be simultaneously adjusted without violating the uncertainties in either differential information or simulations of integral experiments.These unconstrained physics spaces can hide compensating errors.For instance, simulated results of the neutron multiplication factor (k eff ) of the Jezebel benchmark [1] with ENDF/B-VIII.0[2] and JEFF3.3 [3] are nearly the same (both within one standard deviation of the experimental value).However, there are large differences in their 239 Pu nuclear data that are within the differential uncertainties; one example of this is that the inelastic scattering cross section differs by more than 25% for some incident neutron energies where experimental data are scarce.Figure 1 shows results obtained when exchanging ENDF/B-VIII.0data for JEFF3.3 for a single energy group and a single reaction.These differences can be large: for inelastic scattering, the difference between the two libraries is 5.5 times the experiment standard deviation (123 pcm).However, if one integrates the red and the blue across all reactions, they cancel each other out, highlighting the unconstrained physics space between 239 Pu nuclear data.
Compensating errors hiding in these unconstrained physics spaces adversely impact the predictive power of application simulations, if there are no adequately similar validation experiments.While this anecdotal example illustrates the concept, EUCLID (Experiments Underpinned by Computational Learning for Improvements in Nuclear * e-mail: jesson@lanl.govData) developed a formalized process to identify unconstrained physics spaces in nuclear data libraries leveraging machine-learning and expert judgement [4].
The EUCLID project's main goal is to design and execute integral experiments to reduce unconstrained physics spaces; it has focus areas related to nuclear data, radiation transport simulations, machine learning, and criticality experiments.This paper will give a high-level overview of each of the focus areas.
A simplified overview of the project is shown in Figure 2. In the top-center of this figure, we will start with the current U.S. nuclear data library, ENDF/B-VIII.0.Then we will utilize sensitivity methods within radiation transport codes, such as MCNP ® Code Version 6.2 1 [5] to simulate existing validation experiments.Next, we will utilize machine learning to augment expert identification of areas where compensating errors may be hiding [4].New experiments will be optimally designed using machine learning to target these regions (shown in the bottom-right).After the new experiments are completed, the new data are used to produce adjusted libraries with reduced compensating errors while accounting for physics constraints.

New experiments with multiple responses
Nuclear data can be better constrained by validating them with multiple responses, including, but also going beyond, k eff .Critical experiments are the foundation of nuclear data validation due to the low uncertainties associated with these types of experiments (often <0.5% uncertainty including both statistic and systematic contributions).Validation of nuclear data and analytical methods have taken place using critical experiments since the 1940s.One drawback, however, of critical experiments is that k eff includes integrated contributions from many reactions simultaneously (hence the term integral experiment).Therefore, fully constraining nuclear data via k eff is impossible (i.e.there are many solutions that match the differential and integral experiments equally well).This work investigated eight types of responses: critical experiments, LLNL pulsed-sphere neutron-leakage spectra, neutron multiplicity measurements, Rossi-α, reaction rate ratios, neutron leakage spectra, reactivity coefficients, and delayed neutron measurements.The bottomleft photograph in Figure 2 shows the types of systems that can be used to measure many of these responses.In order to utilize validation experiments for nuclear data adjustment (described in Section 5) or experiment optimization (Section 4), three things are needed: 1. measurement results and uncertainties, 2. simulated results and uncertainties, 3. simulated nuclear data sensitivities.
Our team assembled and computed all of these items, available for over one thousand previous benchmark experiments.These are contained within a sensitivity library, which will be released by the Organization for Economic Co-operation and Development/Nuclear Energy Agency (OECD/NEA).Results from each response have been published in other works: LLNL pulsed spheres [6,7], neutron multiplicity measurements [7], Rossi-α [8,9], reaction rate ratios [10], neutron leakage spectra (coming to ANS Winter 2022), reactivity coefficients [11], and delayed neutron measurements [12]. 1 MCNP ® and Monte Carlo N-Particle ® are registered trademarks owned by Triad National Security, LLC, manager and operator of Los Alamos National Laboratory.Any third party use of such registered marks should be properly attributed to Triad National Security, LLC, including the use of the designation as appropriate.For the purposes of visual clarity, the registered trademark symbol is assumed for all references to MCNP within the remainder of this paper.

Simulating sensitivities for multiple response types
Although it is straight-forward to simulate the responses discussed in Section 2, capabilities to easily generate nuclear data sensitivities are limited.Even with recent advances, performing finite-difference calculations (e.g., by creating new perturbed nuclear data files [9]) is time consuming.Therefore, recourse to other methods was taken to generate such nuclear data sensitivities.Table 1 lists the methods that have been used to date to simulate sensitivities for each of the response types investigated in this work.All methods have their own pros and cons.For instance, Monte Carlo (MC) codes typically have an advantage over deterministic codes related to the types of geometries that can be simulated.One downside of MC codes, however, are the larger simulated uncertainties in the sensitivities (i.e., computationally expensive).The methods used for each response type will change as new capabilities are incorporated over time.Some of the methods listed have not been utilized within the sensitivity library and their use is currently being investigated.Simulation capabilities for nuclear data sensitivities were discussed in a separate work at this meeting, which also focused on a new fixed-source sensitivity (FSEN) capability within MCNP [13].

Optimizing experiments to reduce compensating errors
Almost any integral experiment will serve as a constraint on nuclear data, but the ideal experiment for reducing compensating errors is one that minimizes the overall uncertainty in the region where errors are expected.By reducing the uncertainty, the space where nuclear data can be adjusted without violating the differential data is limited.We formulate the experimental design problem as an optimization problem to minimize the log-determinant of the adjusted covariance, a criterion called D-optimality [14].
With the responses described in Section 2 and the sensitivities in Section 3, we can predict the adjusted nuclear data covariance Σ ′ with the generalized linear least squares (GLLS) model where y e are experimental observations, y c are computational predictions, S is the sensitivity matrix, Σ e is the expected experimental covariance matrix, Σ c is the computational covariance matrix, and Σ is the prior nuclear data covariance matrix.The optimization problem is arg min where S is the space of sensitivity matrices spanned by the space of potential experiments and responses.The solution to equation 3 is a set of experiments that minimize the credible region of the adjusted nuclear data and minimizes the expected entropy of the posterior nuclear data  distribution.The D-optimality criterion allows the combination of multiple responses, unlike one-at-a-time metrics, such as nuclear data similarity, c k , [15] by accounting for the sensitivity similarities and experimental correlations of different responses.D-optimality measures the information content of a set of responses and helps to screen unproductive responses before an experiment.
The optimization problem in equation 3 is noisy, expensive, and often constrained.We apply Gaussian process optimization that models the D-optimality criterion and sequentially maximizes it [16,17].This approach to optimization eliminates the need to perform large parametric studies of the design space, allowing quick testing of experiment geometries and material selections while minimizing the number of MCNP evaluations.Multiple rounds of optimization are needed in designing the experiment as we increase experiment fidelity.At each stage, we can measure how safety and engineering modifications affect the uncertainty reduction through changes in D-optimality.

Adjusting nuclear data using multiple response types
EUCLID has developed a web-applet to perform nuclear data adjustment.This adjustment is performed using GLLS (Equations 1-2), and incorporates all 8 of the response types described in Section 2. In addition, the adjustment utilized 1 H, 9 Be, 12 C, 27 Al, 56 Fe, 235,238 U, 239,240 Pu nuclear data and covariances.Adjustment was used to generate the nuclear data prior for optimizing the integral experiment in Section 4. In this way, the prior does not only contain differential constrains in the form of nuclear data mean values and covariances, but also information from all integral responses we deemed reliable enough to be fed into the optimized design as prior integral-experiment knowledge.Adjustment will also be used for our final assessment whether we succeeded in reducing uncertainties between 239 Pu nuclear for the fission, inelastic scattering, elastic scattering, and (n,γ) cross sections as well as ν and prompt fission neutron spectrum.The sensitivities, uncertainties, and bias associated with each response will result in changing the adjusted nuclear data.Recent results have shown that some response types can help constrain nuclear data in different ways than critical experiments alone.These results are not shown here due to page constraints, but were shown in the presentation and will be the focus of future work [18].

Conclusions and future work
The EUCLID project will reduce compensating errors in nuclear data through advancements in criticality experiments, radiation transport simulations, machine learning, and nuclear data.Currently a sensitivity library with eight different response types has been generated and is being used to identify unconstrained physics spaces, where compensating errors could hide, for experiment design, and nuclear data adjustment.
EUCLID has an experiment design which is currently being performed at NCERC [19].This experiment includes two configurations that have greatly different scattering sensitivities.One configuration utilizes a large mass of Pu in a geometry that maximizes leakage (and therefore minimizes scattering sensitivities) while the other configuration essentially minimizes leakage (and has much larger scattering sensitivities).In addition to k eff , many other responses will be measured for each configuration.After the experiment is complete, a nuclear data adjustment will be performed using the new measurements, while incorporating physics constraints.
While the EUCLID project is targeting 239 Pu reactions, a similar approach could be applied to other nuclides of interest.Compensating errors between different nuclides could also be explored.

Figure 1 .
Figure 1.Differences in k eff shown as individual reaction and energy-dependent 239 Pu ENDF/B-VIII.0nuclear data are replaced with JEFF-3.3 nuclear data.Energy-integrated changes, given in blue/red values at the top are provided as multiples of the experiment standard deviation (123 pcm).

Figure 2 .
Figure 2. The EUCLID project will design validation experiments optimized to resolve compensating errors and adjust nuclear data to experiments.

Table 1 .
Methods used to simulate nuclear data sensitivities for various responses.
* Requires combining KSEN results of multiple files