The EWGRD Round Robin Measurement Exercise

This paper presents the results of a round robin exercise carried out to compare specific activity measurements performed by eight European organisations on a set of ten neutron activation detectors containing the radio-nuclides 110mAg, 60Co, 54Mn, 46Sc and 94Nb. The purpose of the exercise was to demonstrate the level of consistency between the participating organisations in blind tests of measurements relevant to reactor metrology. The samples used were selected from a stock of pre-existing irradiated material held at SCK•CEN. Taking turns over a period of approximately 9 months, the participating organisations received the samples, measured them and provided their results to an independent referee who collated and compared the data. The inter-comparison has demonstrated good agreement between the participants with standard deviations for each dosimeter varying between 1.6% and 3.1%. The paper provides results of the EWGRD Round Robin in an anonymised form together with discussion and conclusions which may be drawn from the exercise.


Introduction
Experimentally determined neutron activation reaction rates continue to perform an integral role within assessments of reactor dosimetry being routinely used to support, adjust and validate the multiplicity of calculational tools.It is therefore essential that such measurements are determined without systematic bias and with well-characterised uncertainties.Relying as they do upon gamma radiation spectrometry methods to determine neutron activation reaction rates in-situ, it is important for the reactor dosimetry community to establish the capabilities of the analysts and the facilities used.Benchmarks and blind tests are usually proposed for this purpose.
In response, the European Working Group on Reactor Dosimetry (EWGRD) has recently commissioned a round robin exercise to compare neutron activation measurements performed by eight European organisations.The aim of the exercise was to carry out measurements of the specific activities of commonly used radio-nuclides in a selection of ten irradiated neutron activation samples, also referred to as "dosimeters".The selected set of dosimeters comprised two each of silver, cobalt, iron, titanium and niobium: the activities measured were 110m Ag, 60 Co, 54 Mn, 46 Sc and 94 Nb.
Once the set of activated dosimeters had been established, the exercise began with their measurement at SCK•CEN who, on completion, forwarded the entire set to the facilities of the next participant and so on.In total eight organisations/institutions took part in the exercise as follows: • Budapest University of Technology and Economics (BME) On completion, each participant was asked to forward its responses to an independent EWGRD member from a non-participating organisation who collated the data and acted as referee.

Production of Samples
The dosimeters were prepared at SCK•CEN, Mol and were chosen from the inventory of irradiated materials arising from previous experiments at Mol.They were chosen on the basis of material, mass and dimensions in order to include different self-absorption and geometry corrections in the subsequent analyses.It was intended that the dosimeters would have an original activity of ∼5 kBq or more to avoid excessive counting times but also to have activity levels low enough to meet IAEA requirements for exempt transport, thereby simplifying transport arrangements between participants.
Table 1 lists the details of the EWGRD Round Robin samples provided by SCK•CEN.These details were made available to each participant at the outset of the exercise.

Measurements
Once the set of activated dosimeters had been established, the exercise began with their measurement at SCK•CEN in April 2012.The final recipient, CEA Valduc, completed their measurements in December 2012 and returned the samples to SCK•CEN for disposal.Thus all measurements were completed within a period of 9 months.All results were normalised to the reference date of 1 April 2012, 00:00 GMT.
Participants were requested to determine the following data for each dosimeter: mass (g), activity (Bq), specific activity (Bq/g) and associated uncertainties (at one standard deviation, 1 ).In addition, participants were asked to provide brief details of the experimental technique used including type of detector, calibration technique, acquisition system, nuclear data (gamma yields, half-lives etc.) and, if required, the magnitude of corrections (gamma-transparency, dead-time, coincidence summing, etc.).Table 2 provides a brief summary of the information provided to the referee.Please note: Table 2 reflects only the data provided to the referee and it is not the intention of this paper to describe the approach of each contributor consistently.Notwithstanding, it is evident from the information in Table 2 that there is considerable variation in the approaches taken, especially with regard to calibration, correction and uncertainties.

Protocol
As soon as results from participating organisations had been received, the referee performed an initial comparison to test for consistency.This first comparison identified potentially significant inconsistencies affecting 17 of the responses from amongst four of the participants.On the basis that the objectives of the exercise would be better fulfilled by a comparison without avoidable error, the EWGRD agreed to write to the organisations that had submitted the identified results.Those organisations were informed that there were potential discrepancies in their data and were invited to check, clarify and, if necessary, re-submit their results.This was done without identifying which measurements to address or how the results were affected.In response, three of the four affected organisations resubmitted their data citing errors in post-processing whilst the other provided further clarification without revising results.Thus, in the end errors were revealed in 13 of the 80 original data (16%) from 3 of the 8 organisations (38%).Once corrections were received, the comparisons proceeded as planned and there was no further discussion of results with any of the contributors.

Results
Results of the final specific activity measurements are summarised in Table 3.For each sample, the table gives the mean and standard deviation of the 8 individual results provided by the participants.The table also provides the maximum and minimum values for each sample and, under the headings "No.sig", the respective deviations of these extreme values from the mean expressed as multiples of the associated sample standard deviations ( sample ).
Presented in the same format, Table 4 provides a summary of the sample mass data provided to the referee.
Table 5 shows how individual respondents fared on each of the samples.The data are presented anonymously and are expressed as the ratio of each result to the mean of the eight measurements obtained for the sample, i.e.R i,j / < R i,j > j where R i,j is the measurement of sample i by organisation j and < R i,j > j is the mean of all measurements of sample i.These are referred to henceforth as "normalised values".The table also provides the mean value and standard deviation ( org ) of the normalised values for each organisation which are compared with the typical measurement uncertainty of each organisation (bottom row) which, for simplicity, has been calculated as the mean of the 06005-p.3organisation's sample measurement uncertainties.Figure 1 is a graph of the data contained in Table 5 which also shows the uncertainties on individual measurements as well as the positions of ±1 sample and ±2 sample (dashed lines).

Mass Data
The results presented in Table 4 show that mass data supplied by the eight contributors are all in excellent agreement: sample standard deviations ( sample ) vary between 0.01% and 0.84% with all 80 data lying within ±2.5 sample of their associated mean values, i.e. there are no outliers.Not all contributors provided uncertainties for mass measurements, so for brevity, individual mass uncertainties are not presented.However, it can be reported that the uncertainties that were provided ranged from negligible to 1.46% with the majority of values at ∼0.4% broadly consistent with the statistics given in Table 4.
In summary, it can be seen that the organisations are all providing mutually consistent mass values with associated uncertainties appropriately estimated, where provided.It is also clear that for samples with masses comparable with those measured here, mass measurement uncertainty may comprise a significant contribution which must be taken into account in the overall uncertainty assessment.

Specific Activity Data
Specific activity results presented in Table 3 show that there is good agreement between the contributors for each of the samples.Sample standard deviations ( sample ) vary between 1.6% and 3.1% and all 80 data lie within ±1.8 sample suggesting a modestly tighter than expected distribution, compared to sample , and no outliers.This is reflected in the normalised results which show maximum and minimum values of 0.95 and 1.05.
For the sake of brevity, individual measurement uncertainties are not tabulated in this paper.However, they are shown in Fig. 1  and 3 to 8, expression of the differences compared to the mean relative to the individual measurement uncertainties gives the following distribution for the 70 measurements: Although the measurement uncertainties appear reasonable compared with the discrepancies obtained (Table 5), this distribution is broader than would be expected from a normal distribution probably because of the systematic nature of measurement uncertainties and correlation of discrepancies (see Fig. 2).The apparent asymmetry (i.e.64% > 0, 36% < 0) results from the omission of Organisation 2 from the distribution.Means of the normalised values given in Table 5 indicate the average level of agreement versus the mean for each organisation: they vary between 0.974 and 1.032.This indicates discrepancies broadly comparable with the average measurement uncertainties given at the bottom of the table, as expected.However, what is also apparent is that the variability evident in the organisations' normalised data (standard deviation, org ) is noticeably smaller than the sample standard deviations sample , varying between 0.6% and 2.3% but being in the main no more than ∼1%.This demonstrates the influence of correlated systematic uncertainties between the measurements performed by individual organisations.Another demonstration of this is seen in Fig. 2 which graphically compares the mean normalised values with error bars representing the uncertainty on the mean org / √ 8.This shows that when the statistical variability of the data is reduced, there are modest but significant systematic differences between the participants.
It is emphasised that the differences described above are all comparable with the declared uncertainties (of which most of the contributors acknowledge systematic components) and do not indicate anything untoward in the data provided by the eight organisations.However, the findings do illustrate a potential pitfall in the use of such measurements in support of calculational dosimetry assessments.It is credible that an analyst may use a set of plant activation measurements (similar to those described here) to validate and/or adjust his/her data on the basis that measurements obtained from different dosimeters are independent and uncorrelated.As seen here, if supplied by a single laboratory they may, in reality, be highly correlated.In such a case, the analyst may attribute undue confidence to the ensuing dosimetry assessment which may have correlated systematic uncertainties across the whole of the assessed neutron spectrum (i.e.fast and thermal).Of course, an adjustment process appropriately addressing this correlation would avoid such a pitfall but the challenge remains to obtain the right covariances.
Whilst they are insufficient to be conclusive, the data plotted in Fig. 2 leave open the possibility that the results of some of the contributors may also be correlated (e.g.Organisations 2, 3 and 8 are all on average about ∼2% lower than the mean).Such correlations would be more likely to occur if organisations were to use common standards in calibration (calibration is the dominant source of systematic uncertainty).Lesser effects could result with the use of common nuclear data or correction factors.In the present exercise, however, the information provided by the participants (Table 2) does not suggest sufficient commonality in approach.Nevertheless this serves as a useful reminder to the dosimetry community to be vigilant against the effects of correlation even when not expected.

Quality Assurance
There were initial errors affecting 16% of the original data set affecting 38% of participants.These errors were identified and resolved with a minimum of communication between the referee and participants and were wholly attributable to trivial computational errors in post-analysis.Typical examples of these errors included: errors in spreadsheet formulae (e.g.incorrect cell references), and numerical errors in data entry.
It is concluded, therefore, that the -spectrometry technologies used by the participants were applied without technical issue.However, the presence of errors in post-analysis demonstrates the necessity of checking all aspects of neutron activation analysis and the role to be played by effective Quality Assurance.
Blind trials such as this round robin exercise pose a greater challenge to the analyst than many routine applications because discrepancies are difficult to spot.The measurements were performed on single radio-nuclides within samples independently produced in different neutron irradiations; thereby providing no corroboration for the analyst.The initial error rate obtained here may therefore be interpreted as an indication of the performance of organizations in the most challenging circumstances.

Summary and Conclusions
The European Working Group on Reactor Dosimetry has undertaken a round robin exercise to compare neutron activation measurements performed by eight European organisations on a set of ten irradiated dosimetry samples containing the radio-nuclides 110m Ag, 60 Co, 54 Mn, 46 Sc and 94 Nb.The results presented anonymously in this paper show that: • The eight organisations have provided consistent measurements of specific activity for each of the ten samples although modest systematic differences are observed.• The eight organisations also provided consistent measurements for sample mass.
• Compared to sample mean specific activity values, each of the 80 data reported lie within ±5% • Standard deviations of the results lie between 1.6% and 3.1% for each of the ten samples and are comparable with organisations' assessed uncertainties.• The normalised data for each organisation show noticeably smaller standard deviations than the data for each sample indicating the presence of correlation within each organisations data.• Discrepancies apparent in the initial results of these blind tests were wholly attributable to trivial errors in post-processing and not to the use of the applied -spectrometry technologies.
The EWGRD would like to thank the participating authors and their organisations for their support in the completion of the exercise and the production of this paper.In particular, the authors would like acknowledge the support of SCK• CEN whose commitment and initiative made the exercise possible.

Figure 2 .
Figure 2. Graphical Comparison of Mean Normalised Values Obtained form Each Organisation.

Table 1 .
Properties of EWGRD Round Robin Samples.

Table 2 .
EWGRD Round Robin: Summary of Measurement Details Provided to Referee (continued on next page).

Table 2 .
EWGRD Round Robin: Summary of Measurement Details Provided to Referee (continued from previous page).

Table 3 .
Results of Specific Activity Measurements.

Table 5 .
Specific Activity Results by Organisation (normalised values).
Graphical Comparison of Normalised Values Obtained from Participant's Responses.(Note: horizontal coordinates, i.e. sample number, have been artificially separated to aid visibility.)