Enable adaptive mesh support on libMesh tallies #3185
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
This PR allows for the use of adaptive meshes (which have an adaptive mesh refinement hierarchy) in unstructured mesh tallies which use a
LibMesh
. Adaptive meshes (unlike libMesh meshes which setallow_renumbering = false
) do not guarantee that active elements (which we want to tally on) are contiguous in memory relative to their DoF ids, which results in errors when OpenMC computes bin indices as bin are defined over[0, num_active_elem]
. This results in scrambled tallies for libMesh meshes which have an adaptivity hierarchy. An additional issue is that thelibMesh::EquationSystems
object added by OpenMC to enableLibMesh
exodus output reacts poorly to adaptive meshes, throwing errors when the mesh is refined/coarsened or dumped to exodus after an OpenMC solve.To fix the first issue, an indirection layer is added which maps between element DoF ids and bin indices. This is generated when
LibMesh
is initialized based on a constant adaptivity flag (adaptive_
). The indirection layer is used inget_bin_from_element(...)
andget_element_from_bin(...)
only whenadaptive_ = true
. This should result in no performance changes for existing users that use libMesh-based tallies without adaptivity, though it does result in a small (but noticeable) decrease in tally performance on adaptive libMesh meshes compared to creating a deep copy of the mesh which only contains active elements (due to cache misses).The second issue is resolved by constructing and setting up the
libMesh::EquationSystems
class inLibMesh::add_score(...)
if it hasn't been initialized.I've tested these changes in Cardinal to make sure adaptive mesh tallies work, and the results generated are equivalent to the previous approach we took. Regression tests for libMesh tallies also pass on my machine, so OpenMC users should be unaffected by this fix for adaptive meshes.
Closes #3182
Checklist
I have followed the style guidelines for Python source files (if applicable)I have made corresponding changes to the documentation (if applicable)I have added tests that prove my fix is effective or that my feature works (if applicable)