Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable adaptive mesh support on libMesh tallies #3185

Open
wants to merge 6 commits into
base: develop
Choose a base branch
from

Conversation

nuclearkevin
Copy link

@nuclearkevin nuclearkevin commented Nov 5, 2024

Description

This PR allows for the use of adaptive meshes (which have an adaptive mesh refinement hierarchy) in unstructured mesh tallies which use a LibMesh. Adaptive meshes (unlike libMesh meshes which set allow_renumbering = false) do not guarantee that active elements (which we want to tally on) are contiguous in memory relative to their DoF ids, which results in errors when OpenMC computes bin indices as bin are defined over [0, num_active_elem]. This results in scrambled tallies for libMesh meshes which have an adaptivity hierarchy. An additional issue is that the libMesh::EquationSystems object added by OpenMC to enable LibMesh exodus output reacts poorly to adaptive meshes, throwing errors when the mesh is refined/coarsened or dumped to exodus after an OpenMC solve.

To fix the first issue, an indirection layer is added which maps between element DoF ids and bin indices. This is generated when LibMesh is initialized based on a constant adaptivity flag (adaptive_). The indirection layer is used in get_bin_from_element(...) and get_element_from_bin(...) only when adaptive_ = true. This should result in no performance changes for existing users that use libMesh-based tallies without adaptivity, though it does result in a small (but noticeable) decrease in tally performance on adaptive libMesh meshes compared to creating a deep copy of the mesh which only contains active elements (due to cache misses).

The second issue is resolved by constructing and setting up the libMesh::EquationSystems class in LibMesh::add_score(...) if it hasn't been initialized.

I've tested these changes in Cardinal to make sure adaptive mesh tallies work, and the results generated are equivalent to the previous approach we took. Regression tests for libMesh tallies also pass on my machine, so OpenMC users should be unaffected by this fix for adaptive meshes.

Closes #3182

Checklist

  • I have performed a self-review of my own code
  • I have run clang-format (version 15) on any C++ source files (if applicable)
  • I have followed the style guidelines for Python source files (if applicable)
  • I have made corresponding changes to the documentation (if applicable)
  • I have added tests that prove my fix is effective or that my feature works (if applicable)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Enable adaptive mesh support on libMesh unstructured mesh tallies
1 participant