Skip to content

Commit

Permalink
update joss paper
Browse files Browse the repository at this point in the history
  • Loading branch information
rprat-pro committed May 22, 2024
1 parent 15e0231 commit 201b8ef
Show file tree
Hide file tree
Showing 2 changed files with 59 additions and 6 deletions.
50 changes: 50 additions & 0 deletions doc/paper/joss/paper.bib
Original file line number Diff line number Diff line change
Expand Up @@ -38,3 +38,53 @@ @article{verlet1967computer
publisher={APS}
}

@article{kloss2012models,
title={Models, algorithms and validation for opensource DEM and CFD--DEM},
author={Kloss, Christoph and Goniva, Christoph and Hager, Alice and Amberger, Stefan and Pirker, Stefan},
journal={Progress in Computational Fluid Dynamics, an International Journal},
volume={12},
number={2-3},
pages={140--152},
year={2012},
publisher={Inderscience Publishers}
}

@inproceedings{dirand2018tins,
title={Tins: A task-based dynamic helper core strategy for in situ analytics},
author={Dirand, Estelle and Colombet, Laurent and Raffin, Bruno},
booktitle={Supercomputing Frontiers: 4th Asian Conference, SCFA 2018, Singapore, March 26-29, 2018, Proceedings 4},
pages={159--178},
year={2018},
organization={Springer}
}

@article{govender2018study,
title={A study of shape non-uniformity and poly-dispersity in hopper discharge of spherical and polyhedral particle systems using the Blaze-DEM GPU code},
author={Govender, Nicolin and Wilke, Daniel N and Pizette, Patrick and Abriak, Nor-Edine},
journal={Applied Mathematics and Computation},
volume={319},
pages={318--336},
year={2018},
publisher={Elsevier}
}

@inproceedings{cieren2014exastamp,
title={ExaStamp: A parallel framework for molecular dynamics on heterogeneous clusters},
author={Cieren, Emmanuel and Colombet, Laurent and Pitoiset, Samuel and Namyst, Raymond},
booktitle={Euro-Par 2014: Parallel Processing Workshops: Euro-Par 2014 International Workshops, Porto, Portugal, August 25-26, 2014, Revised Selected Papers, Part II 20},
pages={121--132},
year={2014},
organization={Springer}
}


@article{prat2020amr,
title={AMR-based molecular dynamics for non-uniform, highly dynamic particle simulations},
author={Prat, Rapha{\"e}l and Carrard, Thierry and Soulard, Laurent and Durand, Olivier and Namyst, Raymond and Colombet, Laurent},
journal={Computer Physics Communications},
volume={253},
pages={107177},
year={2020},
publisher={Elsevier}
}

15 changes: 9 additions & 6 deletions doc/paper/joss/paper.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
title: 'ExaDEM: an HPC application based on ExaNBody targetting DEM simulations with spheropolyhedra'
title: 'ExaDEM: a HPC application based on ExaNBody targetting DEM simulations with polyhedron particles'
tags:
- DEM
- HPC
Expand Down Expand Up @@ -44,15 +44,18 @@ bibliography: paper.bib

`ExaDEM` is a Discrete Element Method (`DEM`) code developed using the exaNBody framework [@carrard2023exanbody] at the french atomic commission (CEA). This software provides `DEM` functionalities to model spheres and spheropolyhedra while proposing performance optimizations on current HPC plateforms. A notable aspect of `ExaDEM` is its hybrid parallelization approach, which combines the use of MPI and Threads (OpenMP). Additionally, `ExaDEM` offers compatibility with `MPI`+`GPU`s, using the `CUDA` programming model (named `Onika` layer) for DEM simulations with spherical particles. Written in `C++17`, `ExaDEM` aims to provide HPC capabilities for scientific community. `ExaDEM` intends to embed the physics of interest developed in the `Rockable` code develooped at CNRS.


# Statement of needs

The behavior of granular media is still an open issue for the scientific community and DEM simulations improve our knowledges by studying phenomena that are unreachables, or expensive, to examine with experimentations. However, to repoduce such phenomena, we need to simulate a representative number of particles that can reach thousands particles to billion of particles. To simulate thousand particles, a current single processor can achieved such simulations while simulating millions of particles required HPC ressources. These simulations are either limited by the memory footprint, either by the runtime. The `DEM` method has the advantage to be naturarly parallel and several works exist for this subjects, spatial domain decomposition [@plimpton1995fast], thread parallelization over cells and so on. In this paper we highlight our code `ExaDEM` designed to achieve large scale DEM simulation on HPC platform. This code relies on many features of the exaNBody framework, including data structure management, MPI+X parallelization and add-on modules (IO, Paraview).

# DEM Background

The `DEM` method, used to study granular media, falls within the scope of so-called N-body methods. It consists in numerically reproducing the evolution of a set of particles over time. Time is discretized into time steps and at each time step n, we solve Newton's equation f=ma to deduce the acceleration for each particle and then calculate its velocity, which will give us the new positions at time n+1. "f" is computed from interaction between particules, i.e. contact interactions or external forces such as gravity. A usual numerical scheme used is the Velocity Verlet and to model contact interaction, Hooke is widely used. The DEM method allows to simulate rigid bodies with differents shape: from spherical particle to polyhedral particles.

A crucial point for `DEM` simulation code is dicted by the need to figure out quickly the nearest neighbor particles to compute contact interactions. The common way to do it is to use the fuse between the linked cells method [@ciccotti1987simulation] and Verlet lists [@verlet1967computer] that limits the refresh rate of neighbor lists while optimizing the neighbor search using a cartesian grid of cells (complexity of N).

# Statement of needs

The behavior of granular media is still an open issue for the scientific community and DEM simulations improve our knowledges by studying phenomena that are unreachables, or expensive, to examine with experimentations. However, to repoduce such phenomena, we need to simulate a representative number of particles that can reach thousands particles to billion of particles. To simulate thousand particles, a current single processor can achieved such simulations while simulating millions of particles required HPC ressources. These simulations are either limited by the memory footprint, either by the runtime. The `DEM` method has the advantage to be naturarly parallel and several works exist for this subjects, spatial domain decomposition [@plimpton1995fast], thread parallelization over cells and so on. In this paper we highlight our code ExaDEM designed to achieve large scale DEM simulation on HPC platform. This code relies on many features of the exaNBody framework, including data structure management, MPI+X parallelization and add-on modules (IO, Paraview).
Several DEM softwares have been developped over the last year and propose HPC features such as LIGGGTHS [@kloss2012models] based on LAMMPS [@thompson2022lammps] data structures (Molecular Dynamics code) with spherical particles (MPI) or Blaze-DEM [@govender2018study] with spheres and polyhedra on GPU using CUDA. `ExaDEM`'s objective is to position itself in the literature as a software product that combines MPI parallelization with OpenMP thread parallelization and CUDA GPU parallelization for polyhedral and spherical particles. As LIGGGTHS with LAMMPS, `ExaDEM` takes advantage of several HPC developpements done in `ExaSTAMP` (Molecular Dynamics code) [@cieren2014exastamp] that have been mutualized in the `ExaNBody` framework such as `AMR` data structures [@prat2020amr] or In-situ analysis [@dirand2018tins].

# Implementation

Expand All @@ -71,7 +74,7 @@ Finally, it is important to note that the design of `ExaDEM` lead by the framewo

# Main features

![Simulation of 500 thousands octahedra in a rotating drum \label{fig:rotating-drum}](./rotating-drum.png "test"){width=80%}
![Simulation of 500 thousands octahedra in a rotating drum running on 128 mpi processes with 8 OpenMP threads per mpi process. \label{fig:rotating-drum}](./rotating-drum.png "test"){width=80%}

`ExaDEM` attends to meet scientific expectations, espacially for fuel nuclear simulations consisting in rotating drum (see figure \ref{fig:rotating-drum}) or compression simulations. To do it, `ExaDEM` provides the following features:

Expand All @@ -91,4 +94,4 @@ All these functionalities are subject to evolution in line with new development

# Future of `ExaDEM`

In the upcoming years, we plan to add several `DEM` functionnalities such as drivers, fragmentation, or diagnostics in order to propose to physicians. On the other hand, we will develop other parallel strategies, espacially on GPU, to run on future supercomputers. A specific focus will be done for simulation sphero-polyhedron particles. `ExaDEM` is available under an APACHE 2 license at https://github.com/Collab4exaNBody/exaDEM.git and the documentation is available at https://github.com/Collab4exaNBody/doc_exaDEM.git.
In the upcoming years, we plan to add several `DEM` functionnalities such as complexe geometries (propeller), particle fragmentation, or more diagnostics. On the other hand, we will develop other parallel strategies, espacially on GPU, to run on future supercomputers. A specific focus will be done for simulation sphero-polyhedron particles. `ExaDEM` is available under an APACHE 2 license at https://github.com/Collab4exaNBody/exaDEM.git and the documentation is available at https://github.com/Collab4exaNBody/doc_exaDEM.git.

0 comments on commit 201b8ef

Please sign in to comment.