Keynote: Prof. Martin Kronbichler (University of Augsburg)
Abstract: My talk will present experiences and insights from the development of high-performance matrix-free operator evaluation algorithms with the deal.II finite element library. Originally, our efforts have concentrated on hexahedral finite elements, where integrals can be computed very efficiently with sum factorization techniques, but we have also looked into other element shapes recently, motivated by encouraging results from the Nektar++ team. The resulting implementations come with an arithmetic intensity of one to five Flop/byte, with memory transfer primarily due to the access into input and output vectors as well as some geometry or variable coefficient data. This enables a cost per degree of freedom that is almost constant for a broad range of polynomial degrees of cell and face integrals of high order continuous and discontinuous finite elements, and allows to select the polynomial degree as a parameter to trade against mesh generation limitations. From a hardware perspective, the Flop/byte ratio suggests that memory bandwidth is the main performance limit on many CPU and GPU architectures. Furthermore, many downstream solvers, such as multigrid smoothers, explicit time stepping, or conjugate gradient solvers, are now no longer dominated by the matrix-vector products, and vector operations need to be explicitly taken into consideration as well. My talk will show the effect of algorithmic optimizations on fluid dynamics simulations on massively parallel computers.
Biography: Martin Kronbichler is a Professor at University of Augsburg, Germany. He holds a PhD degree in scientific computing with specialization in numerical analysis from Uppsala University, Sweden (2012). His research interests include high-order finite element methods for flow problems with matrix-free implementations, efficient numerical linear algebra, and their parallel and high-performance implementation on emerging exascale hardware using generic numerical software.
Chi Hin Chan (Imperial College London)
Pathways to spiral-defect chaos
Abstract: Rayleigh-Bénard convection concerns the behaviour of a fluid confined between two parallel walls, heated from below. The extent of heating is quantified by the Rayleigh number. Slightly above the critical Rayleigh number, a regime was identified where convection patterns display spatial-temporal chaotic characteristics, despite the theoretical predictions of the existence of stationary ideal straight rolls (ISRs). This chaotic behavior, referred to as spiral-defect chaos (SDC) consist of highly time-dependent and spatially disordered convection patterns. It is presently known that ISRs and SDC coexist as bistable attractors in a large extended domain. The behaviour of the system, whether ISRs or SDC, remains uncertain given an initial condition. In a smaller domain, we have identified a number of stationary, traveling wave states that resemble the local features of SDC. We propose that these states interact with each other, leading to SDC in a larger domain. Following this assumption, our objective is to investigate the transition mechanisms underlying these states using the linear stability tools of nektar++. The findings of the analyses will be presented in this workshop.
Sehun Chun (Yonsei University)
Nektar++ for Computational Neuroscience Projects: Progress, issues, and future
Jakub Fabisiak (Warsaw University of Technology)
Quantification of mixing due to hydrodynamic instability invoked kinematics in corrugated channel flows
Abstract: Improving mixing efficiency is a way to improve performance of numerous flow-based devices and a common approach is flow turbulisation, but however effective, this approach is not always desirable or realizable. Processing of highly viscous fluids, microfluidic applications or biological flows containing sheer-sensitive molecules calls for laminar mixing to be applied. Mixing in the laminar regime is particularly difficult since flows remain dominated by viscous effects and lacks strong advection and stirring. It is known that channel corrugation might result in the onset of hydrodynamic instabilities, leading to complex flow patterns and consequently improved mixing via principles of chaotic advection. In this work we perform Direct Numerical Simulation (DNS) of low Reynolds number, pressure driven flow in a doubly-periodic, corrugated channel (both symmetrically, and asymmetrically, with one wall corrugated and one flat) to analyse mixing inspired by hydrodynamic instabilities. Flows of varying Reynolds and Schmidt numbers are simulated to analyse mixing in different advective and diffusive conditions. We compare obtained asymmetric channel results with results in symmetry preserving corrugated channel using the rate of exponential decay of variance and negative index Sobolev mix-norm – norm similar to variance (and L2 norms in general), but overcoming its main flaw, which is performance in limit cases of low diffusivity. As the variance time derivative explicitly depends on diffusivity only, it can fail to properly measure mixing in cases where the advective stirring highly exceed the diffusion impact on mixing, while the mix-norm includes “smoothing” operator, overcoming this problem. Sobolev mix-norms, as well as the flow itself, were calculated with the Nektar++ tools. We also utilize and visualise the strange-eigenmodes – structures persistent in time in cases with laminar mixing induced by the chaotic advection and corresponding to eigen functions of the advection diffusion operator.
Aidan Forknall (Loughborough University)
Implementation of a Combustion Module within the Nektar++ Framework
Combustion modelling methods are implemented into the Nektar++ framework to create a reactive flow solver capable of accurately modelling combustion flow regimes at low Mach numbers. The study investigates how the high-order resolution of a mixing flow field translates to modelling of turbulent diffusion flames, in regard to both cost and accuracy. First, a non-reacting pseudo-compressible flow solver has been developed based on the Incompressible Navier-Stokes code, using a low Mach variable density approach able to simulate mixing between fuel and oxidizer streams. For modelling of reactive flow, results of a 1D flamelet generated manifold (FGM) formulation dependant on mixture fraction, Z, are tabulated and coupled into the solving process. Validations are presented with turbulent cold flow mixing and canonical reacting flow cases, including combustor relevant geometry such as Sandia Propane-Jet flow and Sandia Piloted CH4/Air Flames.
Jakub Gałecki (Warsaw University of Technology)
Is the grass greener on the other side? Adventures with the least-squares finite element method
Abstract: The least-squares finite element method (LSFEM) emerged as an alternative to classical Galerkin FEM around 30 years ago. Since then, it has successfully found applications in many fields of computational physics, including fluid dynamics. The method possesses some desirable properties – it always leads to symmetric, positive-definite algebraic systems (even for non-self-adjoint operators), it does not necessitate the fulfillment of compatibility (LBB) conditions, and it is generally more stable than its classical counterpart. Conversely, it requires recasting equations to the first order (introducing additional degrees of freedom) and involves a more computationally expensive assembly procedure. This talk will attempt to evaluate whether the benefits of LSFEM outweigh its drawbacks and offer a practical perspective on how it can be employed to efficiently solve incompressible flow problems. We will show an alternative equation splitting scheme, which, in conjunction with LSFEM, leads to an enhanced convergence rate. Finally, we will discuss computational aspects of the method using the author’s high-order code L3STER as a point of reference. In particular, we will show how to structure assembly so that it fully utilizes CPU caches, and how hybrid parallelism can be leveraged for improved scalability.
Stanisław Gepner (Warsaw University of Technology)
Continuation methods and turning on bifurcations with Nektar++
Continuation methods are often used when tracking interesting states of a dynamical system, such as unstable saddle points that are important for the organization of system’s dynamics. In turbulent flows and turbulent transition process itself such states play an organizing role and act by attracting the flow solution along their stable manifold only to eject it along respective unstable trajectories and resulting in complex evolution patterns. This fact alone makes such states an interesting research topic. With the use of continuation techniques one is able to determine conditions when such states are created as well as navigate respective branches of those solutions. One of the problems, though is the ability of standard tools to navigate and converge close to the bifurcation points, where systems often become singular. In this work we will present our approach and discuss the Nektar++ implementation of the methods used for the continuation and turning around bifurcations.
João Isler (Imperial College London)
Turbomachinery applications using Nektar++
Kaloyan Kirilov (Imperial College London)
Recent advances in NekMesh – mesh generation and adaptation
Abstract: High-order mesh generation tools typically adopt the bottom-up approach – first creating a valid linear mesh followed by high-order tools for mesh curving. This poses a significant challenge for the linear mesh generation when very complex geometries are considered. Alternatively, one can produce the linear mesh with a third-part/commercial mesh generator and curve it in-house. NekMesh did have such capability, limited to a projection of only the edge quadrature points on the CAD. Now we present a complete redesign of this pipeline, which does not only projects edge and face nodes to the CAD B-Rep, but also integrates all NekMesh high-order modules for surface and volume optimizations, boundary-layer splitting and untangling into the process.
By implementing this new pipeline, we have achieved the unique capability to produce meshes that exhibit the similar surface&volume quality to the ones produced with the bottom-up approach, but also with the flexibility, robustness and user-friendly graphical user interface (GUI) of the commercial mesh generator. To demonstrate the capabilities of our new pipeline and showcase the improved mesh quality, we will show various simple and automotive geometries.
The second part of this talk focuses on a posteriori mesh adaptation for compressible flows, specifically highlighting current activities in NekMesh and Nektar++. Initially, we will provide a brief overview of the existing r & p adaptation techniques and the new isoparametric h-adaptation. Then, these will be combined in r-p, h-p and h-r-p adaptations, showing the advantages of applying all three adaptation techniques simultaneously.
Abhishek Kumar (Coventry University)
Optimal inflow perturbation for the mixed baroclinic convection
Abstract: In this study, our objective is to mitigate the buoyant instability inherent in mixed baroclinic convection within a cavity by employing an advanced optimisation technique. We consider a nearly semi-cylindrical cavity featuring an upper free surface through which fluid can enter, and porous lower boundaries through which fluid can exit. The cavity consists of a semicircular lower boundary, two adiabatic sidewalls, and extends infinitely in the third direction. This specific configuration is pertinent to situations involving molten solid materials, such as in metallurgical casting processes.
Our previous stability analysis of this problem revealed the presence of three-dimensional unstable modes [1]. Additionally, the receptivity analysis indicated that the through-flow at the centre of the inlet exhibits the highest receptivity. Given the inlet’s receptiveness, our focus is on identifying the optimal perturbation in the inflow profile capable of suppressing this instability. To accomplish this, we implemented an optimisation solver within the Nektar++ framework [2] to search for the optimal inflow perturbation with a prescribed time-dependence [3]. Preliminary results demonstrate that the optimised inflow profile effectively dampens the linear growth of the unstable mode for a specific duration.
References:
[1] Kumar and Pothérat, J. Fluid Mech. 885, A40 (2020).
[2] Moxey et al., Comput. Phys. Commun. 249, 107110 (2020).
[3] Mao, Blackburn, and Sherwin, Computers and Fluids, 121, 133-144 (2015).
Mohsen Lahooti (Newcastle University)
Wake and performance of a propeller under forced vibration
Abstract: Forced vibration of E779 propeller is investigated. Incompressible Navier-Stokes equations in moving reference frame with absolute velocity formulations are used to capture the flow dynamics. Governing equations are discretised using high-order spectral/hp element method and implemented in Nektar++ framework. Flow turbulence is resolved using implicit LES approach. The flow dynamics is established initially for four advanced ratios J=0.25,0.45,0.50 and 0.65 without considering the vibration. Forced vibration then considered in normal direction to the free stream with different vibration frequencies and amplitudes for J=0.50 and J=0.65. Propellers wake and fluid forces are investigated under these vibrations. The simulations results indicates that even at lowest vibration amplitude the wake structure experiences substantial changes. Tip vortices breaks to a twin one when the blades aligned with the vibration direction and the whole wake expands and contracts behind the propeller.
Ganlin Lyu (Imperial College London)
Stable Riemann inflow boundary conditions for DG compressible flow simulations
Abstract: Motivated by enabling high-fidelity, discontinuous Galerkin (DG) simulation of transonic boundary layer flows we consider a near-body reduced domain embedded in outer low-fidelity simulations. We investigate the design and construction of stable Riemann boundary conditions that enforce desired quantities of interest (such as the pressure distribution) from the outer low-fidelity simulations or a known state on to the high-fidelity DG inner domain. Although our primary focus is on the inflow boundary conditions similar analysis can also be adopted to outflow boundary condition design. In a typical DG discretisation, the expansion inside each spectral/hp element is coupled to adjacent elements through numerical flux evaluated by the solution to a Riemann problem leading to a weak enforcement of the boundary condition. A successful boundary condition enforcement should be well-posed and lead to a stable numerical solution if unsteady physics such as turbulence are absent. We therefore undertake a linear stability analysis approach to examine the performance of the boundary conditions for the advection terms in the compressible flow simulations. To reduce the problem to its simplest form we consider the piecewise constant approximation in the element to produce a system with minimal degrees of freedom. We show that this analysis can be either element level or global level, while the former identifies the necessary conditions for a stable full simulation. We demonstrate that entropy compatibility is critical to maintain stability but the other condition is a free choice for the subsonic inflow. Finally we validate this analysis by designing a pressure compatible inflow, where only the entropy-pressure compatible inflow condition is the stable choice. This inflow condition is then enforced in a reduced domain of a wing section normal to the leading edge of the CRM-NLF model and DLR-F5 model taken out of full 3D RANS simulations at Mach 0.86 and 0.8, and chord Reynolds numbers of 8.5 million and 1.0 million respectively. The results show that the designed inflow condition leads to a desired agreement not only on pressure distribution but also on velocity fields, while the other types of inflow conditions cannot achieve this.
Allen Sanderson (University of Utah)
Exploiting Heterogeneous Systems – GPUs
To exploit parallelism, many codes are being deployed on heterogeneous systems so to make use of GPUs. Those developing codes have two options, write for a specific hardware (CUDA, HIP, SYCL, etc) or utilize a middle layer, Kokkos that encapsulates different hardware. In this talk the lessons learned from working on two different codes that utilize Kokkos and how these lessons can help guide Nektar++ as it transitions to heterogeneous systems.
Ed Threlfall (UK Atomic Energy Authority)
ExCALIBUR Project NEPTUNE: Nektar++ for fusion plasma simulation
Abstract: Project NEPTUNE (NEutrals and Plasma Turbulence Numerics for the Exascale; part of the UK’s ExCALIBUR programme) aims to provide fusion plasma simulation capability compatible with the coming landscape of exascale-class hardware, focussing in particular on the `edge’ region of the plasma where it comes into contact with the material of the tokamak. This represents an extremely challenging problem, particularly in view of the need to use kinetic theory where the plasma is not in local thermal equilibrium (though equilibrium fluids must be used where possible in order to manage computational expense). The plasma fluid component of NEPTUNE is expected to be a set of solvers built on Nektar++, treating the equations of motion for a charged, multicomponent, and strongly magnetized fluid, and leveraging the advantages of the spectral / hp method over existing finite-difference plasma codes. The non-equilibrium matter is treated using a particle simulation code developed in-house by the NEPTUNE team, which interfaces with, and in particular is able to leverage the meshing capabilities of, Nektar++. This talk will explain the rationale and structure of code developed under Project NEPTUNE and will outline current work toward coupled 3D plasma turbulence, neutral kinetics, and atomic reactions capability. Work toward true performance portability will be shown, with SYCL as the chosen DSL and including SYCL implementation of some existing Nektar++ code.
The support of the UK Meteorological Office and Strategic Priorities Fund is acknowledged.
Nikesh Yadav (Warsaw University of Technology and Nicolaus Copernicus Astronomical Center of the Polish Academy of Sciences)
Deep learning-based prediction of flow behind circular cylinders using solutions from Nektar++ Solver
Abstract: Much attention has been paid to deep learning and machine learning techniques to reduce the computational cost of computational fluid dynamics simulations. This work addresses the prediction of steady-state flows through many stationary cylinders using a deep-learning model and examines the accuracy of the predicted velocity fields. A deep learning model predicts the x and y components of the velocity field for a given cylinder configuration. The accuracy of the predicted velocity field is investigated with a focus on the velocity profile of the fluid flow and the fluid forces acting on the cylinder. This research focuses on the flow around a large number of cylinders and consists of the following two studies. The first of two studies in this study predicted a steady flow through a stationary cylinder, which is reported in this study. In this study, all cylinders are considered immobile and fixed in space. Fluid flow is guided by boundary conditions. In this research, we use a U-Net-like architecture to build a deep learning model. U-Net is typically applied to image segmentation problems. However, in this research, we apply U-Net to physical problems. Examine the accuracy of the predicted velocity field associated with the velocity profile of the fluid flow and the fluid forces acting on the cylinder. The current model accurately predicts flow when the number of cylinders is equal to or close to the number in the training data set. Extrapolating the predictions to a smaller number of cylinders introduces an error that can be interpreted as internal friction in the fluid. The fluid force results acting on cylinders suggest that the current deep learning model has good generalization performance for systems with a large number of cylinders. This study will help in designing the wind turbine arrangements for achieving the optimal power output for a given flow condition.
Jacques Xing (King’s College London)
Implementation of the Parareal and PFASST Algorithms in Nektar++
Abstract: Parallel-in-time algorithms are more and more recognized as a promising solution to increase computational concurrency after the maximum speed-up has been achieved from spatial parallelism due to communication overhead when the problem size per processor has become too small in distributed memory high-performance computing architecture. Two popular time-parallel approaches are the Parareal algorithm, first introduced by Lions et al. (2001), and the Parallel Full Approximation Scheme in Space and Time (PFASST) technique of Emmett and Minion (2012). Both methods are based on the use of multi-level time-integration, exploiting a fine and a coarse time integrators in combination with an iterative procedure to achieve parallelism in time. The implementation of these two parallel-in-time algorithms in the Nektar++ spectral/hp element framework is described. The extension of the MPI topology to allow concurrency in time is first presented. Following, the implementation of new drivers for the Parareal and PFASST algorithms is described. Finally, application examples for the 1D advection and the 2D diffusion equations are demonstrated.