Turbulence is ubiquitous in environmental and engineering fluid flows, and is encountered routinely in on a regular basis life. A greater understanding of those turbulent processes might present useful insights throughout quite a lot of analysis areas — enhancing the prediction of cloud formation by atmospheric transport and the spreading of wildfires by turbulent vitality alternate, understanding sedimentation of deposits in rivers, and enhancing the effectivity of combustion in plane engines to scale back emissions, to call a number of. Nevertheless, regardless of its significance, our present understanding and our potential to reliably predict such flows stays restricted. That is primarily attributed to the extremely chaotic nature and the big spatial and temporal scales these fluid flows occupy, starting from energetic, large-scale actions on the order of a number of meters on the high-end, the place vitality is injected into the fluid circulate, all the way in which right down to micrometers (μm) on the low-end, the place the turbulence is dissipated into warmth by viscous friction.
A strong instrument to know these turbulent flows is the direct numerical simulation (DNS), which offers an in depth illustration of the unsteady three-dimensional flow-field with out making any approximations or simplifications. Extra particularly, this method makes use of a discrete grid with sufficiently small grid spacing to seize the underlying steady equations that govern the dynamics of the system (on this case, variable-density Navier-Stokes equations, which govern all fluid circulate dynamics). When the grid spacing is sufficiently small, the discrete grid factors are sufficient to symbolize the true (steady) equations with out the lack of accuracy. Whereas that is engaging, such simulations require great computational assets to be able to seize the right fluid-flow behaviors throughout such a variety of spatial scales.
The precise span in spatial decision to which direct numerical calculations have to be utilized depends upon the duty and is decided by the Reynolds quantity, which compares inertial to viscous forces. Usually, the Reynolds quantity can vary between 102 as much as 107 (even bigger for atmospheric or interstellar issues). In 3D, the grid dimension for the decision required scales roughly with the Reynolds quantity to the ability of 4.5! Due to this sturdy scaling dependency, simulating such flows is mostly restricted to circulate regimes with reasonable Reynolds numbers, and sometimes requires entry to high-performance computing methods with tens of millions of CPU/GPU cores.
In “A TensorFlow simulation framework for scientific computing of fluid flows on tensor processing models”, we introduce a brand new simulation framework that allows the computation of fluid flows with TPUs. By leveraging newest advances on TensorFlow software program and TPU-hardware structure, this software program instrument permits detailed large-scale simulations of turbulent flows at unprecedented scale, pushing the boundaries of scientific discovery and turbulence evaluation. We reveal that this framework scales effectively to accommodate the size of the issue or, alternatively, improved run instances, which is exceptional since most large-scale distributed computation frameworks exhibit decreased effectivity with scaling. The software program is out there as an open-source mission on GitHub.
Giant-scale scientific computation with accelerators
The software program solves variable-density Navier-Stokes equations on TPU architectures utilizing the TensorFlow framework. The one-instruction, multiple-data (SIMD) method is adopted for parallelization of the TPU solver implementation. The finite distinction operators on a colocated structured mesh are solid as filters of the convolution perform of TensorFlow, leveraging TPU’s matrix multiply unit (MXU). The framework takes benefit of the low-latency high-bandwidth inter-chips interconnect (ICI) between the TPU accelerators. As well as, by leveraging the single-precision floating-point computations and extremely optimized executable via the accelerated linear algebra (XLA) compiler, it’s potential to carry out large-scale simulations with glorious scaling on TPU {hardware} architectures.
This analysis effort demonstrates that the graph-based TensorFlow together with new varieties of ML particular objective {hardware}, can be utilized as a programming paradigm to unravel partial differential equations representing multiphysics flows. The latter is achieved by augmenting the Navier-Stokes equations with bodily fashions to account for chemical reactions, heat-transfer, and density adjustments to allow, for instance, simulations of cloud formation and wildfires.
It’s value noting that this framework is the primary open-source computational fluid dynamics (CFD) framework for high-performance, large-scale simulations to completely leverage the cloud accelerators which have turn into widespread (and turn into a commodity) with the development of machine studying (ML) in recent times. Whereas our work focuses on utilizing TPU accelerators, the code might be simply adjusted for different accelerators, resembling GPU clusters.
This framework demonstrates a option to significantly cut back the fee and turn-around time related to working large-scale scientific CFD simulations and allows even larger iteration pace in fields, resembling local weather and climate analysis. For the reason that framework is applied utilizing TensorFlow, an ML language, it additionally allows the prepared integration with ML strategies and permits the exploration of ML approaches on CFD issues. With the overall accessibility of TPU and GPU {hardware}, this method lowers the barrier for researchers to contribute to our understanding of large-scale turbulent methods.
Framework validation and homogeneous isotropic turbulence
Past demonstrating the efficiency and the scaling capabilities, additionally it is vital to validate the correctness of this framework to make sure that when it’s used for CFD issues, we get cheap outcomes. For this objective, researchers sometimes use idealized benchmark issues throughout CFD solver improvement, a lot of which we adopted in our work (extra particulars within the paper).
One such benchmark for turbulence evaluation is homogeneous isotropic turbulence (HIT), which is a canonical and properly studied circulate by which the statistical properties, resembling kinetic vitality, are invariant underneath translations and rotations of the coordinate axes. By pushing the decision to the bounds of the present state-of-the-art, we have been in a position to carry out direct numerical simulations with greater than eight billion levels of freedom — equal to a three-dimensional mesh with 2,048 grid factors alongside every of the three instructions. We used 512 TPU-v4 cores, distributing the computation of the grid factors alongside the x, y, and z axes to a distribution of [2,2,128] cores, respectively, optimized for the efficiency on TPU. The wall clock time per timestep was round 425 milliseconds and the circulate was simulated for a complete of 400,000 timesteps. 50 TB information, which incorporates the rate and density fields, is saved for 400 timesteps (each 1,000th step). To our information, this is without doubt one of the largest turbulent circulate simulations of its sort performed up to now.
Because of the advanced, chaotic nature of the turbulent circulate subject, which extends throughout a number of magnitudes of decision, simulating the system in excessive decision is critical. As a result of we make use of a fine-resolution grid with eight billion factors, we’re in a position to precisely resolve the sphere.
Contours of x-component of velocity alongside the z midplane. The excessive decision of the simulation is vital to precisely symbolize the turbulent subject.
The turbulent kinetic vitality and dissipation charges are two statistical portions generally used to research a turbulent circulate. The temporal decay of those properties in a turbulent subject with out further vitality injection is because of viscous dissipation and the decay asymptotes comply with the anticipated analytical energy legislation. That is in settlement with the theoretical asymptotes and observations reported within the literature and thus, validates our framework.
Strong line: Temporal evolution of turbulent kinetic vitality (ok). Dashed line: Analytical energy legal guidelines for decaying homogeneous isotropic turbulence (n=1.3) (Ⲧl: eddy turnover time).
Strong line: Temporal evolution of dissipation price (ε). Dashed line: Analytical energy legal guidelines for decaying homogeneous isotropic turbulence (n=1.3).
The vitality spectrum of a turbulent circulate represents the vitality content material throughout wavenumber, the place the wavenumber ok is proportional to the inverse wavelength λ (i.e., ok ∝ 1/λ). Usually, the spectrum might be qualitatively divided into three ranges: supply vary, inertial vary and viscous dissipative vary (from left to proper on the wavenumber axis, beneath). The bottom wavenumbers within the supply vary correspond to the biggest turbulent eddies, which have probably the most vitality content material. These massive eddies switch vitality to turbulence within the intermediate wavenumbers (inertial vary), which is statistically isotropic (i.e., basically uniform in all instructions). The smallest eddies, similar to the biggest wavenumbers, are dissipated into thermal vitality by the viscosity of the fluid. By advantage of the high-quality grid having 2,048 factors in every of the three spatial instructions, we’re in a position to resolve the circulate subject as much as the size scale at which viscous dissipation takes place. This direct numerical simulation method is probably the most correct because it doesn’t require any closure mannequin to approximate the vitality cascade beneath the grid dimension.
Spectrum of turbulent kinetic vitality at totally different time situations. The spectrum is normalized by the instantaneous integral size (l) and the turbulent kinetic vitality (ok).
A brand new period for turbulent flows analysis
Extra just lately, we prolonged this framework to foretell wildfires and atmospheric flows, which is related for climate-risk evaluation. Aside from enabling high-fidelity simulations of advanced turbulent flows, this simulation framework additionally offers capabilities for scientific machine studying (SciML) — for instance, downsampling from a high-quality to a rough grid (mannequin discount) or constructing fashions that run at decrease decision whereas nonetheless capturing the right dynamic behaviors. It might additionally present avenues for additional scientific discovery, resembling constructing ML-based fashions to raised parameterize microphysics of turbulent flows, together with bodily relationships between temperature, strain, vapor fraction, and so forth., and will enhance upon numerous management duties, e.g., to scale back the vitality consumption of buildings or discover extra environment friendly propeller shapes. Whereas engaging, a essential bottleneck in SciML has been the provision of information for coaching. To discover this, now we have been working with teams at Stanford and Kaggle to make the information from our high-resolution HIT simulation accessible via a community-hosted web-platform, BLASTNet, to supply broad entry to high-fidelity information to the analysis neighborhood by way of a network-of-datasets method. We hope that the provision of those rising high-fidelity simulation instruments at the side of community-driven datasets will result in vital advances in numerous areas of fluid mechanics.
Acknowledgements
We want to thank Qing Wang, Yi-Fan Chen, and John Anderson for consulting and recommendation, Tyler Russell and Carla Bromberg for program administration.