July 2, 2024, 10:30
|
LES simulations workes on single core but not parallel
|
#1
|
|
New Member
Oliver vG
Join Date: May 2024
Posts: 5
Rep Power: 2
|
Hey everyone,
when I run my LES simulation of a stirred vessel on a single core it works but it's slow. Therefore I would like to do it in parallel but when I run the case I get the following error:
Code:
mpirun -np 2 foamRun -parallel
/*---------------------------------------------------------------------------*\
========= |
\\ / F ield | OpenFOAM: The Open Source CFD Toolbox
\\ / O peration | Website: https://openfoam.org
\\ / A nd | Version: 11
\\/ M anipulation |
\*---------------------------------------------------------------------------*/
Build : 11-e1fc8c682ae6
Exec : foamRun -parallel
Date : Jul 02 2024
Time : 15:16:12
Host : "MICROCAPS17"
PID : 116247
I/O : uncollated
Case : /home/ovg/openfoam11/cases/beaker_water_air_NCC_turbulent
nProcs : 2
Slaves : 1("MICROCAPS17.116248")
Pstream initialised with:
floatTransfer : false
nProcsSimpleSum : 0
commsType : nonBlocking
polling iterations : 0
sigFpe : Enabling floating point exception trapping (FOAM_SIGFPE).
fileModificationChecking : Monitoring run-time modified files using timeStampMaster (fileModificationSkew 10)
allowSystemOperations : Allowing user-supplied system call operations
// * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * //
Create time
Create mesh for time = 0
patchToPatch: Calculating couplings between 12080 source faces and 12080 target faces
Source min/average/max coverage = 1/1/1
Target min/average/max coverage = 1/1/1
Source average openness/error/depth/angle = 6.75728e-17/2.70342e-15/8.6106e-16/1.39602e-07
Source max openness/error/depth/angle = 4.87611e-16/3.81359e-14/1.94881e-14/1.70755e-06
24882 couplings calculated in 1.07282s
fvMeshStitcher: Connecting
3/2 small couplings removed/added to nonConformalCyclic_on_nonCouple1
Cell min/avg/max openness = 0/5.77373e-17/8.12641e-16
Selecting fvMeshDistributor distributor
Selecting distributor ptscotch
Selecting fvMeshMover motionSolver
Selecting motion solver: solidBody
Selecting solid-body motion function rotatingMotion
- selecting cells using cellZone rotating
Selecting solver multiphaseEuler
Selecting phaseSystem basicMultiphaseSystem
No MRF models present
Selecting phaseModel for air: purePhaseModel
Selecting diameterModel for phase air: isothermal
Selecting thermodynamics package
{
type heRhoThermo;
mixture pureMixture;
transport const;
thermo hConst;
equationOfState perfectGas;
specie specie;
energy sensibleInternalEnergy;
}
Calculating face flux field phi.air
Selecting turbulence model type LES
Selecting LES turbulence model continuousGasKEqn
Selecting LES delta type cubeRootVol
LES
{
model continuousGasKEqn;
turbulence on;
printCoeffs on;
delta cubeRootVol;
cubeRootVolCoeffs
{
}
Ck 0.094;
Ce 1.048;
alphaInversion 0.7;
}
Selecting thermophysical transport type LES
Selecting default LES thermophysical transport model unityLewisEddyDiffusivity
Selecting phaseModel for water: purePhaseModel
Selecting diameterModel for phase water: constant
Selecting thermodynamics package
{
type heRhoThermo;
mixture pureMixture;
transport const;
thermo eConst;
equationOfState rPolynomial;
specie specie;
energy sensibleInternalEnergy;
}
Calculating face flux field phi.water
Selecting turbulence model type LES
Selecting LES turbulence model NicenoKEqn
Selecting LES delta type cubeRootVol
LES
{
model NicenoKEqn;
turbulence on;
printCoeffs on;
delta cubeRootVol;
cubeRootVolCoeffs
{
}
Ck 0.094;
Ce 1.048;
alphaInversion 0.3;
Cp 0.094;
Cmub 0.6;
}
Selecting thermophysical transport type LES
Selecting default LES thermophysical transport model unityLewisEddyDiffusivity
Selecting interfaceSurfaceTensionModel for air_water: constant
Selecting dragModel blending method for air_water: linear
Selecting dragModel for air_dispersedIn_water: SchillerNaumann
Selecting dragModel for water_dispersedIn_air: SchillerNaumann
Selecting dragModel for air_segregatedWith_water: segregated
Selecting virtualMassModel blending method for air_water: linear
Selecting virtualMassModel for air_dispersedIn_water: constantCoefficient
Selecting virtualMassModel for water_dispersedIn_air: constantCoefficient
Selecting heatTransferModel blending method for air_water: linear
Selecting heatTransferModel for air_dispersedIn_water: RanzMarshall
Selecting heatTransferModel for water_dispersedIn_air: RanzMarshall
Courant Number mean: 0 max: 0
Creating fvConstraints from "system/fvConstraints"
Selecting finite volume constraint type limitPressure
Name: limitp
min 10000
PIMPLE: No convergence criteria found
PIMPLE: No corrector convergence criteria found
Calculations will do 3 corrections
PIMPLE: Operating solver in transient mode with 3 outer correctors
Starting time loop
Courant Number mean: 0 max: 0
fvMeshStitcher: Disconnecting
Cell min/avg/max openness = 0/5.58149e-17/2.81629e-16
Time = 0.0001s
PIMPLE: Iteration 1
patchToPatch: Calculating couplings between 12080 source faces and 12080 target faces
Source min/average/max coverage = 0.54636/0.999965/1
Target min/average/max coverage = 0.546338/0.999965/1
Source average openness/error/depth/angle = 1.49006e-16/1.0756e-07/0.000207874/0.030887
Source max openness/error/depth/angle = 8.03938e-16/0.00077327/0.0043669/0.828155
106162 couplings calculated in 0.346676s
fvMeshStitcher: Connecting
smoothSolver: Solving for MeshPhi, Initial residual = 1, Final residual = 0.00984366, No Iterations 41
Cell min/avg/max openness = 0/6.12717e-17/3.48184e-16
Cell min/avg/max volume conservation error = -1.64681e-14/3.38563e-18/1.74052e-14
nonConformalCyclic_on_nonCouple1 min/avg/max mesh flux error = 0/0/0
GAMG: Solving for pcorr, Initial residual = 1, Final residual = 0.000866911, No Iterations 8
MULES: Solving for alpha.air
MULES: Solving for alpha.water
air fraction, min, max = 0.196069 0 1
water fraction, min, max = 0.803931 0 1.00014
Phase-sum volume fraction, min, max = 1 1 1.00014
MULES: Solving for alpha.air
MULES: Solving for alpha.water
air fraction, min, max = 0.196069 -4.3281e-28 1
water fraction, min, max = 0.803931 -1.15986e-27 1.00014
Phase-sum volume fraction, min, max = 1 1 1.00014
Constructing momentum equations
[1] #0 Foam::error::printStack(Foam::Ostream&)[0] #0 Foam::error::printStack(Foam::Ostream&) at ??:?
at ??:?
[0] #1 Foam::sigFpe::sigHandler(int)[1] #1 Foam::sigFpe::sigHandler(int) at ??:?
[1] #2 ? at ??:?
[0] #2 ? in "/usr/lib/x86_64-linux-gnu/libc.so.6"
in "/usr/lib/x86_64-linux-gnu/libc.so.6"
[1] #3 Foam::divide(Foam::Field<double>&, Foam::UList<double> const&, Foam::UList<double> const&)[0] #3 Foam::divide(Foam::Field<double>&, Foam::UList<double> const&, Foam::UList<double> const&) at ??:?
at ??:?
[0] #4 [1] #4 Foam::tmp<Foam::GeometricField<double, Foam::fvPatchField, Foam::volMesh> > Foam::operator/<Foam::fvPatchField, Foam::volMesh>(Foam::GeometricField<double, Foam::fvPatchField, Foam::volMesh> const&, Foam::GeometricField<double, Foam::fvPatchField, Foam::volMesh> const&)Foam::tmp<Foam::GeometricField<double, Foam::fvPatchField, Foam::volMesh> > Foam::operator/<Foam::fvPatchField, Foam::volMesh>(Foam::GeometricField<double, Foam::fvPatchField, Foam::volMesh> const&, Foam::GeometricField<double, Foam::fvPatchField, Foam::volMesh> const&) at ??:?
[1] #5 Foam::OneResistanceHeatTransferPhaseSystem<Foam::MomentumTransferPhaseSystem<Foam::phaseSystem> >::heatTransfer() const at ??:?
[0] #5 Foam::OneResistanceHeatTransferPhaseSystem<Foam::MomentumTransferPhaseSystem<Foam::phaseSystem> >::heatTransfer() const at ??:?
[1] #6 non-virtual thunk to Foam::PhaseTransferPhaseSystem<Foam::OneResistanceHeatTransferPhaseSystem<Foam::MomentumTransferPhaseSystem<Foam::phaseSystem> > >::heatTransfer() const at ??:?
[0] #6 non-virtual thunk to Foam::PhaseTransferPhaseSystem<Foam::OneResistanceHeatTransferPhaseSystem<Foam::MomentumTransferPhaseSystem<Foam::phaseSystem> > >::heatTransfer() const at ??:?
[1] #7 Foam::solvers::multiphaseEuler::energyPredictor() at ??:?
[0] #7 Foam::solvers::multiphaseEuler::energyPredictor() at ??:?
[1] #8 Foam::solvers::multiphaseEuler::thermophysicalPredictor() at ??:?
[0] #8 Foam::solvers::multiphaseEuler::thermophysicalPredictor() at ??:?
[1] #9 at ??:?
[0] #9 ?? in "/opt/openfoam11/platforms/linux64GccDPInt32Opt/bin/foamRun"
[1] #10 ? in "/opt/openfoam11/platforms/linux64GccDPInt32Opt/bin/foamRun"
[0] #10 ? in "/usr/lib/x86_64-linux-gnu/libc.so.6"
in "/usr/lib/x86_64-linux-gnu/libc.so.6"
[1] #11 [0] #11 __libc_start_main__libc_start_main in "/usr/lib/x86_64-linux-g in "/usr/lib/x86_64-linux-gnu/libc.so.6"
[0] #12 nu/libc.so.6"
[1] #12 ?? in "/opt/openfoam11/platforms/linux64GccDPInt32Opt/bin/foamRun"
[MICROCAPS17:116247] *** Process received signal ***
[MICROCAPS17:116247] Signal: Floating point exception (8)
[MICROCAPS17:116247] Signal code: (-6)
[MICROCAPS17:116247] Failing at address: 0x3e80001c617
[MICROCAPS17:116247] [ 0] /usr/lib/x86_64-linux-gnu/libc.so.6(+0x42520)[0x7f5200559520]
[MICROCAPS17:116247] [ 1] /usr/lib/x86_64-linux-gnu/libc.so.6(pthread_kill+0x12c)[0x7f52005ad9fc]
[MICROCAPS17:116247] [ 2] /usr/lib/x86_64-linux-gnu/libc.so.6(raise+0x16)[0x7f5200559476]
[MICROCAPS17:116247] [ 3] /usr/lib/x86_64-linux-gnu/libc.so.6(+0x42520)[0x7f5200559520]
[MICROCAPS17:116247] [ 4] in "/opt/openfoam11/platforms/linux64GccDPInt32Opt/bin/foamRun"
/opt/openfoam11/platforms/linux64GccDPInt32Opt/lib/libOpenFOAM.so(_ZN4Foam6divideERNS_5FieldIdEERKNS_5UListIdEES6_+0x54)[0x7f52011e7154]
[MICROCAPS17:116247] [ 5] [MICROCAPS17:116248] *** Process received signal ***
[MICROCAPS17:116248] Signal: Floating point exception (8)
[MICROCAPS17:116248] Signal code: (-6)
[MICROCAPS17:116248] Failing at address: 0x3e80001c618
[MICROCAPS17:116248] [ 0] /usr/lib/x86_64-linux-gnu/libc.so.6(+0x42520)[0x7f7b987dc520]
[MICROCAPS17:116248] [ 1] /usr/lib/x86_64-linux-gnu/libc.so.6(pthread_kill+0x12c)[0x7f7b988309fc]
[MICROCAPS17:116248] [ 2] /usr/lib/x86_64-linux-gnu/libc.so.6(raise+0x16)[0x7f7b987dc476]
[MICROCAPS17:116248] [ 3] /usr/lib/x86_64-linux-gnu/libc.so.6(+0x42520)[0x7f7b987dc520]
[MICROCAPS17:116248] [ 4] /opt/openfoam11/platforms/linux64GccDPInt32Opt/lib/libfiniteVolume.so(_ZN4FoamdvINS_12fvPatchFieldENS_7volMeshEEENS_3tmpINS_14GeometricFieldIdT_T0_EEEERKS7_SA_+0x355)[0x7f5202987e75]
[MICROCAPS17:116247] [ 6] /opt/openfoam11/platforms/linux64GccDPInt32Opt/lib/libOpenFOAM.so(_ZN4Foam6divideERNS_5FieldIdEERKNS_5UListIdEES6_+0x54)[0x7f7b9946a154]
[MICROCAPS17:116248] [ 5] /opt/openfoam11/platforms/linux64GccDPInt32Opt/lib/libmultiphaseSystems.so(_ZNK4Foam36OneResistanceHeatTransferPhaseSystemINS_27MomentumTransferPhaseSystemINS_11phaseSystemEEEE12heatTransferEv+0x349)[0x7f51f6c83779]
[MICROCAPS17:116247] [ 7] /opt/openfoam11/platforms/linux64GccDPInt32Opt/lib/libmultiphaseSystems.so(_ZThn8_NK4Foam24PhaseTransferPhaseSystemINS_36OneResistanceHeatTransferPhaseSystemINS_27MomentumTransferPhaseSystemINS_11phaseSystemEEEEEE12heatTransferEv+0x30)[0x7f51f6c84050]
[MICROCAPS17:116247] [ 8] /opt/openfoam11/platforms/linux64GccDPInt32Opt/lib/libmultiphaseEuler.so(_ZN4Foam7solvers15multiphaseEuler15energyPredictorEv+0x4f)[0x7f51fc586fef]
[MICROCAPS17:116247] [ 9] /opt/openfoam11/platforms/linux64GccDPInt32Opt/lib/libmultiphaseEuler.so(_ZN4Foam7solvers15multiphaseEuler23thermophysicalPredictorEv+0x78)[0x7f51fc587e08]
[MICROCAPS17:116247] [10] foamRun(+0x3fb8)[0x557be255bfb8]
[MICROCAPS17:116247] [11] /usr/lib/x86_64-linux-gnu/libc.so.6(+0x29d90)[0x7f5200540d90]
[MICROCAPS17:116247] [12] /usr/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0x80)[0x7f5200540e40]
[MICROCAPS17:116247] [13] foamRun(+0x4765)[0x557be255c765]
[MICROCAPS17:116247] *** End of error message ***
/opt/openfoam11/platforms/linux64GccDPInt32Opt/lib/libfiniteVolume.so(_ZN4FoamdvINS_12fvPatchFieldENS_7volMeshEEENS_3tmpINS_14GeometricFieldIdT_T0_EEEERKS7_SA_+0x355)[0x7f7b9ac0ae75]
[MICROCAPS17:116248] [ 6] /opt/openfoam11/platforms/linux64GccDPInt32Opt/lib/libmultiphaseSystems.so(_ZNK4Foam36OneResistanceHeatTransferPhaseSystemINS_27MomentumTransferPhaseSystemINS_11phaseSystemEEEE12heatTransferEv+0x349)[0x7f7b942fd779]
[MICROCAPS17:116248] [ 7] /opt/openfoam11/platforms/linux64GccDPInt32Opt/lib/libmultiphaseSystems.so(_ZThn8_NK4Foam24PhaseTransferPhaseSystemINS_36OneResistanceHeatTransferPhaseSystemINS_27MomentumTransferPhaseSystemINS_11phaseSystemEEEEEE12heatTransferEv+0x30)[0x7f7b942fe050]
[MICROCAPS17:116248] [ 8] /opt/openfoam11/platforms/linux64GccDPInt32Opt/lib/libmultiphaseEuler.so(_ZN4Foam7solvers15multiphaseEuler15energyPredictorEv+0x4f)[0x7f7b94809fef]
[MICROCAPS17:116248] [ 9] /opt/openfoam11/platforms/linux64GccDPInt32Opt/lib/libmultiphaseEuler.so(_ZN4Foam7solvers15multiphaseEuler23thermophysicalPredictorEv+0x78)[0x7f7b9480ae08]
[MICROCAPS17:116248] [10] foamRun(+0x3fb8)[0x5570ccc92fb8]
[MICROCAPS17:116248] [11] /usr/lib/x86_64-linux-gnu/libc.so.6(+0x29d90)[0x7f7b987c3d90]
[MICROCAPS17:116248] [12] /usr/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0x80)[0x7f7b987c3e40]
[MICROCAPS17:116248] [13] foamRun(+0x4765)[0x5570ccc93765]
[MICROCAPS17:116248] *** End of error message ***
--------------------------------------------------------------------------
Primary job terminated normally, but 1 process returned
a non-zero exit code. Per user-direction, the job has been aborted.
--------------------------------------------------------------------------
--------------------------------------------------------------------------
mpirun noticed that process rank 0 with PID 0 on node MICROCAPS17 exited on signal 8 (Floating point exception).
--------------------------------------------------------------------------
I think it is somehow related to my fvScheme or fvSolution but I can't figure out where the issue is exactly.
Code:
/*--------------------------------*- C++ -*----------------------------------*\
========= |
\\ / F ield | OpenFOAM: The Open Source CFD Toolbox
\\ / O peration | Website: https://openfoam.org
\\ / A nd | Version: 11
\\/ M anipulation |
\*---------------------------------------------------------------------------*/
FoamFile
{
format ascii;
class dictionary;
location "system";
object fvSolution;
}
// * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * //
solvers
{
"alpha.*"
{
nAlphaCorr 1;
nAlphaSubCycles 2;
}
p_rgh
{
solver GAMG;
smoother DIC;
tolerance 1e-8;
relTol 0;
}
p_rghFinal
{
$p_rgh;
relTol 0;
}
"pcorr.*"
{
$p_rghFinal;
tolerance 0.001;
relTol 0;
}
"U.*"
{
solver smoothSolver;
smoother symGaussSeidel;
tolerance 1e-7;
relTol 0;
minIter 1;
}
"e.*"
{
solver smoothSolver;
smoother symGaussSeidel;
tolerance 1e-8;
relTol 0;
minIter 1;
// maxIter 100;
}
"(k|epsilon).*"
{
solver smoothSolver;
smoother symGaussSeidel;
tolerance 1e-7;
relTol 0;
minIter 1;
}
MeshPhi
{
solver smoothSolver;
smoother symGaussSeidel;
tolerance 1e-2;
relTol 0;
}
}
PIMPLE
{
nOuterCorrectors 3;
nCorrectors 1;
nNonOrthogonalCorrectors 0;
pRefCell 0;
pRefValue 0;
correctPhi yes;
correctMeshPhi yes;
faceMomentum no;
VmDdtCorrection yes;
dragCorrection yes;
partialElimination no;
momentumPredictor yes;
}
relaxationFactors
{
equations
{
".*" 1;
}
// ************************************************************************* //
Code:
/*--------------------------------*- C++ -*----------------------------------*\
========= |
\\ / F ield | OpenFOAM: The Open Source CFD Toolbox
\\ / O peration | Website: https://openfoam.org
\\ / A nd | Version: 11
\\/ M anipulation |
\*---------------------------------------------------------------------------*/
FoamFile
{
format ascii;
class dictionary;
location "system";
object fvSchemes;
}
// * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * //
ddtSchemes
{
default Euler;
}
gradSchemes
{
default Gauss linear;
}
divSchemes
{
default none;
div(phi,alpha.air) Gauss vanLeer;
div(phi,alpha.water) Gauss vanLeer;
div(phir,alpha.water,alpha.air) Gauss vanLeer;
div(phir,alpha.air,alpha.water) Gauss vanLeer;
div(alphaRhoPhi,U) Gauss limitedLinearV 1;
div(phi,U) Gauss limitedLinearV 1;
"div\(alphaRhoPhi,(h|e)\)" Gauss limitedLinear 1;
div(alphaRhoPhi,K) Gauss limitedLinear 1;
div(alphaRhoPhi,(p|rho)) Gauss limitedLinear 1;
div(alphaRhoPhi,k) Gauss limitedLinear 1;
div((((alpha*rho)*nuEff)*dev2(T(grad(U))))) Gauss linear;
}
laplacianSchemes
{
default Gauss linear uncorrected;
}
interpolationSchemes
{
default linear;
}
snGradSchemes
{
default uncorrected;
}
// ************************************************************************* //
I already found a post about this issue ( error when running in parallel, but succeed in a single core.) but this does not work in my case.
|
|
|
Similar Threads
Linear Mode
