|
[Sponsors] |
![]() |
![]() |
#1 |
Member
Julian
Join Date: Sep 2019
Posts: 32
Rep Power: 7 ![]() |
I am running a LES case with a 17 million cell mesh from blockMesh, using the pimpleFoam solver in OF 5.0.
The case takes a large amount of CPU hours, so I am looking for approaches to speed up the calculation. One of these is using the renumberMesh utility. I am encountering a crash with the pressure solver when I try and use renumberMesh in parallel. I am not sure if they are related, so I ran one case with * renumberMesh in serial before decomposePar and one case with * renumberMesh in parallel after decomposePar. That is, 1. Generate a mesh with blockMesh. 2. Decompose the mesh for 180 processors via decomposePar. 3. Run renumberMesh in parallel on 180 procs: renumberMesh -overwrite -parallel (SLURM submission script handles the parallel portion) 4. Run the simulation on 180 procs with pimpleFoam. The simulation runs fine for a while. The residuals and CFL number look ok. Then, the simulation crashes with the following log excerpt. I've also attached my fvSolution dictionary below. Thank you for any advice or input. Code:
Courant Number mean: 0.01953872075 max: 0.5964071291 Time = 0.027228 PIMPLE: iteration 1 [0] Starting inflgen, time = 8023 s [0] The number of vortons is 8032 [0] Finishing inflgen, time = 8023 s smoothSolver: Solving for Ux, Initial residual = 0.0005660971091, Final residual = 3.365824579e-06, No Iterations 1 smoothSolver: Solving for Uy, Initial residual = 0.001857616441, Final residual = 1.248576987e-05, No Iterations 1 smoothSolver: Solving for Uz, Initial residual = 0.001845039049, Final residual = 1.264188109e-05, No Iterations 1 GAMG: Solving for p, Initial residual = 0.02449620851, Final residual = 4.450573519e-07, No Iterations 13 time step continuity errors : sum local = 7.999202759e-12, global = -2.499062109e-14, cumulative = 2.316567643e-12 PIMPLE: iteration 2 smoothSolver: Solving for Ux, Initial residual = 1.32914688e-05, Final residual = 1.25106515e-07, No Iterations 1 smoothSolver: Solving for Uy, Initial residual = 4.68221212e-05, Final residual = 5.277816827e-07, No Iterations 1 smoothSolver: Solving for Uz, Initial residual = 4.765156009e-05, Final residual = 5.373488844e-07, No Iterations 1 GAMG: Solving for p, Initial residual = 0.03288688617, Final residual = 6.918512333e-07, No Iterations 13 time step continuity errors : sum local = 1.243177916e-11, global = 3.739872997e-15, cumulative = 2.320307516e-12 PIMPLE: iteration 3 smoothSolver: Solving for Ux, Initial residual = 4.273234509e-06, Final residual = 4.73847033e-08, No Iterations 1 smoothSolver: Solving for Uy, Initial residual = 1.546522661e-05, Final residual = 1.842282608e-07, No Iterations 1 smoothSolver: Solving for Uz, Initial residual = 1.610282976e-05, Final residual = 1.97009005e-07, No Iterations 1 [9] #0 Foam::error::printStack(Foam::Ostream&)-------------------------------------------------------------------------- A process has executed an operation involving a call to the "fork()" system call to create a child process. Open MPI is currently operating in a condition that could result in memory corruption or other system errors; your job may hang, crash, or produce silent data corruption. The use of fork() (or system() or other calls that create child processes) is strongly discouraged. The process that invoked fork was: Local host: [[49135,0],9] (PID 54348) If you are *absolutely sure* that your application will successfully and correctly survive a call to fork(), you may disable this warning by setting the mpi_warn_on_fork MCA parameter to 0. -------------------------------------------------------------------------- in "/home/user/OpenFOAM/OpenFOAM-5.x/platforms/linux64GccDPInt32Opt/lib/libOpenFOAM.so" [9] #1 Foam::sigSegv::sigHandler(int) in "/home/user/OpenFOAM/OpenFOAM-5.x/platforms/linux64GccDPInt32Opt/lib/libOpenFOAM.so" [9] #2 ? in "/lib64/libc.so.6" [9] #3 ? at btl_vader_component.c:0 [9] #4 opal_progress in "/apps2/openmpi/3.1.0-gcc/lib/libopen-pal.so.40" [9] #5 ompi_sync_wait_mt in "/apps2/openmpi/3.1.0-gcc/lib/libopen-pal.so.40" [9] #6 ompi_request_default_wait in "/apps2/openmpi/3.1.0-gcc/lib/libmpi.so.40" [9] #7 ompi_coll_base_sendrecv_actual in "/apps2/openmpi/3.1.0-gcc/lib/libmpi.so.40" [9] #8 ompi_coll_base_allreduce_intra_recursivedoubling in "/apps2/openmpi/3.1.0-gcc/lib/libmpi.so.40" [9] #9 MPI_Allreduce in "/apps2/openmpi/3.1.0-gcc/lib/libmpi.so.40" [9] #10 void Foam::allReduce<double, Foam::sumOp<double> >(double&, int, ompi_datatype_t*, ompi_op_t*, Foam::sumOp<double> const&, int, int) in "/home/user/OpenFOAM/OpenFOAM-5.x/platforms/linux64GccDPInt32Opt/lib/openmpi-system/libPstream.so" [9] #11 Foam::reduce(double&, Foam::sumOp<double> const&, int, int) in "/home/user/OpenFOAM/OpenFOAM-5.x/platforms/linux64GccDPInt32Opt/lib/openmpi-system/libPstream.so" [9] #12 Foam::PCG::solve(Foam::Field<double>&, Foam::Field<double> const&, unsigned char) const in "/home/user/OpenFOAM/OpenFOAM-5.x/platforms/linux64GccDPInt32Opt/lib/libOpenFOAM.so" [9] #13 Foam::GAMGSolver::solveCoarsestLevel(Foam::Field<double>&, Foam::Field<double> const&) const in "/home/user/OpenFOAM/OpenFOAM-5.x/platforms/linux64GccDPInt32Opt/lib/libOpenFOAM.so" [9] #14 Foam::GAMGSolver::Vcycle(Foam::PtrList<Foam::lduMatrix::smoother> const&, Foam::Field<double>&, Foam::Field<double> const&, Foam::Field<double>&, Foam::Field<double>&, Foam::Field<double>&, Foam::Field<double>&, Foam::Field<double>&, Foam::PtrList<Foam::Field<double> >&, Foam::PtrList<Foam::Field<double> >&, unsigned char) const in "/home/user/OpenFOAM/OpenFOAM-5.x/platforms/linux64GccDPInt32Opt/lib/libOpenFOAM.so" [9] #15 Foam::GAMGSolver::solve(Foam::Field<double>&, Foam::Field<double> const&, unsigned char) const in "/home/user/OpenFOAM/OpenFOAM-5.x/platforms/linux64GccDPInt32Opt/lib/libOpenFOAM.so" [9] #16 Foam::fvMatrix<double>::solveSegregated(Foam::dictionary const&) in "/home/user/OpenFOAM/OpenFOAM-5.x/platforms/linux64GccDPInt32Opt/lib/libfiniteVolume.so" [9] #17 Foam::fvMatrix<double>::solve(Foam::dictionary const&) in "/home/user/OpenFOAM/OpenFOAM-5.x/platforms/linux64GccDPInt32Opt/bin/pimpleFoam" [9] #18 ? in "/home/user/OpenFOAM/OpenFOAM-5.x/platforms/linux64GccDPInt32Opt/bin/pimpleFoam" [9] #19 __libc_start_main in "/lib64/libc.so.6" [9] #20 ? in "/home/user/OpenFOAM/OpenFOAM-5.x/platforms/linux64GccDPInt32Opt/bin/pimpleFoam" [cn374:54348] *** Process received signal *** [cn374:54348] Signal: Segmentation fault (11) [cn374:54348] Signal code: (-6) [cn374:54348] Failing at address: 0x9bbee0000d44c Code:
/*--------------------------------*- C++ -*----------------------------------*\ | ========= | | | \\ / F ield | OpenFOAM: The Open Source CFD Toolbox | | \\ / O peration | Version: 5 | | \\ / A nd | Web: www.OpenFOAM.org | | \\/ M anipulation | | \*---------------------------------------------------------------------------*/ FoamFile { version 2.0; format ascii; class dictionary; location "system"; object fvSolution; } // * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * // solvers { "(p|rho)" { solver GAMG; tolerance 1e-6; relTol 0.01; smoother GaussSeidel; } "(p|rho)Final" { $p; relTol 0; } "(U|e|k|nuTilda)" { solver smoothSolver; smoother symGaussSeidel; tolerance 1e-6; relTol 0.01; } "(U|e|k|nuTilda)Final" { $U; relTol 0; } } PIMPLE { momentumPredictor yes; nOuterCorrectors 3; nCorrectors 2; nNonOrthogonalCorrectors 0; } relaxationFactors { equations { ".*" 1; } } // ************************************************************************* // |
|
![]() |
![]() |
![]() |
|
|