[sita]

Hi all,


I'm using OpenFOAM-5.x on a Scientific Linux cluster. Whenever I run a case in parallel (np = 12), I get some MPI-related warnings. One from OpenFOAM:

--> FOAM Warning : 
    From function static bool Foam::UPstream::init(int&, char**&)
    in file UPstream.C at line 99
    mpi does not seem to have thread support. There might be issues with e.g. threaded IO

And some from the system:

11 more processes have sent help message help-mpi-btl-openib-cpc-base.txt / no cpcs 
for port
[upwind3:21943] Set MCA parameter "orte_base_help_aggregate" to 0 to see all help / error messages

Apart from these warnings the simulation runs fine, but still, I'd rather not get warnings of course. I was wondering: are these warnings related? And what should I do to get rid of these warnings?


Many thanks,
Sita


EDIT: I finally found the rest of the warnings (the ones preceding the "11 more processes have sent help message..." warning):

water.29115hfi_wait_for_device: The /dev/hfi1_0 device failed to appear after 15.0 seconds: Connection timed out
water.29113hfi_wait_for_device: The /dev/hfi1_0 device failed to appear after 15.0 seconds: Connection timed out
water.29119hfi_wait_for_device: The /dev/hfi1_0 device failed to appear after 15.0 seconds: Connection timed out
water.29114hfi_wait_for_device: The /dev/hfi1_0 device failed to appear after 15.0 seconds: Connection timed out
water.29117hfi_wait_for_device: The /dev/hfi1_0 device failed to appear after 15.0 seconds: Connection timed out
water.29112hfi_wait_for_device: The /dev/hfi1_0 device failed to appear after 15.0 seconds: Connection timed out
water.29123hfi_wait_for_device: The /dev/hfi1_0 device failed to appear after 15.0 seconds: Connection timed out
water.29118hfi_wait_for_device: The /dev/hfi1_0 device failed to appear after 15.0 seconds: Connection timed out
water.29120hfi_wait_for_device: The /dev/hfi1_0 device failed to appear after 15.0 seconds: Connection timed out
water.29122hfi_wait_for_device: The /dev/hfi1_0 device failed to appear after 15.0 seconds: Connection timed out
water.29116hfi_wait_for_device: The /dev/hfi1_0 device failed to appear after 15.0 seconds: Connection timed out
water.29121hfi_wait_for_device: The /dev/hfi1_0 device failed to appear after 15.0 seconds: Connection timed out
[water][[11549,1],2][btl_openib.c:880:mca_btl_openib_add_procs] could not prepare openib device for use
--------------------------------------------------------------------------
The OpenFabrics (openib) BTL failed to initialize while trying to
allocate some locked memory.  This typically can indicate that the
memlock limits are set too low.  For most HPC installations, the
memlock limits should be set to "unlimited".  The failure occured
here:

  Local host:    water
  OMPI source:   btl_openib.c:794
  Function:      ompi_free_list_init_ex_new()
  Device:        mlx4_0
  Memlock limit: 65536

You may need to consult with your system administrator to get this
problem fixed.  This FAQ entry on the Open MPI web site may also be
helpful:

    http://www.open-mpi.org/faq/?category=openfabrics#ib-locked-pages
--------------------------------------------------------------------------
[water][[11549,1],10][btl_openib.c:880:mca_btl_openib_add_procs] could not prepare openib device for use
[water][[11549,1],8][btl_openib.c:880:mca_btl_openib_add_procs] could not prepare openib device for use
[water][[11549,1],1][btl_openib.c:880:mca_btl_openib_add_procs] could not prepare openib device for use
[water][[11549,1],6][btl_openib.c:880:mca_btl_openib_add_procs] could not prepare openib device for use
[water][[11549,1],7][btl_openib.c:880:mca_btl_openib_add_procs] could not prepare openib device for use
[water][[11549,1],3][btl_openib.c:880:mca_btl_openib_add_procs] could not prepare openib device for use
[water][[11549,1],9][btl_openib.c:880:mca_btl_openib_add_procs] could not prepare openib device for use
[water][[11549,1],5][btl_openib.c:880:mca_btl_openib_add_procs] could not prepare openib device for use
[water][[11549,1],11][btl_openib.c:880:mca_btl_openib_add_procs] could not prepare openib device for use
[water][[11549,1],4][btl_openib.c:880:mca_btl_openib_add_procs] could not prepare openib device for use
[water][[11549,1],0][btl_openib.c:880:mca_btl_openib_add_procs] could not prepare openib device for use

Our cluster uses infiniband, do I need to give mpirun specific instructions to detect that, or will it be detected automatically?


EDIT 2: Following the instructions on the Open MPI website didn't help so far. Setting memlock to unlimited through ulimit isn't allowed without root privileges, and also editing limits.conf, or adding a file to limits.d, didn't have any effect (at least, not without restarting, and restarting is a no-go here)

[sita]

Never mind, found a workaround: I installed the ThirdParty OpenMPI, which OpenFOAM is using now, instead of the SYSTEMOPENMPI


Still, if anyone knows how to fix these warning messages I'd love to hear about it.