[ocelittle]

I solved this problem; it took me long enough to figure out though I ended up creating an account to post about it. Hopefully, anyone else stuck on this error can use this post to get back on track.

The problem:
Case runs fine in serial but aborts in parallel with an error after a number of iterations

The error:

FOAM FATAL ERROR: (openfoam-2306)
[0] face 34 area does not match neighbour by 0.00525999018275% -- possible face ordering problem.
patch:procBoundary0to1 my area:0.00759092566161 neighbour area:0.00759052639017 matching tolerance:3.95072990508e-07
Mesh face:5190 vertices:4((0.413203006365 2.23398810656 0.3) (0.48979735459 2.23398256701 0.3) (0.489077545139 2.33306188982 0.3) (0.412442235825 2.33306701845 0.3))
If you are certain your matching is correct you can increase the 'matchTolerance' setting in the patch dictionary in the boundary file.
Rerun with processor debug flag set for more information.

The case(s):
I am running a modified FSI tutorial case I extended to 3D by adding layers of cells between the "empty" frontAndBack patch in the blockMeshDict. I am coupling pimpleFoam and solidDisplacementFoam with preCICE via the openfoam-adapter. No edits to any other files in the fluid-openfoam or solid-openfoam directory except fluid-openfoam/decomposePar.

(Turns out the original 2D preCICE perpendicular flap tutorial case won't complete its parallel run either for the same reason - source it from https://precice.org/tutorials-perpendicular-flap.html via https://github.com/precice/tutorials...endicular-flap. But watch out! There's been an update to the preCICE precice-config.xml syntax and the tutorials haven't been updated yet, I'm happy to supply my own file though.)

Further information:
OpenFoam version 2306
preCICE version 3.0.0
openfoam-adapter version 1.3.0
pimpleFoam set up to run on 16 and solidDisplacementFoam on 1 core


Approaching the error:

1. Investigated the mismatching face(s)

   All mismatching faces were automatically written to a .vtk file I loaded into paraview. The faces made up the entire boundary patch between to processors. It did not cut the coupled interface and was not near it either.

2. Checked decomposition methods

   Assuming this to be some sort of decomposition issue, as I had deviated from the tutorial by changing simple --> scotch decomposition, I reverted and gave it another try with 4 cores. Same result at a different time-step. I ran into this result over and over again no matter the method or number of cores I distributed the case over.

3. Checked uncoupled fluid case

   As the coupled case results in mesh morphing via dynamicFvMesh I ran the uncoupled case in parallel (scotch/16) and everything worked fine.

4. Investigated dynamicMeshDict

   At this point I scoured the available online resources for clues. I will append a list further down with all the links someone else may find helpful as well. Most of the content available online relates to cyclic patches, boundary conditions adapted for personal use, or FSI that isn't preCICE. At this point I made an assumption and took a leap of faith that coupling with preCICE wasn't the issue and it was strictly some issue I was having with dynamicFvMesh or generally the mesh motion. If everything else didn't work, I would come back here.

5. Checked matchTolerance set by decomposePar in processor*/constant/polymesh/boundary

   As part of the error refers to a mismatch I considered increasing the the matchTolerance values to allow the parallel run to progress as long as possible. It did allow the case to run for a bit longer, but in the back of my mind, I knew this was a bodge and I couldn't accept it as a valid step towards my ultimate FSI goal: investigating extreme mesh deformations.

6. Checked writePrecision in controlDict

   I had a hunch that the cellDisplacement data was not precise enough so I increased the writePrecision from 6 to 12 to no result.

7. Solved in constant/polymesh/boundary (or blockMeshDict)

   Next on my list of possible solutions was turning the case into a
   "proper" 3D case as I had hithero relied on my frontAndBack patch to be of type "empty". Didn't bother editing blockMeshDict initially and went straight to constant/polymesh/boundary and changed "empty" to "patch" and edited the files in 0 correspondingly. Parallel run completed without a hitch.
   
   I don't know why this fixed it and would be grateful for an explanation.

Other resources I consulted, which someone else might find helpful:
Generally, just to find your way around dynamic meshes:
http://www.wolfdynamics.com/training...s_2021_OF8.pdf

Running into trouble with your own boundary condition:
Processor Poly Patch Error

area does not match neighbour by ... % -- possible face ordering problem

If the interface you're trying to use is cyclic:
Error while using cyclic boundaries and dynamicMesh