[alexeym]

Dear all,

It is more or less a rant.

Guess it was unfortunate day but still I am wondering if there is any logic behind OpenFOAM's version numbering? It consists of 3 digits (yet), let us say A.B.C. In general in OSS world version A.B.C is compatible with "(A\.B\..*)". And A.*.* is in general incompatible with (A+1).*.*.

Still...

While going from 2.2.1 to 2.2.2 naming of G field in turbulent models was suddenly changed. OK not a big issue. Still it has broken my turbulent models.

2.3.0 -> 2.3.1, wedge patch API changed. And it was not just addition of a methods, it was change of API (patchNormal -> n, cosAngle suddenly appeared). Well, this again broke my code.

Is there a way to employ conditional compilation except by providing -D flag during compilation?

Thanks for your attention. End of the rant.

[wyldckat]

Hi Alexey,

You're not the first person to have this issue and certainly not the last. And if you had to support foam-extend and Caelus-CML, it would get even more crazy...
For the macro's management system, two examples I can think of are:

• waves2Foam - http://openfoamwiki.net/index.php/Co...Versions_of_OF
  • Which even has explained on the wiki how the version tagging is done, so that in the code there can be of some use for "#ifdef".
• swak4Foam - http://openfoamwiki.net/index.php/Contrib/swak4Foam
  • This is actually the first tool-kit I know of that had to implement such a strategy. I was with following (and helping a bit) this project, that I found some larger changes, where 2-4 weeks after a minor release of OpenFOAM was made, there was a substantial update to the function object API interface.

Now, as for the reasons for these changes between minor versions... the main reasons are simple:

• Resources: Lack of enough resources. This to say that there aren't (or least there weren't, not sure how it's now) enough people at OpenCFD to make such a version numbering strategy to work according to the usual convention.
• Science: The development of scientific simulation software. This to say that we should never assume things work as intended with each new version, minor or major or bug fix. Always do your own battery of tests to ascertain if things still work as intended.

On the "resource" front, just compare:

• On the same line of work, compare the team size of OpenCFD over the years, versus the team size at CD-adapco and/or ANSYS.
• On the OSS line of work, compare the team size of OpenCFD with the ones for LibreOffice, GNOME, KDE and so on. I'm not even mentioning the Linux Kernel, because it's massive and very well funded.

Therefore, if there aren't enough resources to go around, then what do you propose for at least the following issues:

1. When should such new features be added in?
2. If it's a design flaw, when should it be fixed?

The closest to the optimum development line we've gotten so far for a "proper" line of development, was soon after OpenFOAM's 10th anniversary (~4 months ago?), Henry Weller finally made a public OpenFOAM-dev repository: https://github.com/OpenFOAM/OpenFOAM-dev/
This was asked... I don't know exactly how many years ago (probably since OpenFOAM 1.1 or before?), but the oldest accurate time frame I remember of was when this blog post was published: http://olesenm.github.io/2009/11/24/...FOAM-versions/ - back in November 2009, because of this thread: http://www.cfd-online.com/Forums/ope...1-5-1-6-a.html


[RANT]
Such a repository hadn't been made public sooner for several reasons... and OpenCFD's own development repository still isn't public either. But neither is Engys' nor ICON's forks, ever since from their own start of forking from OpenFOAM: http://openfoamwiki.net/index.php/Fo...Variants#Forks
And do keep in mind that they are all entitled to do so!

Now, in-spite of OpenFOAM's changes between minor versions, there is one great advantage that we have over any other closed source code: Anyone can do their own fork/variant and provide "proper" minor versions, similarly to how major Linux distributions patch the versions that are installed in each major version. But again, this requires resources, which if someone isn't willing to either pay for or contribute to, will likely never happen... or if it does, it will only work for as long as someone is willing to work on it.
[/RANT]

Best regards,
Bruno

[alexeym]

Dear Bruno,

Thank you for the answer. Below are my comments (maybe too naive).

Concerning conditional compilation, both examples use Allwmake script and pass OpenFOAM version number as external definition (swak4Foam creates header with #defines, waves2Foam pass version number via -D flag in Make/options file). So I guess there is no native mechanism for conditional compilation.

Resources: Lack of enough resources.

Comparison with ANSYS is not quite correct. OpenFOAM does not have GUI and many other things present in ANSYS.

Science: The development of scientific simulation software. This to say that we should never assume things work as intended with each new version, minor or major or bug fix. Always do your own battery of tests to ascertain if things still work as intended.

Does this mean there is absolutely no meaning in versioning? Also guess we need to separate API versioning, and physics included in the solvers versioning. So maybe it is even worth to have separate versioning for solvers/models and library?

Therefore, if there aren't enough resources to go around, then what do you propose for at least the following issues:
When should such new features be added in?
If it's a design flaw, when should it be fixed?

So in fact there is a need for API and physics included in the solvers/libraries versioning (also we need to somehow include developers' desire to rename variables in solvers, which leads to incompatible fvSchemes/fvSolution changes). And yet this can be done within semantic versioning scheme. Surely version will change more rapidly (or maybe not) but still it would be more convenient for developers/users, so can finally increase available resources.

[wyldckat]

Hi Alexey,

Originally Posted by alexeym

Concerning conditional compilation, both examples use Allwmake script and pass OpenFOAM version number as external definition (swak4Foam creates header with #defines, waves2Foam pass version number via -D flag in Make/options file). So I guess there is no native mechanism for conditional compilation.

One was proposed several months (over 3 years) ago: http://www.openfoam.org/mantisbt/view.php?id=293 - It was closed and no comment was given by Henry. My guess is that this didn't affect OpenFOAM itself, it only affected 3rd-party code.

Originally Posted by alexeym

Comparison with ANSYS is not quite correct. OpenFOAM does not have GUI and many other things present in ANSYS.

It takes as many people working on the solvers and physics modelling, as it takes to do the GUI. When OpenFOAM had a GUI (FoamX), they only had 1 or two people working on it.
Keep in mind that the company personnel is divided into 4 major sections:

1. Software development
2. Software support
3. Delivering courses
4. Business management

Software maintenance is usually mixed into the first two. ANSYS has at least 10x more people than OpenCFD: http://www.openfoam.org/version2.3.0/ - at the bottom of that page you can see that there were 10 people on the whole team. These 10 people were divided throughout the 4 major sections, even if ESI was backing them up with business management work.

Originally Posted by alexeym

Does this mean there is absolutely no meaning in versioning? Also guess we need to separate API versioning, and physics included in the solvers versioning. So maybe it is even worth to have separate versioning for solvers/models and library?

Version numbers are good points of reference. Even though the developers aren't enforcing a stricter and stable bug fixing strategy, the version numbers are still critical.

Perhaps what you're pushing for is that there should be 4 lines of development (i.e. the usual in OSS):

1. Stable - always compatible within the major version.
2. Bug fixes, branched into:
   1. Stable bug fixes, which go into the next stable minor version.
   2. Unstable bug fixes, which go into the next major version only.
3. Development - done either publicly or behind the scenes.

They've technically been mixing the two branches for bug fixes.

Originally Posted by alexeym

So in fact there is a need for API and physics included in the solvers/libraries versioning (also we need to somehow include developers' desire to rename variables in solvers, which leads to incompatible fvSchemes/fvSolution changes).

Having version numbers per library... well, you can propose that on the bug tracker: http://www.openfoam.org/bugs/ - but you're going to need a very strong and convincing case for them to implement it, as well as stating why you're proposing this.

Originally Posted by alexeym

And yet this can be done within semantic versioning scheme.

Mmm... it doesn't sound like "very elegant code", and OpenFOAM is all about being elegant, in the sense of keeping the code easy to maintain... and not being a snowball of hacks on top of other hacks...
Once OpenFOAM starts adding in more and more macros, it'll become a major mess to maintain

But again, if you can make a good case for them to abide to either not making so many changes to the API between minor versions...

Originally Posted by alexeym

Surely version will change more rapidly (or maybe not) but still it would be more convenient for developers/users, so can finally increase available resources.

The problem they will very likely point out is that this will make it harder to maintain.

What I can imagine happening, based on your description, is that they will then have several variants of each library, which won't be compatible with all other libraries versions.

The main way to avoid these weird massive changes between minor versions would be to make sure to have the whole API backbone properly defined for each major version and then enforcing that said API cannot change within the same major version.
But what I expect their answer will be is something along these lines: if you're willing to sponsor or contribute such a feature, we'll do it and/or adopt it.


The way I can see it happening, is if each major break that rose from bug fixes and implementations between minor versions is properly diagnosed and defined how it could have been avoided, they'll welcome the solutions that come with such a diagnosis endeavour.

Best regards,
Bruno

[alexeym]

Hi Bruno,

Thanks for your answers. Below again my maybe naive comments.

Originally Posted by wyldckat

One was proposed several months (over 3 years) ago: http://www.openfoam.org/mantisbt/view.php?id=293 - It was closed and no comment was given by Henry. My guess is that this didn't affect OpenFOAM itself, it only affected 3rd-party code.

So this basically means - you are free to develop any hacks to workaround our strange decisions about API changes between versions. This either means that OpenFOAM's versions are just random numbers, or third part software is something undesirable (or at least undesirable for OpenFOAM, as foam-extend community is quite happy about swak4Foam and PyFoam).

Version numbers are good points of reference. Even though the developers aren't enforcing a stricter and stable bug fixing strategy, the version numbers are still critical.

Then why choose quite misleading 3-number version numbering instead of just using natural numbers? In fact my initial question was about logic behind those three numbers. How developers decide that this release is 2.3.0 and that 2.3.1?

The main way to avoid these weird massive changes between minor versions would be to make sure to have the whole API backbone properly defined for each major version and then enforcing that said API cannot change within the same major version.
But what I expect their answer will be is something along these lines: if you're willing to sponsor or contribute such a feature, we'll do it and/or adopt it.

So this basically means: you are free to make a fork and impose your own version numbers.

Cordialement,

Alexey

[wyldckat]

Hi Alexey,

Originally Posted by alexeym

So this basically means - you are free to develop any hacks to workaround our strange decisions about API changes between versions.

Uhm, yes, it's open source, we're (almost) free to do whatever we want
As for strange changes... well, it's very hard to design a perfect infrastructure that never needs to be changed ever again.

Originally Posted by alexeym

This either means that OpenFOAM's versions are just random numbers,

They aren't random. They simply aren't fully abiding to the criteria you're asking for, even though many others do it.

Originally Posted by alexeym

or third part software is something undesirable (or at least undesirable for OpenFOAM, as foam-extend community is quite happy about swak4Foam and PyFoam).

We'll... software development is sometimes tricky, mainly because of the "support" side of the development. And if the project itself is not properly funded/supported... then we have to rely on what the authors can provide. Because there are a few sides to this, where:

1. OpenFOAM is not more "stable", simply because they don't have enough resources to make it so. If you search for some of the older threads back in 2005, or look at this page: http://openfoam.com/news/6th-anniversary.php - you'll see that things aren't as great as we might want them to be
   Then try to compare to this:
   1. http://www.libreoffice.org/about-us/credits/
   2. https://wiki.documentfoundation.org/...an#4.3_release
   3. and check why OpenOffice was in a stand-still for so long.
2. Third-party software that is built around OpenFOAM could simply take advantage of the repositories features for multiple branches and therefore provide code dedicated to each version. It's a bit harder to maintain, but at least it keeps the code clean from messy macros all over the place I wrote about not long ago... ah, here: http://www.cfd-online.com/Forums/blogs/wyldckat/2499-ideas-managing-repository-supports-several-openfoam-versions.html
   
3. Uhm... and foam-extend also has its flaws... and I don't want to point them out... otherwise I might be called in to fix them
   But if you look at swak4Foam's development history, you'll see that it also has to deal with changes between 1.6-ext and 3.0-ext.

Originally Posted by alexeym

Then why choose quite misleading 3-number version numbering instead of just using natural numbers? In fact my initial question was about logic behind those three numbers. How developers decide that this release is 2.3.0 and that 2.3.1?

Well, it's harder to tell apart versions if they were named by the year they were released in
For example, if OpenFOAM 2.3.0 was 2014a and 2.3.1 was 2014b. And then foam-extend would also have its own 2014a (3.1) and 2014b (3.1.1)...

The decision for the numbers is somewhat simple - If the changes were only related to bugs and some features that were missing in the previous stable release, then it's still in the same major release. 2.3.0 and 2.3.1 share this fact and if you compare the two release notes pages:

1. http://www.openfoam.org/version2.3.0/
   

   Version 2.3.0 is a major new version containing significant developments, outlined below:

2. http://www.openfoam.org/version2.3.1/
   

   Version 2.3.1 consolidates the major version 2.3.0 release through a large number of bug fixes and code improvements, included in over 400 code commits and over 800 file changes.
   [...]
   Version 2.3.1 also contains small, new developments, some of which are listed below.

The other example is when the change from 1.7.x to 2.0.0 occurred because there were a very sizeable number of changes that made the code considerably different: http://www.openfoam.org/version2.0.0/

Originally Posted by alexeym

So this basically means: you are free to make a fork and impose your own version numbers.

Uhm... well, if it weren't for this kind of freedom, we wouldn't have Ubuntu and Fedora and may other Linux Distributions, with their own stable and supported Linux kernel versions that are maintained throughout the lifetime of each distribution version!

---------

By the way, did you check if the problem you initially referred to here on this thread is stated in the release notes for 2.3.1?

---------

But let's see a practical scenario, based on a recent bug fix:

• https://github.com/OpenFOAM/OpenFOAM...7e2d607818a30c
• which refers to this bug report: http://www.openfoam.org/mantisbt/view.php?id=1652

How do you suggest that such as situation be handled? Namely, when should this fix be released in a stable version?


Best regards,
Bruno

[alexeym]

Hi Bruno,

Thank you for the answer.

Originally Posted by wyldckat

They aren't random. They simply aren't fully abiding to the criteria you're asking for, even though many others do it.

Well, it is like saying: "Russian bureaucracy is not fully deterministic (comply to the country laws) but in general...". Finally you try to avoid interactions with the state as you never know what will happen next.

Unfortunately comparison with LibreOffice is not quite correct, as it is twice older, and surely have wider audience.

Well, it's harder to tell apart versions if they were named by the year they were released in
For example, if OpenFOAM 2.3.0 was 2014a and 2.3.1 was 2014b. And then foam-extend would also have its own 2014a (3.1) and 2014b (3.1.1)...

Since any way people have to read a) release notes, b) run set of tests to check if developers have broken something in the new version, well... versions just can be 1, 2, 3, 4, 5... (0.0.x if 3 digits are critical). In my opinion version numbering is something to indicate not only addition of new features but also backward compatibility. And if developers in general break API/ACI (configuration interface ) every version... well it is acceptable in "forever beta". I use rather narrow subset of OpenFOAM, and still encountered several times backward incompatible API changes while doing minor version upgrade. I skip mopping about constant desire to rename variables so even fvSchemes become incompatible between versions.

By the way, did you check if the problem you initially referred to here on this thread is stated in the release notes for 2.3.1?

If you refer to "release summary", in general "We have renamed patchNormal method of wedgePolyPatch class into n" does not go to this page.

But let's see a practical scenario, based on a recent bug fix:

• https://github.com/OpenFOAM/OpenFOAM...7e2d607818a30c
• which refers to this bug report: http://www.openfoam.org/mantisbt/view.php?id=1652

How do you suggest that such as situation be handled? Namely, when should this fix be released in a stable version?

Maybe I miss something but it is a feature request, is it not? And as it a) breaks old configuration files, b) from the first view do not break API. The patch can be included into next minor release 2.4.0 if it contains fallback to the previous behavior (i.e. if we did not find <entryName>Coeffs, look for csvFileCoeffs). But still it does not, it should go to 3.x.x releases, also there is workaround to achieve similar behavior (though less elegant) without any patches.

Cordialement

[wyldckat]

Hi Alexey,

Originally Posted by alexeym

Well, it is like saying: "Russian bureaucracy is not fully deterministic (comply to the country laws) but in general...". Finally you try to avoid interactions with the state as you never know what will happen next.

Now I'm starting to see even better the "big picture"... that gives more support as to why politics and economics are unwilling to abide to the universal laws of the universe

Originally Posted by alexeym

Unfortunately comparison with LibreOffice is not quite correct, as it is twice older, and surely have wider audience.

I'm not sure by which dates you are measuring, but either way, yes, LibreOffice has a much wider audience.

Originally Posted by alexeym

Maybe I miss something but it is a feature request, is it not? And as it a) breaks old configuration files, b) from the first view do not break API. The patch can be included into next minor release 2.4.0 if it contains fallback to the previous behavior (i.e. if we did not find <entryName>Coeffs, look for csvFileCoeffs). But still it does not, it should go to 3.x.x releases, also there is workaround to achieve similar behavior (though less elegant) without any patches.

Problem is that such a scenario would require to have 3 versions in public development:

• 2.3.x bug fixes and possible associated stable releases.
• 2.4-dev development line
• 3-dev development line

Mmm... now that I think about it... I'm starting to see your point. They might be OK with the idea of dropping their current number version system for one with less one digit and skipping the need for "2.4-dev" entirely.

Furthermore, GCC has (roughly) switched to that numbering convention about a year ago, when they began the development of GCC 5.0 and recently released 5.1, beginning 6.0 when 5.1 was released. The 5.0 was originally going to be 4.10, but they ended up deciding that the numbering was outdated. Their current development guidelines explain it in more detail: https://gcc.gnu.org/develop.html

The Linux kernel did the same, when it was going to develop... what was it... 2.6.40? And instead switched to 3.0.

Firefox, Chrome and a few others went ahead with tons of new versions, although they still follow the 3 or 4 number convention... it's just the major number goes up a lot more frequently.


Took me a while to see your point, but now I do agree with you, at least for future developments. Fixing OpenFOAM 2.3 at this point would be... complicated
In theory, this is possibly what they originally had in mind when they began OpenFOAM 1.0, but... things rarely goes exactly as planned in any number of situations .


OK, I support this idea of yours! Go ahead and report this as a bug/feature request, I'll comment in favour of this idea as well!

Best regards,
Bruno

[alexeym]

Hi Bruno,

Originally Posted by wyldckat

I'm not sure by which dates you are measuring, but either way, yes, LibreOffice has a much wider audience.

As LibreOffice is a fork of OpenOffice and OpenOffice is a descendant of StarOffice (https://wiki.openoffice.org/wiki/A_B...OpenOffice.org) we can assume LibreOffice development started in 1986

Took me a while to see your point, but now I do agree with you, at least for future developments. Fixing OpenFOAM 2.3 at this point would be... complicated
In theory, this is possibly what they originally had in mind when they began OpenFOAM 1.0, but... things rarely goes exactly as planned in any number of situations .


OK, I support this idea of yours! Go ahead and report this as a bug/feature request, I'll comment in favour of this idea as well!

Now I am not quite sure that I presented the idea correctly. My proposal is to put following meaning in those three numbers representing version (N1.N2.N3):

N3 is a bug-fix branch. No new features, no API changes. So everything that works with N1.N2.N3 MUST work with N1.N2.(N3 + L) (but something that was not working in N1.N2.N3 MAY start working in N1.N2.(N3 + L) due to bug fixes). From one point of view this number is not significant, still it can be used for reference (like "the bug was fixed in 2.3.2"). Surely this approach introduces necessity to formalize WHEN this number is increased.

N2 is a backward compatible API/ACI changes. In this branch new features are introduced.

And finally N1 can introduce backward incompatible changes. Guess one can assume than software is completely rewritten between N1 and N1+1.

(Hm, thought I have mentioned "semantic versioning" (http://semver.org) in my responses but can not find it in the thread, so guess that is why the proposal was not clear at all)

[wyldckat]

Hi Alexey,

Originally Posted by alexeym

As LibreOffice is a fork of OpenOffice and OpenOffice is a descendant of StarOffice (https://wiki.openoffice.org/wiki/A_B...OpenOffice.org) we can assume LibreOffice development started in 1986

The development in FOAM started in 1989, so it's only a difference of 3 years :

Originally Posted by http://cfd.direct/openfoam/

OpenFOAM was created by Henry Weller in 1989 under the name “FOAM” [...]

Originally Posted by alexeym

Now I am not quite sure that I presented the idea correctly. My proposal is to put following meaning in those three numbers representing version (N1.N2.N3):

[...]

(Hm, thought I have mentioned "semantic versioning" (http://semver.org) in my responses but can not find it in the thread, so guess that is why the proposal was not clear at all)

Wow! Well, this is certainly a good start for convincing them, given the website is considerably detailed in how to handle versions.

But I've got a feeling that it's more likely they'll accept the idea of going with mostly major versions, than abiding to this stricter version numbering of "major.minor.patch" .

I say (write) this based on what I've seen so far in the development of OpenFOAM. They originally - OpenFOAM 1.0 to 1.4 - didn't even have patch versions and the minor versions were most of the time API/ACI incompatible between them.
I do know of a case where the 3rd party code was resilient enough to go from 1.4.1 (or 1.5 ?) to 2.2 without the need for changes in the code: http://openfoamwiki.net/index.php/Contrib_icoStructFoam - but this is somewhat of a rare situation.

Again, based on what I know, the most likely answers are:

1. Either it stays the same as it currently is, namely with "major.minor.patch", unless sponsors come forward to back up the resources needed for maintaining 2 or 3 simultaneous versions that abide to the strict semantic version system.
2. Or they are willing to switch to a primarily "major....patch" version, where minor numbers are issued only when deemed necessary... for example, for stable releases. In other words, the releases continue to suffer from the same issues you're complaining about (i.e. broken backward compatibility), but at least it respects the semantic version system.

Because from what I'm seeing so far in the 2.3.x repository... features from OpenFOAM-dev are coming into 2.3.x based on the criteria of being "mostly beneficial and stable". If this work-flow is going to change, it's going to likely only happen starting in the next major (or minor) version is released.
This to say that if you really want this issue to be fixed, you better report this sooner than later!

Best regards,
Bruno

[alexeym]

Hi Bruno,

http://www.openfoam.org/mantisbt/view.php?id=1674

and according to commentary from Henry there will be less API-breaking changes as they will go into -dev repository.

[wyldckat]

Hi Alexey,

Many thanks for reporting it. And my apologies for not commenting in the bug report (only a few hours ago did I start looking into community topics), although I'm not sure what I could have stated, given Henry's answer.

Nonetheless, if you or anyone else happen to spot API/ACI breaking behaviour, let us try and point it out to them as soon as possible.

Best regards,
Bruno