[dominik_christ]

This thread is intended for discussion of the draft of future community project guidelines for foam-extent. The draft is on the wiki http://openfoamwiki.net/index.php/Project_Guidelines

IMPORTANT: If there are new points (or alternatives to existing ones) that you would like to add to the project guidelines draft, please create an account on openfoamwiki.org and edit the wiki page!

For genuine discussion and exchange of arguments, please post here.

Thanks!
Dominik

[tomislav_maric]

I haven't posted here for a very long time, but this post made me log in again.

It is nice to see that things are moving forward, regarding a hopefully open development model.

How about using the github pull-request system? There is a chance of having a much easier way of communicating during the review + correction phase.

The communication stays bound to the code in question - on the portal.
There are tools that allow highlighted code snippets that visually pin-point the issues. It is also important that the communication stays open to the public.

One more question: if the goal is to step towards an open development model, why is CodeM different from CodeC? I would expect to see a review process where temporary reviewers are involved in the pull request. In that case, there are some temporary and possibly multiple CodeM roles distributed among the developers for each pull request.

Ideally, the CodeC, having learned how to code clean code, should become a CodeM for his project, and start participating in the review process for other projects. And thusly, the developer base grows.

An open discussion, easily searchable on the internet, can also be a source of information on the design background and rationalle - something quite useful for new library users and possible future contributors. They have a chance of seeing what "doesn't fly" in the sense of code style, as well as language constructs.

[holger_marschall]

Hi all!

I agree with Tomislav in the point that CodeC/CodeM interaction should be revised a bit towards continuity and growth of developer base.

Regarding the question in Wiki:

We need full access to re-direct/delete/stickify posts. Is this possible with cfd-online?

Do we want to have an own separate forum section, i.e. cfd-online.com/Forums/foam/ beside cfd-online.com/Forums/openfoam/? Anyway, the right person to ask would probably be Jonas Larsson.

Regarding infrastructure:
I am willing to help. However, there are a lot of facilities available. Hence, a 'landing page' providing an overview and redirecting to different sites is a very good idea!

best regards,
Holger

[dominik_christ]

Hi Tomislav and Holger, thanks for starting the discussion. Please take this one step further and turn e.g. "should be revised" into action by formulating an alternative and writing it on the wiki page. This will help us to achieve concrete results and ideally we would have 2, 3, 4 options for each topic to choose from.

Let's get it rolling,
Dominik

[tomislav_maric]

@Dominik: Hi, I've edited the site to add the github pull-request based workflow as well as some notes on using Atlassian Bitbucket for the same purpose. Atlassian hosts its web based services for free for open source projects, where teams can be easily organized, projects can be defined, etc.

@all:

Speaking generally, I believe that the mainstream solutions should be followed and kept fully transparent and open, with a clear division of responsibilities.

Teams doing the work on a contribution should be responsible for it and act as a contact base when questions arise about the code: it's use, availability, new feature requests, etc - who else can give a valid estimate of what is working, how much time would an extension take, etc. Teams are easily organized on services like github and bitbucket, and the services are free of charge for OS projects anyway. We have all we need to do the work there. My guess is, since the official release lives on github already, that this would be the better option, although it really makes no difference.

Be it gihub (e.g. boost C++ library), or Bitbucket (e.g. Fenics project), we should stick to what has already been tried out and shown to work well for others.

The Fenics project is a nice example. Check out the video showing the code swarm with analyzed git data here. Projects like this usually start with just a few people, but can only survive with a wider developer base, IMHO.

[holger_marschall]

Hi all,

supporting the established method of pull request, here is a link to read a bit more [link]. Tomislav has already provided the website explaining the work-flow [link].

Before we start anything, let's discuss THE central point IMHO, which we then should clearly fix in the mission statement: Let me try to make my point, which I would like to put for discussion before changing anything.

The point refers especially to what we have seen so far from OpenFOAM/OpenCFD and the question is: Precisely, in what do we want to be different and how to achieve this in a realistic way? What I'd really like to see is a genuine community-driven open development model rather than an formalized way of jointly producing open source. OpenFOAM developers (have a look here for the official definition under item 5) seem to remain focused on distribution issues (i.e. dissemination of source to make business). This is OK (to avoid misunderstanding), however, this is not open development. Here is a citation (from [link])

‘What we are seeing is some projects and entities who are saying here is the source code, it is under Apache licence or GPL or whatever, but the decisions about how the code got to that state are behind some wall – it is not in public.’ This, he argues, is detrimental to the process of coding. Although the end product might be formally classed as open source, it is not a good way to develop sustainable products. He says: ‘You can use it and you have all the rights, but what you don’t have as a developer is an understanding of how the code got that way. So you run into some bug and you say, “Why is the code this way?” There is no historical context or archiving or rationale.’

I have just had a look at openfoam.com/org and their commitment to the users and open source statement:

OpenFOAM comes with full commercial support from ESI-OpenCFD, including software support, contracted developments and a programme of training courses. These activities fund the development, maintenance and release of OpenFOAM to make it an extremely viable commercial open source product. [...]
As the copyright holder, the Foundation licenses OpenFOAM free and open source only, under the GNU general public licence (GPL). The GPL gives users the freedom to modify [...]

What a commitment!? What freedom!? IMO, this is not a commitment but a business model; still this is fair enough (ESI/OpenCFD IS A COMMERCIAL COMPANY). However, the only wording that comes into mind related to freedom is 'free labor'. This has little to do with the original concept of an OSS development model. IMO, this is also the reason why we see very few commits in the unsupported contrib repository: I think most people do not feel any sense of belonging to this kind of project and contribute into such projects where they have no say and no opt of influencing the way the software is developing.

This said, there should be concrete changes to the mission statement first, before going technical. What about a project manifesto to clearly say why it is worth participating? Here is mine (inspired by other OSS development projects)

\begin{description}
	\item[Mission] Our mission is to develop and provide a free and open-source framework for Computational Fluid Dynamics and related fields of Computational Continuum Mechanics --  available under the terms of the GNU General Public License, and based on OpenFOAM\textsuperscript{\textregistered{}} 
	  technology -- remixed from OpenFOAM\textsuperscript{\textregistered{}}, the Open Source CFD toolbox (\url{www.openfoam.com}) and under an open and collaborative development model with a global community. Allow all parties to jointly develop and contribute source code, validation examples, training material and documentation.
	\item[Vision] In our vision, we see:
	  \begin{itemize}
			\item Users contributing and collaborating in the community and around the world
			\item Software that is open source and free (in the sense of freedom) and of high quality adhering to a monolithic standard 
			  in both code and its documentation
			\item A community that is enjoyable and rewarding to participate in, nurture and support the Extend online community by 
			  facilitating transparent and open discussion and by exchanging information and knowledge.
			\item A project that acts autonomously and in a sustainable manner.
			\item A project dedicated to transparency and openness while providing the infrastructural core for exchanging information, 
			  knowledge and know-how in the spirit of the OpenSource development model.
		\end{itemize}
	\item[Values] We avow ourselves to the following key values:
		\begin{itemize}
			\item Usability \& Quality\\
			  (code integrity \& compatibility, validation \& verification standards)
			\item Openness \& Transparency\\
			  encourage open and transparent collaboration throughout all of our work processes; ensure open and free accessibility, both of 
			  the open source code and of its documentation.
			\item Community\\
			  What we do, we do as a community.
			\item Freedom \& Trust\\
			  freedom for all to use, copy, modify, and distribute the code and protect those freedoms using the GNU/GPL; freedom to be a part of the 
			  community and to participate in the future development of the project; web of trust as a foundation of the project; trust in the project to 
			  deliver on its promises, trust within working groups \ldots{}
		\end{itemize}
\end{description}

These are my 2 cents to start a discussion, which I consider very worthwhile and necessary targeted at the project's direction... let's "manifest" a project spirit first, then fix technical questions alongside with discussion of roles for community members.

[dominik_christ]

Hi Holger,
Thanks for your detailed input. As you have surely seen already, there is a suggestion for a mission statement in the draft.

Please join in with Tomislav and put the alternative mission statement from your post into the wiki page as well, so it is clearly visible for everyone (and more convenient to read than latex source ) and does not get lost in discussion history here.

Big thanks!
Dominik

[philippose]

Hello everyone,
A Good Evening and TGIF :-) I was looking through this thread, and I must say... really great :-) I hope that the idea takes on momentum and evolves into a functioning ecosystem.

A couple of thoughts / ideas from my side....:

OpenFOAM (or foam-extend) is a very (very) specialised software not only because of the area of computer aided engineering (Computational Fluid Dynamics / Computational Continuum Mechanics) it serves, but also because of its almost fanatical (in a positive sense) use of C++ with the emphasis on the "++".

Unfortunately, these two strengths of OpenFOAM also work continuously against its survival because the right kind of people... engineers who are fearless of hard core vector mathematics, partial differential equations and physics, and at the same time are also hard core C++ junkies... are hard to come by.

This implies that a fairly rigid hierarchy for the development / bugfix / contribution process needs to be put into place in order to ensure speedy evaluation of these contributions, successful integration into the mainline code base, and effective long-term maintenance.

Due to this reason, I think the concept that is being proposed here is definitely the right way to go. The use of the specific roles "Contributers", "Maintainers", "Topic Admin" and "Project Head" would form an inverted triangle hierarchy through which the proposed changes get funnelled.

The process seems to lean heavily on the Linux Kernel Development model (http://www.linuxfoundation.org/conte...inux-community). In my opinion, the OpenFOAM development hierarchy needs to be divided like the Linux Development Model, into:

(warning... might sound quite similar to the proposal on the openfoamwiki.net, sorry for that...)

[Top Level] -> Project Head -> Someone with very good technical knowledge of the mathematics, physics, code structure and history of OpenFOAM... preferably one of the original founders of the project.

Ideally, the Project head is the only one who merges code into the Main line.


[Topic Level] -> Topic Admin -> Someone with a very good technical knowledge of a particular topic either already present in the code base (for example, Incompressible Turbulence Models) or a topic which has been earmarked for addition into the code base. This technical knowledge should ideally be both.... from the mathematics and physics point of view, and also from the coding point of view.

The Topic Admin is the one who chooses which submissions from within her/his area are mature enough to be passed on to the Project Head for merging into the Main line. This is the person who takes final responsibility of all the code in a given topic.


[Sub-Topic Level] -> Maintainers -> Someone who takes on the role of maintaining a particular area within a topic (for example, the set of k-Omega Turbulence Models within the Incompressible Turbulence Models). This level might go a couple of levels deep depending on the complexity of the topic, and/or may consist of multiple people who are in close correspondence with each other.

The Maintainers are the contact points for the contributors, and form the first official point of entry of patches / contributions into the code base. They need to be responsive, friendly, and have a lot of patience.

I think a non-responsive or unfriendly maintainer would be a major de-motivation for the large number of extremely enthusiastic people out there who want to help, but are either not the best coders (from an OpenFOAM code point of view) or not the best in the particular field/topic.


As for communication, I guess an OpenFOAM forum divided into relevant topics (one for each Topic Admin) coupled with mailing lists for patch submission and/or a Git based workflow would work quite well.


The problems I can see are....:

1. Testing of contributions: Since OpenFOAM is a software for physical simulation, just pure unit-testing of the source code will not be sufficient. Neither is a contribution which compiles flawlessly and runs without any warnings. The contributions have to have some way of testing / proving that the results obtained are what they were expected to be.

Also, question is.... how to decide with a relatively high degree of reliability whether the contribution fails under some specific conditions not originally thought of by the contributor, or affects the results of simulations not directly related to the topic of contribution.

2. Documentation of contributions: How much, what form, code comments, description of the algorithm, special assumptions made, known problems, the list goes on....

3. As of now.... its just the two....


So those were my ramblings regarding this topic.... I hope I didnt bore too many people :-)

Have a great weekend.....

Philippose

[wyldckat]

Just a quick drive-by post:

Originally Posted by dominik_christ

For genuine discussion and exchange of arguments, please post here.

As for fine tuning the discussed topics can be addressed in the talk section of said wiki page: http://openfoamwiki.net/index.php/Ta...ect_Guidelines


edit: I forgot to comment on this:

Originally Posted by holger_marschall

Do we want to have an own separate forum section, i.e. cfd-online.com/Forums/foam/ beside cfd-online.com/Forums/openfoam/?

This seems a bit related to this thread: http://www.cfd-online.com/Forums/sit...ams-forum.html

[tomislav_maric]

@philippose

I agree with 90% of what you wrote, but I just must say that the idea of having a Project Head that does the merges to the master branch, as a single person, does not scale well at all.

In case that we really do get organized and there are multiple projects up and running - one person dealing with merge conflicts for weeks is exactly what we do not want. We need to have more people somehow involved in that process, otherwise there will be no chance to have a steady release rate.

So I would just like to ammend your proposal with a project board, which is then a team of people responsible for merging the feature branches into the dev branch, and consequently, preparing the master release.

@all:

Talking about organization is fine, but we should not forget the fact that we are talking about a code development model and in that aspect, I would follow the original branching model proposed by Hrvoje in the release notes.

• If we just organize the teams around the branching model, it will map well with git.
• This is what all the people currently involved in maintaining the project are used to.
• If we use github / bitbucket, we have a ready-to-go and free (no costs, no maintenance issues, etc) platform for team organization and project planning.

We shouldn't, IMHO, talk about organization detached from the code dev model, because it might happen that what we come up with is not scalable, or not transparent, or different from established practices, ready to be used and already used by thousands of other open source projects.



It really is that simple... Everything is already there, we just need to start.

[holger_marschall]

Hi,

all I'd really like to see is a critical mass of community members readily prepared to launch a true open source development project.

It is as easy as already indicated: let's use the established workflow based on Jira/Confluence/Bitbucket by Atlassian, for instance. For a really professional technical solution here is the link for the form (open source project get it all for free): https://www.atlassian.com/software/v...icense-request. It takes no more than a couple of minutes to fill the form.

I just see that we are currently building up infrastructure twice: http://sourceforge.net/projects/foam-extend/

So IMHO there are two simple questions:

1. Which platform to choose? (I would really recommend Jira/Confluence/Bitbucket by Atlassian)
2. Do we have a critical mass for a go?

The last item essentially ALSO raises the questions: Do we have a commitment by the main developers/contributors/extend-admins? With this, do we succeed in establishing an open source development model for free software? - in contrast to just providing the technical means to jointly produce open source.

I just want to avoid wrong expectations: IMHO there is a huge difference in

• open source: One can look at the source code. It's free to download, use and extend the software; and the license ensures that this stays so. (This I assume is the way most people see OpenFOAM(R)).
• free software (open source development model): users and developers are themselves responsible for their contribution, have a say and options to influence the way the software is developing.

It's the difference of the word "free" in "free beer" and "free speech". (see here: https://www.gnu.org/philosophy/open-...-point.en.html).

By choosing the technical platform (including infrastructure!) we already make a choice with respect to the above topic. However, we need the critical mass and quite a number of project maintainers and developers with us to have serious base for a start!

best regards,
Holger.

[tomislav_maric]

@Holger: yep, agreed with all your points.

Regarding the questions:

Originally Posted by holger_marschall

Hi,

I just see that we are currently building up infrastructure twice: http://sourceforge.net/projects/foam-extend/

So IMHO there are two simple questions:

1. Which platform to choose? (I would really recommend Jira/Confluence/Bitbucket by Atlassian)
2. Do we have a critical mass for a go?

A 1. I did a very silly thing, because git really is that simple and I am that simple minded. It took me about 7 minutes (including the upload time) to set up the repositories both on github and bitbucket:

https://github.com/tmaric/foam-extend-3.0
https://bitbucket.org/tmaric/foam-extend-3.0

To those interested in creating more momentum, sign up and investigate the services (this includes the existing sourceforge). Fork the repositories, read about the platforms, simply check things out.

I propose we set up a poll with a final decision date about using: github, bitbucket and staying at sourceforge, which should help us decide which way to go.

A 2. This is not a question I think wil be answered in a sentence. I don't believe that people will log in to this thread and answer, "yes, we are the critical mass"

It will either work out or not, there is no way to tell...

I would like to see the current maintainers, responsible for the project at this point, to comment on this thread. Since Dominik started this, I guess Hrvoje has some thoughts on this as well.. Bernhard commented on the Wiki so I'm guessing he's up for it...

[tomislav_maric]

Here is a poll for the collaboration platform.

[holger_marschall]

@Tomislav

Reading carefully, the central question was

Do we have a commitment by the main developers/contributors/extend-admins?

So you raised the same question as I did (at the bottom of your last post)

As for your vote, there is stuff missing, IMHO (Jira/Confluence would be really helpful). Please have a look here (scroll to the software items to choose from): https://www.atlassian.com/software/v...icense-request

best regards,
Holger.

[tomislav_maric]

Originally Posted by holger_marschall

@Tomislav

So you raised the same question as I did (at the bottom of your last post)

That's definitely an important question.

As for your vote, there is stuff missing, IMHO (Jira/Confluence would be really helpful). Please have a look here (scroll to the software items to choose from): https://www.atlassian.com/software/v...icense-request

Jira/Confluence are not missing, they are an integral part of the bitbucket web based framework. Same goes for the github pull request system on github.

Once there is a user base there, both services are freely available - so it's pretty clear that if we choose bitbucket, we automatically opt for Jira/Confluence/Greenhopper, etc. On the other hand, if we choose github, we automatically opt for the pull request, team organization, etc on github.. It's unlikely that you'll see someone using github and then working with Jira on bitbucket for the same code - that wouldn't make any sense, since it excludes the possibility to connect code with the collaborative work (highlights during reviews and such stuff)...

[holger_marschall]

Thanks for clarifying. Indeed, technically all is fine.

Edit: However, I just see bitbucket OnDemand as offered by Atlassian available in their Open Source Project License Request!?

Comment: https://confluence.atlassian.com/dis...counts+to+JIRA - all possible

[tomislav_maric]

Originally Posted by holger_marschall

Edit: However, I just see bitbucket OnDemand as offered by Atlassian available in their Open Source Project License Request!?

Comment: https://confluence.atlassian.com/dis...counts+to+JIRA - all possible

Yes, they seem to offer all their services to a "qualified open source project" free of charge:

"
Free for open source

Qualified open source projects can receive free subscriptions for Atlassian OnDemand offerings.

"

Edit: although a good point in considering github as an alternative is the visibility aspect. github hosts 10e06 projects, way more than bitbucket. This IMHO increases the visibility of the project - that is a huge pool of programmers and companies.

[dominik_christ]

Hi all, a few quick comments:

In my opinion, two useful principles to evaluate everything would be:

1) Make it as easy as possible for someone who thinks "I want to help and contribute" to do something useful.

2) Make it as little effort as possible for "heavy contributors".

If I look at some of the points raised so far in the light of these two principles, I see:

a) Currently it is necessary to ask an admin to get write access to the repo. In github/bitbucket, this is not necessary, as anyone can branch of directly (please correct me, if I'm wrong about this).

b) If the project head needs to do all the merges himself, he will not have any time to do any further development, which would not be beneficial to the progress of the project. So putting this task onto multiple shoulders appears better to me.

c) Changes to the present infrastructure should have a benefit regarding the two principles above. Just changing platform for its own sake (or for having "all by one provider") just creates additional work for admins and other heavy contributors.

An (incomplete) list of infrastructure we currently have:

* Code hosting: git on SourceForge / extend-bazaar on openfoamwiki.org
* Bug tracker: Mantis on SourceForge
* Forum: cfd-online
* Wiki: openfoamwiki.org
* Social platform: extend-project.de

Some of this can be, and needs to be, expanded (e.g. landing page on
SourceForge; commit announce sub-forum on cfd-online).

I would find it very useful if you could help with arguments how any changes to this infrastructure would be beneficial in terms of the 2 principles above.


Regarding the critical mass of contributors, this is a kind of hen and egg problem: In a free and open development, nobody can summon a critical mass out of nothing. All that we can do is to make contributing as clear (guidelines), easy (infrastructure) and accessible (public discussion) as possible.

Best regards,
Dominik

[tomislav_maric]

Originally Posted by dominik_christ

Hi all, a few quick comments:

In my opinion, two useful principles to evaluate everything would be:

1) Make it as easy as possible for someone who thinks "I want to help and contribute" to do something useful.

2) Make it as little effort as possible for "heavy contributors".

... snipped ...

* Code hosting: git on SourceForge / extend-bazaar on openfoamwiki.org
* Bug tracker: Mantis on SourceForge
* Forum: cfd-online
* Wiki: openfoamwiki.org
* Social platform: extend-project.de

Some of this can be, and needs to be, expanded (e.g. landing page on
SourceForge; commit announce sub-forum on cfd-online).

I would find it very useful if you could help with arguments how any changes to this infrastructure would be beneficial in terms of the 2 principles above.


... snipped ...

Best regards,
Dominik

From the top of my head:
changing code hosting to github/bitbucket

principle 1)

Both github/bitbucket are based on extensive use of 'forks', for which you don't need a write access to the repository, so anyone can fork the repo and work on it. There is supposed to be a similar option on SourceForge, but I couldn't find it - I stand to be corrected on that point.

principle 2)

Both github/bitbucket make it easier for "heavy contributors" to work with contributors, for reasons stated above - web based front ends like the pull request system. As for their day-to-day work: nothing changes, it's about vim/emacs + git plugins in the end, no hosting platform impacts that.

Cfd-online forum

Compare this forum, and the searchability of the answers with sites that support upvotes and tags, like StackOverflow and Computational Science Stack exchange. Things get answered here, and are left burried in discussions, often difficult to find via web search, in my experience.

One question: what exactly do you mean by 'commit announce sub-forum'?

[dominik_christ]

Hi Tomislav,

by "commit annouce sub-forum" I mean that each commit to the master branch (and maybe nextRelease branch as well) should have a short accompanying email which quickly explains what was fixed or what was added. Then everyone subscribing to this thread would know how relevant this particular change is to him/her.

Best regards,
Dominik