[aerosayan]

As the question states, how do we design our code architecture and write our code in a way that makes it flexible, and allows us to later add different solvers, flux calculations, chemistry models, memory layouts, iteration patterns etc. without breaking everything and a major code refactor?

Obviously no one knows everything from the start, so planning for every different change we might make, is wishful thinking. However, this is also a very important challenge that needs to be solved.

Looking for strategies that might allow us make this process easier.

Example challenge that I faced while improving my code: I have an explicit structured solver, and previously I traversed through a N ROW * M COL large grid, column by column. I wanted to implement a method to store the huge grid in smaller blocks of structured grids (say 8 ROW * 8 COL each, or 32 * 32 or 100 * 100 etc) such that each different columns were closer to each other in memory, thus improving code performance. I don't have an accurate picture, but think of them as the different blocks shown in the picture below.

This example challenge, required somewhat of a major code refactor, and is finally on the way to completion. Essentially, a single change in our memory layout directly caused a major change in our access pattern, thus requiring a major refactor.

It would have been useful if the code was designed with future modifications in mind.

So, basically, I wanted to ask...

1. How do you design your code architecture?
2. What kind of challenges you faced?
3. If possible, kindly share how you solved them.

[arjun]

"Obviously no one knows everything from the start, so planning for every different change we might make, is wishful thinking. "


Thinking thinking and more thinking. This is how.


When I designed Wildkatze which is basically a finite volume framework (Will also add Finite Element Framework too) I spent almost 8 months on just thinking the framework and designing it.

This is why Wildkatze has so many concepts that you do not find in any other Solver. Some examples:

1. C++ based user coding that allows user very very high level of access to internal data unlike any other solver that is precompiled. (Openfoam can do this but need to compile).

2. Concept of partial regions for application of models on them.

3. Phase to phases and region to regions mappings to connect the interactions.

4. Model features to enhance models.

5. Definition of physics models on boundaries and interfaes

Just to name very few of things.

All this because I spent months thinking about how and what kind of models the solver can probably grow into and how this shall all be expanded. Then the whole project was written around the design.

[sbaffini]

I think there are multiple answers to this question. For example, you focus a lot on certain stuff that I typically don't care at all. So what for me might be 10 lines of code, for you might well be 100.

Besides this, and what Arjun already said, if you can't design in advance, the idea is that you recognize patterns while you implement stuff.

In the end, I think, it is not different from general software development, except that there are few first citizen concepts that you may want to honor (regarding the hierarchical relation between grids, variables, solvers, etc.), yet it is not necessarily certain what the best choice is. For example, my code has changed 10 times its current LOC, yet every version was kind of what, at the time, I felt most easy to expand. In practice, it is until it isn't anymore and you change pattern to recover that

[flotus1]

Essentially, a single change in our memory layout directly caused a major change in our access pattern, thus requiring a major refactor.

It would have been useful if the code was designed with future modifications in mind

If I am not mistaken, what you did here falls under the term spatial blocking.
From my experience, you can't have both. Flexibility and generality often infer a performance hit. Spatial blocking isn't something that should be easy to implement after you already have a functioning solver with some level of generality. You can't plan for changes like that, without giving up some level of performance in the first place. Which then defeats the purpose of performance measures like these. Or in other terms: you can't have your cake and eat it.

Example from the LBM solvers I am involved in: For the ability to handle sparsely populated regions, they use an "indirect" memory access pattern. That itself comes at a performance cost, and also makes commonly used performance measures for Cartesian grids difficult to impossible to implement. To counteract that, we order our nodes along a space-filling curve, which somewhat mimics the effect that spatial blocking can have on block-structured grids. That's just the price you pay for generality.
It's just not a feasible strategy to write a solver that can accommodate all changes you may or may not make in the future. With changes and ideas accumulating over time, it just becomes inevitable to rewrite large portions of the code at some point.
Another example: our tools are all written with the assumption that only nearest neighbor information is available. Now there are LBM models that require more than that. If we ever wanted to use them, it would require fundamental changes to pretty much every piece of code we ever wrote. That's just the reality of it, you can't plan for everything. At least not with limited time and resources.

[arjun]

We can't get everything right, we can just plan for it.

I had few major things in my way too.

1. Solver was designed for multi-region with grid grid interface but in start i thought yaa it would work out. So when finally added the GGI, I had to recode each and every physics model with new framework with GGI. Its like writing most of the code again.

2. Second major change came when I moved from QT to Java GUI. I rewrote NOT only GUI but the whole solver to go with it (2.5 years of development). Previous code did not work with Client Server arche.


So yes, if it is needed we need to have will to work with it. Sometimes the changes require lots of efforts.

Originally Posted by sbaffini

I think there are multiple answers to this question. For example, you focus a lot on certain stuff that I typically don't care at all. So what for me might be 10 lines of code, for you might well be 100.

Besides this, and what Arjun already said, if you can't design in advance, the idea is that you recognize patterns while you implement stuff.

In the end, I think, it is not different from general software development, except that there are few first citizen concepts that you may want to honor (regarding the hierarchical relation between grids, variables, solvers, etc.), yet it is not necessarily certain what the best choice is. For example, my code has changed 10 times its current LOC, yet every version was kind of what, at the time, I felt most easy to expand. In practice, it is until it isn't anymore and you change pattern to recover that

[Guvennergiz]

Reserved. I know that I will need these informations some day.

[sbaffini]

To give a more concrete answer, I usually try to have less code as possible, whatever it is that I'm working on. So, as soon as I have two use cases for something, I refactor it (unless I already did it right in the first place).

Still, I try to avoid the "library syndrome", because my code is not a library for others, so I don't have to make certain choices if they don't fit. Thus, for example, I don't have a scalar equation solver that I reuse for anything. I have a dedicated solver for multiple scalars and one for RANS models, besides the main, coupled solver. Still, for example, IO or gradients are designed to be applied to anything, scalar convection schemes can be used for any transport equation besides the main ones, etc.