CFD Online Logo CFD Online URL
www.cfd-online.com
[Sponsors]
Home > Forums > General Forums > Main CFD Forum

Least approximation of derivative

Register Blogs Community New Posts Updated Threads Search

Like Tree3Likes
  • 1 Post By sbaffini
  • 1 Post By Eifoehn4
  • 1 Post By Eifoehn4

Reply
 
LinkBack Thread Tools Search this Thread Display Modes
Old   May 12, 2020, 01:05
Default Least approximation of derivative
  #1
Senior Member
 
Join Date: Oct 2017
Location: United States
Posts: 233
Blog Entries: 1
Rep Power: 10
TurbJet is on a distinguished road
Greetings,

Recently, I've read some papers in which they are using least-square to approximate the 2bd-order derivatives of a given 1-D data. I couldn't figure out how to do this, and I couldn't find any helpful references online.

Can anyone give me some hint, or recommend some references?

Appreciate it!


Edit: To be more specific, my purpose is to evaluate the following 2nd-order derivative

\frac{d^2F(\kappa)}{d(\log\kappa)^2}

where \kappa is just the integer wavenumber, and F(\kappa) is a given function in spectral space.

Last edited by TurbJet; May 12, 2020 at 23:04.
TurbJet is offline   Reply With Quote

Old   May 12, 2020, 01:52
Default
  #2
Senior Member
 
Troy Snyder
Join Date: Jul 2009
Location: Akron, OH
Posts: 220
Rep Power: 19
tas38 is on a distinguished road
Quote:
Originally Posted by TurbJet View Post
Greetings,

Recently, I've read some papers in which they are using least-square to approximate the derivatives. I couldn't figure out how to do this, and I couldn't find any reference online.

Can anyone give me some hint, or recommend some references?

Appreciate it!
Are you referring to a cell-centered gradient approximation using a least-squares-based approach?
tas38 is offline   Reply With Quote

Old   May 12, 2020, 05:32
Default
  #3
Senior Member
 
sbaffini's Avatar
 
Paolo Lampitella
Join Date: Mar 2009
Location: Italy
Posts: 2,190
Blog Entries: 29
Rep Power: 39
sbaffini will become famous soon enoughsbaffini will become famous soon enough
Send a message via Skype™ to sbaffini
The general idea is that you write a 1st order taylor series approximation of a variable phi in the neighborhood n of the point c where you need the gradient (it is used for gradients in general) and assume that:

phi_n = phi_c + dphi/dx_i * (x_n-x_c)_i

Write this for all the available neighbors of a given point c and you get a system of equations that, in most cases, needs to be solved by a least squares method because you have more neighbors than gradient components.

However, the exact least squares solution of the equations is available and you don't typically solve it by a numerical procedure.

There is abundant literature on this. I suggest the book by Blazek as textbook reference because it is the one that covers it best (he and Haselbacher actually developed one of the available closed solutions to the problem).

EDIT: a different version of the exact solution (basically, you have the one based on solving the normal equations and the one based on QR factorization) can also be found in several of the early works of Prof. Z. J. Wang but, unfortunately, each one of them has a slight little error, and you need to look at all of them together to catch it.

Let me also highlight that the work of Blazek and Haselbacher is more general and has, indeed, been extended to higher than first order approximations (used for k-exact rexonstructions) as well as to filters for unstructured grids.

Doru Caraeni also put forward some developments/derivations for the easy application for 3rd order codes

Finally, Nishikawa and coworkers have also made several studies recently
LuckyTran likes this.

Last edited by sbaffini; May 12, 2020 at 09:23.
sbaffini is offline   Reply With Quote

Old   May 12, 2020, 14:20
Default
  #4
Senior Member
 
Eifoehn4's Avatar
 
-
Join Date: Jul 2012
Location: Germany
Posts: 184
Rep Power: 14
Eifoehn4 is on a distinguished road
Or you simply use this slide:

edit: By the way it is called Least Squares!

sbaffini likes this.
Eifoehn4 is offline   Reply With Quote

Old   May 12, 2020, 22:37
Default
  #5
Senior Member
 
Join Date: Oct 2017
Location: United States
Posts: 233
Blog Entries: 1
Rep Power: 10
TurbJet is on a distinguished road
Quote:
Originally Posted by tas38 View Post
Are you referring to a cell-centered gradient approximation using a least-squares-based approach?
Maybe, but I am not sure. The paper I read is using this least-squares approximation to approximate the 2nd-order derivative of the energy flux of HIT in spectral space. The reason they're using this is because the underlying grid is non-uniform.
TurbJet is offline   Reply With Quote

Old   May 12, 2020, 22:42
Default
  #6
Senior Member
 
Join Date: Oct 2017
Location: United States
Posts: 233
Blog Entries: 1
Rep Power: 10
TurbJet is on a distinguished road
Quote:
Originally Posted by Eifoehn4 View Post
Or you simply use this slide:

edit: By the way it is called Least Squares!

Is this applicable to 1D case? I am just looking for an approach to differentiate a function on a non-uniform grid. The paper I referred to use this least-squares approximation to find out the 2nd-derivative of a given dataset.
TurbJet is offline   Reply With Quote

Old   May 12, 2020, 22:49
Default
  #7
Senior Member
 
Join Date: Oct 2017
Location: United States
Posts: 233
Blog Entries: 1
Rep Power: 10
TurbJet is on a distinguished road
Quote:
Originally Posted by sbaffini View Post
The general idea is that you write a 1st order taylor series approximation of a variable phi in the neighborhood n of the point c where you need the gradient (it is used for gradients in general) and assume that:

phi_n = phi_c + dphi/dx_i * (x_n-x_c)_i

Write this for all the available neighbors of a given point c and you get a system of equations that, in most cases, needs to be solved by a least squares method because you have more neighbors than gradient components.

However, the exact least squares solution of the equations is available and you don't typically solve it by a numerical procedure.

There is abundant literature on this. I suggest the book by Blazek as textbook reference because it is the one that covers it best (he and Haselbacher actually developed one of the available closed solutions to the problem).

EDIT: a different version of the exact solution (basically, you have the one based on solving the normal equations and the one based on QR factorization) can also be found in several of the early works of Prof. Z. J. Wang but, unfortunately, each one of them has a slight little error, and you need to look at all of them together to catch it.

Let me also highlight that the work of Blazek and Haselbacher is more general and has, indeed, been extended to higher than first order approximations (used for k-exact rexonstructions) as well as to filters for unstructured grids.

Doru Caraeni also put forward some developments/derivations for the easy application for 3rd order codes

Finally, Nishikawa and coworkers have also made several studies recently
I did some digging after I posted this thread, most of the contents are applied to CFD (at least from what I can tell). However, I am looking for a general approach that can be simply applied to evaluate differentiations of a given 1D data with non-uniform grid; and that's it, I don't need fancy algorithms that can solve complex fluid problems. So I think my need is just a simple numerical analysis of 1-D derivative. Do you think the approach you mentioned can be used for this purpose?
TurbJet is offline   Reply With Quote

Old   May 13, 2020, 03:19
Default
  #8
Senior Member
 
Eifoehn4's Avatar
 
-
Join Date: Jul 2012
Location: Germany
Posts: 184
Rep Power: 14
Eifoehn4 is on a distinguished road
Quote:
Originally Posted by TurbJet View Post
Is this applicable to 1D case? I am just looking for an approach to differentiate a function on a non-uniform grid. The paper I referred to use this least-squares approximation to find out the 2nd-derivative of a given dataset.
Sure you can! In 1D with equidistant distributed data this will result in a standard 2nd order FD stencil. However you can use it also for non-equidistant data. For a second order derivative simply apply the operator twice.

Quote:
Originally Posted by TurbJet View Post
Greetings,

Edit: To be more specific, my purpose is to evaluate the following 2nd-order derivative

\frac{d^2F(\kappa)}{d(\log\kappa)^2}

where \kappa is just the integer wavenumber, and F(\kappa) is a given function in spectral space.
I think you make the problem harder than it is. Note that

\frac{dF}{d(\log\kappa)} = \frac{1}{\frac{d(\log\kappa)}{dF}}  = \frac{1}{\frac{d(\log\kappa)}{dF}\frac{d \kappa}{d\kappa}}= \frac{1}{\frac{d(\log\kappa)}{d \kappa }\frac{d \kappa}{dF}} = \frac{1}{\frac{1}{\kappa }\frac{d \kappa}{dF}}  = \kappa \frac{dF}{d \kappa}

and therefore

\frac{d^2 F}{d(\log\kappa)^2} = \kappa \frac{dF}{d \kappa} + \kappa^2 \frac{d^2 F}{d \kappa^2}.

This is more easy to discretisize.

Regards
TurbJet likes this.

Last edited by Eifoehn4; May 13, 2020 at 05:43.
Eifoehn4 is offline   Reply With Quote

Old   May 13, 2020, 07:10
Default
  #9
Senior Member
 
sbaffini's Avatar
 
Paolo Lampitella
Join Date: Mar 2009
Location: Italy
Posts: 2,190
Blog Entries: 29
Rep Power: 39
sbaffini will become famous soon enoughsbaffini will become famous soon enough
Send a message via Skype™ to sbaffini
Quote:
Originally Posted by Eifoehn4 View Post
Sure you can! In 1D with equidistant distributed data this will result in a standard 2nd order FD stencil. However you can use it also for non-equidistant data. For a second order derivative simply apply the operator twice.



I think you make the problem harder than it is. Note that

\frac{dF}{d(\log\kappa)} = \frac{1}{\frac{d(\log\kappa)}{dF}}  = \frac{1}{\frac{d(\log\kappa)}{dF}\frac{d \kappa}{d\kappa}}= \frac{1}{\frac{d(\log\kappa)}{d \kappa }\frac{d \kappa}{dF}} = \frac{1}{\frac{1}{\kappa }\frac{d \kappa}{dF}}  = \kappa \frac{dF}{d \kappa}

and therefore

\frac{d^2 F}{d(\log\kappa)^2} = \kappa \frac{dF}{d \kappa} + \kappa^2 \frac{d^2 F}{d \kappa^2}.

This is more easy to discretisize.

Regards
Nice point, but probably they have very different accuracy when discretized for typical energy spectra, say,
for a function of the form F(k) = \frac{k^4}{\left(k^2+1\right)^{17/6}}, representative of the energy at large large scales (i.e., no dissipative range).

When you look at this in log k the peak is much more spread (besides the fact that the delta log(k) are much smaller than the delta k).

However, without knowing the original use case for least squares here, Ferziger & Peric has straighforward formulas for both 1st and 2nd derivatives in 1D for non uniform grids, without need to resort to anything fancy.
sbaffini is offline   Reply With Quote

Old   May 13, 2020, 08:02
Default
  #10
Senior Member
 
Eifoehn4's Avatar
 
-
Join Date: Jul 2012
Location: Germany
Posts: 184
Rep Power: 14
Eifoehn4 is on a distinguished road
Quote:
Originally Posted by sbaffini View Post
Nice point, but probably they have very different accuracy when discretized for typical energy spectra, say,
for a function of the form F(k) = \frac{k^4}{\left(k^2+1\right)^{17/6}}, representative of the energy at large large scales (i.e., no dissipative range).

When you look at this in log k the peak is much more spread (besides the fact that the delta log(k) are much smaller than the delta k).

However, without knowing the original use case for least squares here, Ferziger & Peric has straighforward formulas for both 1st and 2nd derivatives in 1D for non uniform grids, without need to resort to anything fancy.
That's true. I think he has to test, which representation works better.
Eifoehn4 is offline   Reply With Quote

Old   May 13, 2020, 16:29
Default
  #11
Senior Member
 
Join Date: Oct 2017
Location: United States
Posts: 233
Blog Entries: 1
Rep Power: 10
TurbJet is on a distinguished road
Quote:
Originally Posted by Eifoehn4 View Post
That's true. I think he has to test, which representation works better.
Thanks for your previous reply. So I'll probably try both approaches.
TurbJet is offline   Reply With Quote

Old   May 13, 2020, 16:37
Default
  #12
Senior Member
 
Join Date: Oct 2017
Location: United States
Posts: 233
Blog Entries: 1
Rep Power: 10
TurbJet is on a distinguished road
Quote:
Originally Posted by sbaffini View Post
Nice point, but probably they have very different accuracy when discretized for typical energy spectra, say,
for a function of the form F(k) = \frac{k^4}{\left(k^2+1\right)^{17/6}}, representative of the energy at large large scales (i.e., no dissipative range).

When you look at this in log k the peak is much more spread (besides the fact that the delta log(k) are much smaller than the delta k).

However, without knowing the original use case for least squares here, Ferziger & Peric has straighforward formulas for both 1st and 2nd derivatives in 1D for non uniform grids, without need to resort to anything fancy.
Could you explain more on this specific case that you gave? Why this spreading peak would affect the accuracy? Is it because it can't be accurately represented on a uniform grid?
TurbJet is offline   Reply With Quote

Old   May 13, 2020, 17:18
Default
  #13
Senior Member
 
sbaffini's Avatar
 
Paolo Lampitella
Join Date: Mar 2009
Location: Italy
Posts: 2,190
Blog Entries: 29
Rep Power: 39
sbaffini will become famous soon enoughsbaffini will become famous soon enough
Send a message via Skype™ to sbaffini
Well, the function I used is just an example of how the energy spectra behave at low wave numbers (I used k^4, but k^2 would have been another possibility), where they tipically have a peak.

Now, what approach is more suited obviously depends from what the final goal is, but I would point out the following:

1) The formula with log k is just a second derivative, while the formula with k has two derivatives (first and second) multiplied by k and k^2 respectively. More work and typically more error.

2) The steps in k are much larger than those in log k, so this is necessarily linked to higher error

3) Now that I think more about it, this might simply be a restatement of 2 but, if you plot the function above as function of k and of log k, the latter is much more smoother around the peak, which means that any given formula would give less error.

When I wrote the message this morning, I actually tried both formulas and they indeed differ at low wavenumbers for the function I mentioned. I deleted the script I used, but I suggest you to make a similar test to better understand them both.
sbaffini is offline   Reply With Quote

Old   May 13, 2020, 19:08
Default
  #14
Senior Member
 
Join Date: Oct 2017
Location: United States
Posts: 233
Blog Entries: 1
Rep Power: 10
TurbJet is on a distinguished road
Quote:
Originally Posted by sbaffini View Post
Well, the function I used is just an example of how the energy spectra behave at low wave numbers (I used k^4, but k^2 would have been another possibility), where they tipically have a peak.

Now, what approach is more suited obviously depends from what the final goal is, but I would point out the following:

1) The formula with log k is just a second derivative, while the formula with k has two derivatives (first and second) multiplied by k and k^2 respectively. More work and typically more error.

2) The steps in k are much larger than those in log k, so this is necessarily linked to higher error

3) Now that I think more about it, this might simply be a restatement of 2 but, if you plot the function above as function of k and of log k, the latter is much more smoother around the peak, which means that any given formula would give less error.

When I wrote the message this morning, I actually tried both formulas and they indeed differ at low wavenumbers for the function I mentioned. I deleted the script I used, but I suggest you to make a similar test to better understand them both.
I see. Great explanation and thanks a lot!
TurbJet is offline   Reply With Quote

Reply

Tags
derivative, least-squares, numerical analysis, numerical method


Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are Off
Pingbacks are On
Refbacks are On


Similar Threads
Thread Thread Starter Forum Replies Last Post
Error when calculating the second derivative using the derivative function d2dy2 mona.li Tecplot 0 July 8, 2018 15:11
Time derivative of pressure in low mach number approximation hnemati Main CFD Forum 1 June 16, 2016 08:05
Tangential derivative on curved boundary Simbelmynė Main CFD Forum 40 October 14, 2014 07:57
discontinuous second derivative new_at_this Main CFD Forum 9 April 28, 2012 17:55
Different one sided derivative approximation? zonexo Main CFD Forum 2 January 11, 2006 05:40


All times are GMT -4. The time now is 04:40.