Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Grep final set of parameters from fit.log gnuplot file

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Grep final set of parameters from fit.log gnuplot file
# 1  
Old 03-20-2013
Grep final set of parameters from fit.log gnuplot file
I would like to grep the final set of fit parameters from a gnuplot log file to form columns that look like this.
Code:
a_1001 b_1001 x_1001
a_1002 b_1002 x_1002
a_1003 b_1003 x_1003
   .          .         .   
   .          .         .
   .          .         .
a_1250  b_1250 c_1250

At present, the 'fit.log' file looks like this (also see attachment).

Code:

*****************************************************************




 Iteration 0
 WSSR        : 13.5219           delta(WSSR)/WSSR   : 0
 delta(WSSR) : 0                 limit for stopping : 1e-05
 lambda	  : 1.1547

initial set of free parameter values

a_1001          = 1
b_1001          = 1
x_1001          = 1

After 8 iterations the fit converged.
final sum of squares of residuals : 0.00106152
rel. change during last iteration : -1.34952e-06

degrees of freedom    (FIT_NDF)                        : 15
rms of residuals      (FIT_STDFIT) = sqrt(WSSR/ndf)    : 0.00841238
variance of residuals (reduced chisquare) = WSSR/ndf   : 7.07681e-05

Final set of parameters            Asymptotic Standard Error
=======================            ==========================

a_1001          = 0.497897         +/- 0.001983     (0.3982%)
b_1001          = 0.00459204       +/- 0.005632     (122.6%)
x_1001          = 1.57077          +/- 0.6132       (39.04%)


correlation matrix of the fit parameters:

               a_1001 b_1001 x_1001 
a_1001          1.000 
b_1001         -0.003  1.000 
x_1001         -0.000  0.002  1.000 


****************************************************************

 Iteration 0
 WSSR        : 13.5246           delta(WSSR)/WSSR   : 0
 delta(WSSR) : 0                 limit for stopping : 1e-05
 lambda	  : 1.1547

initial set of free parameter values

a_1002          = 1
b_1002          = 1
x_1002          = 1

After 8 iterations the fit converged.
final sum of squares of residuals : 0.0019177
rel. change during last iteration : -1.59332e-09

degrees of freedom    (FIT_NDF)                        : 15
rms of residuals      (FIT_STDFIT) = sqrt(WSSR/ndf)    : 0.0113069
variance of residuals (reduced chisquare) = WSSR/ndf   : 0.000127847

Final set of parameters            Asymptotic Standard Error
=======================            ==========================

a_1002          = 0.49692          +/- 0.002665     (0.5363%)
b_1002          = 0.00889775       +/- 0.007585     (85.24%)
x_1002          = 1.57078          +/- 0.4262       (27.13%)


correlation matrix of the fit parameters:

               a_1002 b_1002 x_1002 
a_1002          1.000 
b_1002         -0.006  1.000 
x_1002         -0.000  0.002  1.000

I only need the final set of parameters as a table and I am wondering if anyone could help using awk or grep to get these parameters out.
fit.txt (2.8 KB) 
# 2  
Old 03-20-2013
I see nothing resembling that output in your input so am confused to which lines you expect grep to match.
This User Gave Thanks to Corona688 For This Post:
kayak
# 3  
Old 03-20-2013
For your test input, is this the first line of output you expect?
Code:
0.497897 0.00459204 1.57077

This User Gave Thanks to hanson44 For This Post:
kayak
# 4  
Old 03-20-2013
Yes, my output should look like this:
Code:
0.497897 0.00459204 1.57077
0.49692 0.00889775 1.57078

And so on.. Thanks
# 5  
Old 03-20-2013
Here's one possible way:

Code:
$cat temp.sh
sed -n "/^Final/,/^correlation/ p" infile > temp1.x # get segments
grep "^[a-z]_[0-9][0-9][0-9][0-9] " temp1.x > temp2.x # get lines
sed "s/.* = \([^ ][^ ]*\) .*/\1/" temp2.x > temp3.x # get numbers
sed "N; N; s/\n/ /g" temp3.x # join lines

$ ./temp.sh
0.497897 0.00459204 1.57077
0.49692 0.00889775 1.57078

If your sed does not support \n, there is bound to be some other simple way to join lines on last step.
This User Gave Thanks to hanson44 For This Post:
kayak
# 6  
Old 03-21-2013
With the sample provided, the following simple awk script seems to work:
Code:
awk -F' *=* +' '/[+]\/-/{printf "%s%s",$2,(++i%3)?" ":"\n"}' fit.txt

If you are using a Solaris/SunOS system, use /usr/xpg4/bin/awk or nawk instead of awk.
This User Gave Thanks to Don Cragun For This Post:
kayak
# 7  
Old 04-03-2013
Quote:
Originally Posted by kayak
Quote:
Originally Posted by kayak
Thanks Don for your reply to my post. But I need a little help. Your proposition below works well for three parameters.

Code:
awk -F' *=* +' '/[+]\/-/{printf "%s%s",$2,(++i%3)?" ":"\n"}' fit.txt

In my latest fit.txt file, I have one additional parameter as follows:

Code:
FIT:    data read from '1000.dat' u 8:(($4/66094)/0.174533)/(sin($8))
        format = x:z
        #datapoints = 18
        residuals are weighted equally (unit weight)

function used for fitting: E_1000(x)
fitted parameters initialized with current variable values



 Iteration 0
 WSSR        : 31.8466           delta(WSSR)/WSSR   : 0
 delta(WSSR) : 0                 limit for stopping : 1e-05
 lambda	  : 1

initial set of free parameter values

a_1000          = 1
b_1000          = 1
c_1000          = 1
d_1000          = 1

After 5 iterations the fit converged.
final sum of squares of residuals : 0.000317809
rel. change during last iteration : 0

degrees of freedom    (FIT_NDF)                        : 14
rms of residuals      (FIT_STDFIT) = sqrt(WSSR/ndf)    : 0.00476452
variance of residuals (reduced chisquare) = WSSR/ndf   : 2.27006e-05

Final set of parameters            Asymptotic Standard Error
=======================            ==========================

a_1000          = 0.498062         +/- 0.001123     (0.2255%)
b_1000          = -0.00287463      +/- 0.003189     (110.9%)
c_1000          = -0.017153        +/- 0.003189     (18.59%)
d_1000          = -0.0145765       +/- 0.003189     (21.88%)


correlation matrix of the fit parameters:

               a_1000 b_1000 c_1000 d_1000 
a_1000          1.000 
b_1000          0.002  1.000 
c_1000          0.012  0.000  1.000 
d_1000          0.010  0.000  0.000  1.000 


*******************************************************************************
Wed Apr  3 12:42:24 2013


FIT:    data read from '1001.dat' u 8:(($4/65884)/0.174533)/(sin($8))
        format = x:z
        #datapoints = 18
        residuals are weighted equally (unit weight)

function used for fitting: E_1001(x)
fitted parameters initialized with current variable values



 Iteration 0
 WSSR        : 31.8512           delta(WSSR)/WSSR   : 0
 delta(WSSR) : 0                 limit for stopping : 1e-05
 lambda	  : 1

initial set of free parameter values

a_1001          = 1
b_1001          = 1
c_1001          = 1
d_1001          = 1

After 5 iterations the fit converged.
final sum of squares of residuals : 3.51671e-05
rel. change during last iteration : -9.63438e-16

degrees of freedom    (FIT_NDF)                        : 14
rms of residuals      (FIT_STDFIT) = sqrt(WSSR/ndf)    : 0.00158491
variance of residuals (reduced chisquare) = WSSR/ndf   : 2.51193e-06

Final set of parameters            Asymptotic Standard Error
=======================            ==========================

a_1001          = 0.497897         +/- 0.0003736    (0.07503%)
b_1001          = -0.00459203      +/- 0.001061     (23.11%)
c_1001          = -0.0162969       +/- 0.001061     (6.511%)
d_1001          = -0.0139438       +/- 0.001061     (7.61%)


correlation matrix of the fit parameters:

               a_1001 b_1001 c_1001 d_1001 
a_1001          1.000 
b_1001          0.003  1.000 
c_1001          0.012  0.000  1.000 
d_1001          0.010  0.000  0.000  1.000 


*******************************************************************************
Wed Apr  3 12:42:24 2013


FIT:    data read from '1002.dat' u 8:(($4/66048)/0.174533)/(sin($8))
        format = x:z
        #datapoints = 18
        residuals are weighted equally (unit weight)

function used for fitting: E_1002(x)
fitted parameters initialized with current variable values



 Iteration 0
 WSSR        : 31.9886           delta(WSSR)/WSSR   : 0
 delta(WSSR) : 0                 limit for stopping : 1e-05
 lambda	  : 1

initial set of free parameter values

a_1002          = 1
b_1002          = 1
c_1002          = 1
d_1002          = 1

After 5 iterations the fit converged.
final sum of squares of residuals : 0.000195293
rel. change during last iteration : -5.41287e-15

degrees of freedom    (FIT_NDF)                        : 14
rms of residuals      (FIT_STDFIT) = sqrt(WSSR/ndf)    : 0.0037349
variance of residuals (reduced chisquare) = WSSR/ndf   : 1.39495e-05

Final set of parameters            Asymptotic Standard Error
=======================            ==========================

a_1002          = 0.49692          +/- 0.0008803    (0.1772%)
b_1002          = -0.00889774      +/- 0.002505     (28.16%)
c_1002          = -0.0210222       +/- 0.002506     (11.92%)
d_1002          = -0.0182511       +/- 0.002506     (13.73%)


correlation matrix of the fit parameters:

               a_1002 b_1002 c_1002 d_1002 
a_1002          1.000 
b_1002          0.006  1.000 
c_1002          0.015  0.000  1.000 
d_1002          0.013  0.000  0.000  1.000


Is it possible to extend your awk line to capture the last parameter such that the output is like:

Code:
a_1001 b_1001 c_1001 d_1001
a_1002 b_1002 c_1002 d_1002
.
.
.

and so on?

Thanks in advance.
Last edited by kayak; 04-03-2013 at 09:37 AM..
Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Different set key multiplot gnuplot

Hello everybody, I am using Windows 10. I need to plot several graphs simultaneously with Gnuplot. The script is: GNUTERM = "wxt" set terminal wxt set termoption enhanced set encoding iso_8859_1 set multiplot layout 2,3 set xtics out set ytics out set xtics nomirror set ytics... (4 Replies)
Discussion started by: supernono06
4 Replies

2. Shell Programming and Scripting

Getopt eval set parameters not happening when the script is called through an application!

Hi, There is an Informatica tool through which unix scripts can be called. Now the requirement in my project is that a parent script calls a child script but this parent script has to be called through the Informatica tool. In Parent script I'm using TEMP=`getopt -o x:y: -l area:,volume:... (1 Reply)
Discussion started by: Panna
1 Replies

3. Shell Programming and Scripting

How do i check if all parameters are set in bash?

Hi, I have 4 parameters passed to my shell and i validate if all four are entered using the following snippet: if then echo "Not entered" else echo "entered" fi I get the following output as 'Not entered even when i enter the values for all prompts. Please advise. Thanks. (5 Replies)
Discussion started by: Jesshelle David
5 Replies

4. Linux

Gnuplot terminal set to 'unknown'

I installed version 4.6.4 of gnuplot recently on my system running 10.04 ubuntu linux. I am getting error Terminal type set to 'unknown' Also when i try to set term X11 or wxt i get error Terminal type set to 'unknown' ^ unknown or ambiguous terminal... (5 Replies)
Discussion started by: Ujjwal1982
5 Replies

5. Shell Programming and Scripting

Log-log plot in gnuplot and limiting the xrange

I am trying to fit a plot in gnuplot using logscale. I have 50000 data points. At first I fit plot in this way. f(x) = b + m*x fit f(x) "xyMSD-all-mal-cel-iso-bcm-thermo2.dat" using 1:2 via m,b I got slope value. Then I tried to get slope value at different range as below. fit f(x)... (2 Replies)
Discussion started by: vjramana
2 Replies

6. Shell Programming and Scripting

gnuplot set key issue

Hi I have a plotting function like this- plot '../processing_old/time0.txt' using 1:6 title "1.6" with boxes fs solid 0.05 ls 5,\ 'time0.txt' using 1:6 title "1.7" with boxes fs solid 0.05 ls 6,\ '../processing_new/time0.txt' using 1:6 title "2.0" with boxes fs solid... (8 Replies)
Discussion started by: jamie_123
8 Replies

7. Solaris

Which file is read by kernel to set its default system kernel parameters values?

Hi gurus Could anybody tell me which file is read by kernel to set its default system kernal parameters values in solaris. Here I am not taking about /etc/system file which is used to load kernal modules or to change any default system kernal parameter value Is it /dev/kmem file or something... (1 Reply)
Discussion started by: girish.batra
1 Replies

8. Shell Programming and Scripting

Call single function multiple times diff set of parameters

Okay, not sure if it can be done, I would think it could be done and I'm just having a hard time with it. fun_close_wait(){ ipVar="${1} ${2}" portVar=`cat "${5}" | cut -d' ' -f 1` for ip in $ipVar do for port in $portVar do netstatVar=`netstat -n | grep... (4 Replies)
Discussion started by: cbo0485
4 Replies

9. UNIX for Dummies Questions & Answers

least-square fit in Gnuplot

Does anyone know how to find the best least square fit in Gnuplot? (6 Replies)
Discussion started by: cosmologist
6 Replies
Login or Register to Ask a Question