Login or Register to Ask a Question and Join Our Community


UNIX for Dummies Questions & Answers

Fitting a range of data

 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers Fitting a range of data
# 1  
Old 07-01-2011
Fitting a range of data
Hi,

I am able to write an awk program that fits using the chi squared minimization method for each number is a data column... but I am wondering if it is possible to do that for a range of numbers at the same time.

If I have a column for the observed data... and then say 10 columns for different predicted model outputs (with different parameters). How can I get awk (or anything else) to choose the column that bests match the observation... (as a whole not for each observation entry but as a whole column)

Any idea?
# 2  
Old 07-01-2011
Post sample input and desired output please.
# 3  
Old 07-01-2011
Shown below...
Last edited by cosmologist; 07-03-2011 at 04:00 PM..
# 4  
Old 07-01-2011
I can see that wavelengths in those two outputs don't match. How should this be handled? Should 4855.39 0.85053 be compared with 4850. 0.933214309725409 0.931954786552259 ...? Should 4860.78 0.853197 be compared with 4850. 0.933214309725409 0.931954786552259 ... or with 4870. 0.82560370032963 0.838598193838834 ...?
# 5  
Old 07-01-2011

Quote:
Originally Posted by bartus11
I can see that wavelengths in those two outputs don't match. How should this be handled? Should 4855.39 0.85053 be compared with 4850. 0.933214309725409 0.931954786552259 ...? Should 4860.78 0.853197 be compared with 4850. 0.933214309725409 0.931954786552259 ... or with 4870. 0.82560370032963 0.838598193838834 ...?

Here are the data with equivalent wavelength for both data and model:

The real data:

Code:
5210.  1
5230.  0.996928
5250.  0.963786
5270.  0.959753
5290.  0.977096
5310.  0.958304
5330.  0.945908
5350.  0.952069
5370.  0.942431
5390.  0.919602
5410.  0.915616
5430.  0.925133
5450.  0.917388
5470.  0.91802
5490.  0.919581
5510.  0.91955
5530.  0.921766
5550.  0.915465
5570.  0.896887
5590.  0.890902
5610.  0.896471
5630.  0.89066
5650.  0.880321
5670.  0.873964
5690.  0.872718
5710.  0.877493
5730.  0.87951
5750.  0.872176
5770.  0.866639
5790.  0.866563
5810.  0.874124
5830.  0.869571
5850.  0.854689
5870.  0.845366
5890.  0.827891
5910.  0.839484
5930.  0.798526
5950.  0.823528
5970.  0.82086
5990.  0.803471
6010.  0.77588
6030.  0.812227
6050.  0.807533

and the model:

Code:
5210. 0 0 0 0 0 0 0 0 0.54413057299014 0.539549586593732 0.537208495028375
5230. 0 0 0 0 0 0 0 0 0.536489406301027 0.531781965027436 0.52952809871459
5250. 0 0 0 0 0 0 0 0 0.527403437712125 0.523142184626677 0.52121961380478
5270. 0 0 0 0 0 0 0 0 0.519496826382556 0.514964575376713 0.512901687334473
5290. 0 0 0 0 0 0 0 0 0.513441650244217 0.508846859539853 0.506672084130968
5310. 0 0 0 0 0 0 0 0 0.506122692112091 0.501499264270718 0.499255799228499
5330. 0 0 0 0 0 0 0 0 0.499595527427119 0.49506305753235 0.492864824967908
5350. 0 0 0 0 0 0 0 0 0.493452535787801 0.488795562662424 0.486527195617676
5370. 0 0 0 0 0 0 0 0 0.487376397072558 0.48279379791647 0.480695634561194
5390. 0 0 0 0 0 0 0 0 0.480841985820101 0.476171243860487 0.474095429389523
5410. 0 0 0 0 0 0 0 0 0.470891495894377 0.467851247422539 0.46669082766175
5430. 0 0 0 0 0 0 0 0 0.468007254260263 0.463540298280889 0.461492045577485
5450. 0 0 0 0 0 0 0 0 0.461694361813326 0.457148683542613 0.455060071461975
5470. 0 0 0 0 0 0 0 0 0.455664264718631 0.451182851041969 0.449102051070411
5490. 0 0 0 0 0 0 0 0 0.45000061427542 0.445441629591892 0.443400823813371
5510. 0 0 0 0 0 0 0 0 0.4442656121283 0.440001295489875 0.438130463172893
5530. 0 0 0 0 0 0 0 0 0.438473475167913 0.433918157349868 0.431916963841614
5550. 0 0 0 0 0 0 0 0 0.432993822663351 0.428623689058661 0.426709358743592
5570. 0 0 0 0 0 0 0 0 0.426942771428621 0.422639241362702 0.420766859373816
5590. 0 0 0 0 0 0 0 0 0.420282665040902 0.416338796070279 0.414729388417436
5610. 0 0 0 0 0 0 0 0 0.415182734200264 0.410894245929534 0.409020340302014
5630. 0 0 0 0 0 0 0 0 0.409152941988428 0.404919618256184 0.403082983358652
5650. 0 0 0 0 0 0 0 0 0.403138275571129 0.398807524433964 0.396883121702955
5670. 0 0 0 0 0 0 0 0 0.396514250862827 0.391883483179336 0.389807137225028
5690. 0 0 0 0 0 0 0 0 0.386213370823822 0.382670050677725 0.381313996602238
5710. 0 0 0 0 0 0 0 0 0.384400041708521 0.380324152451842 0.378563076995352
5730. 0 0 0 0 0 0 0 0 0.379691744810425 0.375384635199743 0.373551576352317
5750. 0 0 0 0 0 0 0 0 0.373882893562448 0.369457850613702 0.367569036199413
5770. 0 0 0 0 0 0 0 0 0.370379372487553 0.366319463746369 0.364591027947468
5790. 0 0 0 0 0 0 0 0 0.366626185343177 0.362502375047792 0.360779220165237
5810. 0 0 0 0 0 0 0 0 0.362895800287634 0.359301222613425 0.357789282849867
5830. 0 0 0 0 0 0 0 0 0.359531612619953 0.355574797930746 0.353989492826432
5850. 0 0 0 0 0 0 0 0 0.356773949611905 0.352656557691075 0.350939005331639
5870. 0 0 0 0 0 0 0 0 0.345183990465925 0.339464441516207 0.336033317117964
5890. 0 0 0 0 0 0 0 0 0.343428215627011 0.338879311742497 0.336774135595918
5910. 0 0 0 0 0 0 0 0 0.339718316876763 0.335513635880201 0.333775091533847
5930. 0 0 0 0 0 0 0 0 0.335849376332197 0.33166404841566 0.329915295813347
5950. 0 0 0 0 0 0 0 0 0.331300844877816 0.327162863261225 0.325424443167925
5970. 0 0 0 0 0 0 0 0 0.329140194823134 0.325305102117672 0.323730102252468
5990. 0 0 0 0 0 0 0 0 0.325530819773669 0.321394735253074 0.319658462537416
6010. 0 0 0 0 0 0 0 0 0.322416653813343 0.31854008036127 0.316977484475622
6030. 0 0 0 0 0 0 0 0 0.318554845152722 0.314678028131211 0.313094696655474
6050. 0 0 0 0 0 0 0 0 0.315442734687541 0.311519263466141 0.309929078765764

# 6  
Old 07-02-2011
Okay.. so now you should tell us how the chi squared minimization method should work on that data (I'm not a statistician, so I have no idea). You can use some pseudo code if you like.
# 7  
Old 07-02-2011
I don't know how to get columns from different files into one file...
But not sure how to do it for 3 or more files.. Smilie
Last edited by cosmologist; 07-03-2011 at 04:03 PM..
 
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Help with sum range of data set together

Input File: 2000 3 1998 2 1997 2 1994 1 1991 1 1989 1 1987 2 1986 2 1985 1 1984 1 . . 10 277256 9 278274 8 282507 7 284837 6 287066 5 292967 (4 Replies)
Discussion started by: perl_beginner
4 Replies

2. Shell Programming and Scripting

Fill data if number range is given

Hi I want to get all numbers if number range is given as input. Eg: INPUT FILE 100-105 107 108-112 OUTPUT REQUIRED: 100 101 102 103 104 105 107 108 109 110 111 112 How can I do it using shell? :confused: Thanks in advance. (11 Replies)
Discussion started by: dashing201
11 Replies

3. Shell Programming and Scripting

Filtering data base on range

Hi All, I have the file in following format. I need to change column 16 and column 17 for those lines which has atleast one numerical digit in column 2 , with the exception on SUPP1 Input file : S00005615|1044|0|0.00|0|0.00| |112|-30|28.1|0|0| |20130331|220.2|2|3|... (2 Replies)
Discussion started by: nua7
2 Replies

4. Shell Programming and Scripting

Display data from a range of dates

I have a data in a file called SCHED which has 5 columns: sched no, date, time, place and remarks. The image is shown below. http://dl.dropbox.com/u/54949888/Screenshot%20from%202013-01-02%2002%3A42%3A25.png Now, I want to display only the schedules which fall under a certain date range which... (2 Replies)
Discussion started by: angilulu
2 Replies

5. UNIX for Dummies Questions & Answers

Output all files fitting the pattern

Hi there I need to create 1 line pipe that outputs list of files whose second character is digit in reverse order (file is a directory and called /tmp)the way I did ls /tmp | grep '^.]*' | sort -r dunno it does not seem working. I tried to do it with another directory that I am sure contain a... (8 Replies)
Discussion started by: FUTURE_EINSTEIN
8 Replies

6. Shell Programming and Scripting

Extracting specific lines of data from a file and related lines of data based on a grep value range?

Hi, I have one file, say file 1, that has data like below where 19900107 is the date, 19900107 12 144 129 0.7380047 19900108 12 168 129 0.3149017 19900109 12 192 129 3.2766666E-02 ... (3 Replies)
Discussion started by: Wynner
3 Replies

7. UNIX for Dummies Questions & Answers

Gnuplot - Histogram Fitting

Hi I am trying to fit my histogram data with a gaussian model and am encountering two problems: 1. I can't seem to fit the histogram data with a model fit y(x) 'bin.txt' using 2:xtic(1) via a,b,c (error: need 2 to 7 using specs) 2. Even when I manually guess the correct parameters for my fit and... (1 Reply)
Discussion started by: goffinj
1 Replies

8. UNIX for Dummies Questions & Answers

Fitting data in gnuplot with xyerrorbars

I was looking at the examples that show how to fit data using gnuplot (ex: gnuplot / misc (2E)) but I can't find a place that shows what to do if I have ranges for the x and y error bars. I tried the common sense: gnuplot> fit f(x) "data.txt" using 3:6:4:5:7:8 via a,b ... (8 Replies)
Discussion started by: cosmologist
8 Replies

9. Shell Programming and Scripting

Range of data from a log file .

I am having a log file in which i need a range of data from specific date range. $cat my.log Jan 07 15:39:03 N/A _M_LocalDirectory listFiles(): listing files from dir Jan 07 15:39:03 N/A _w_fm_log:_f_Push() _w_fm_log_export_ftppush(): Files f Jan 07 05:58:35 N/A _w_fm_log_autoexport ... (6 Replies)
Discussion started by: posix
6 Replies

10. Shell Programming and Scripting

Parse a range of data

Hello, I have a file which has a range of date like: 00:00 test 00:01 test2 00:02 test3 00:03 test4 00:04 test5 00:05 test6 Using input (stdin) i would like to parse the data 00:01 to 00:04. The output file should be like this: 00:01 test2 00:02 test3 00:03 test4 00:04 test5 ... (5 Replies)
Discussion started by: BufferExploder
5 Replies
Login or Register to Ask a Question