Visit Our UNIX and Linux User Community


Read n lines from a text files getting n from within the text file


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Read n lines from a text files getting n from within the text file
# 1  
Old 09-12-2013
Read n lines from a text files getting n from within the text file

I dont even have a sample script cause I dont know where to start from. My data lookes like this
Code:
> sat#16 #data: 15 site:UNZA baseline:  205.9151
  0.008   -165.2465     35.8109     40.6685     21.9148    121.1446     26.4629    -18.4976     33.8722
  0.017   -165.2243     48.2201     40.6908     21.9058    120.8975     26.4179    -18.4794     33.8951
  0.025   -165.1857     41.5258     40.7293     21.9056    120.6503     26.3729    -18.4611     33.9179
  0.033   -165.1438     33.9471     40.7713     21.9072    120.4031     26.3278    -18.4427     33.9407
  0.042   -165.1106     40.8982     40.8045     21.9039    120.1559     26.2826    -18.4242     33.9635
  0.050   -165.0913     38.5970     40.8238     21.8932    119.9087     26.2375    -18.4056     33.9862
  0.058   -165.0968     43.7509     40.8183     21.8692    119.6616     26.1922    -18.3868     34.0089
  0.067   -165.0654     40.7461     40.8497     21.8649    119.4144     26.1470    -18.3680     34.0316
  0.075   -165.0817     46.2518     40.8334     21.8350    119.1672     26.1016    -18.3490     34.0542
  0.083   -165.0412     40.4513     40.8739     21.8355    118.9200     26.0563    -18.3299     34.0768
  0.092   -165.0533     44.7589     40.8617     21.8078    118.6728     26.0108    -18.3107     34.0994
  0.100   -165.0425     43.8650     40.8726     21.7924    118.4256     25.9653    -18.2913     34.1219
  0.108   -165.0378     39.0059     40.8773     21.7736    118.1784     25.9198    -18.2719     34.1444
  0.117   -165.0049     44.6828     40.9102     21.7699    117.9312     25.8741    -18.2523     34.1668
> sat#19 #data: 6 site:UNZA baseline:  235.0682
 17.900   -171.3423     62.8450     63.7259     26.5775    325.4868     15.0105     -7.3923     22.8404
 17.908   -171.5057     55.5991     63.5625     26.6247    325.4057     15.2054     -7.4669     22.8738
 17.917   -171.7347     53.3075     63.3335     26.6444    325.3236     15.4003     -7.5407     22.9067
 17.925   -172.0204     57.9098     63.0478     26.6402    325.2407     15.5952     -7.6138     22.9393
 17.933   -172.2141     50.7400     62.8541     26.6748    325.1568     15.7902     -7.6860     22.9714
 17.942   -172.4885     48.0680     62.5797     26.6751    325.0720     15.9851     -7.7575     23.0031
> sat# 1 #data: 8 site:UNZA baseline:  225.5519
  0.008   -148.7547     75.6631     76.7972     65.5288     89.3165     56.3463    -15.3538     30.6348
  0.017   -148.7668     73.5426     76.7851     65.5199     89.7731     56.3486    -15.3715     30.6350
  0.025   -148.7836     80.0087     76.7683     65.5057     90.2296     56.3488    -15.3892     30.6352
  0.033   -148.7821     76.5570     76.7698     65.5058     90.6859     56.3469    -15.4069     30.6354
  0.042   -148.7916     78.7440     76.7603     65.4952     91.1420     56.3430    -15.4246     30.6356
  0.050   -148.8006     74.6552     76.7513     65.4836     91.5978     56.3370    -15.4423     30.6359
  0.058   -148.8115     73.8469     76.7404     65.4692     92.0532     56.3289    -15.4600     30.6362
  0.067   -148.8211     75.2542     76.7308     65.4545     92.5083     56.3188    -15.4777     30.6365

and so on. I need the sript to read the first line with text and use the #data value to determine how many lines to read below this text line, then read those lines of data, then read the next line with text and use the #data value to count how namy lines to read below that and so on.

To be honext I have no idea where to start from. And honestly I am not completely new to shell scripting but this has challenged me to the core. This is not an assignment, its for my rresearch work am a seniour lecturer in the department of physics at the university of zambia in lusaka, zambia. I hope it is doable.

The read data can be directed to a gnuplot ot output to a file.

Thank you in advance
# 2  
Old 09-12-2013
Try this:

Code:
awk -F: '/#data/ { R=NR+$2} NR<=R' infile

# 3  
Old 09-12-2013
Note that there are only 14 lines following the 1st line, but the 1st line contains #data: 15; is a line missing in your sample, or is the XXX value in the #data: XXX segments of the headers unreliable? (Note that since the header lines all start with a ">" and the data lines only contain floaing point values, the count in the header can be ignored.)

The counts in the other two header lines seem to match the following data.

Once you have read the data lines, what do you want to do with them?
# 4  
Old 09-12-2013
Good points Don Cragun, my assumption was that the value in the data line was less than then number of lines and any additional lines need to be discarded.

In my solution, any processing could be done in the NR <=R block:

Code:
awk -F: '/#data/ { R=NR+$2} NR<=R { "Any processing code here"; print }' infile

# 5  
Old 09-13-2013
Quote:
Originally Posted by Chubler_XL
Good points Don Cragun, my assumption was that the value in the data line was less than then number of lines and any additional lines need to be discarded.

In my solution, any processing could be done in the NR <=R block:

Code:
awk -F: '/#data/ { R=NR+$2} NR<=R { "Any processing code here"; print }' infile

Understood.

The O.P. said the output "can be directed to a gnuplot ot (sic) output to a file.", but didn't specify any options for gnuplot and didn't specify filenames nor whether the headers should be included in the files or stripped from the files. There is no gnuplot man page in the Man Pages section of this forum, but some references I found said that it is very picky about having its input in tab separated fields (and the data lines given in the sample input have no tabs). Should all data be sent to one instantiation of gnuplot or should the data under each header be sent to a different instantiation of gnuplot?

I don't think I can do much more until we get a clarification on the requirements.
# 6  
Old 09-13-2013
Thank you vry much for all your advices, I am now working on ta script to use this method and I will report back just now to let you know the result. I want to plot the data once I have read it. and since I want to plot in different colors, for each block so I will direct my output to a gnuplot script, I hope that works.

I will let you know the result!


Thank you

---------- Post updated at 08:30 AM ---------- Previous update was at 07:17 AM ----------

Good afternoon,

Somehow I am not able to get this to do what I want to do.

I am trying to do is to read the value after #data, that should be the number of data lines in that block, so I should be able to read that number of lines under that header and sent that to a file which I will then call in gnuplot and plot that particular data.
The Final output is a plot of column 1 vs column 7 all the blocks on the same plot. But since the column 1 is not in a continuous sequence, I need to plot each block of data separately and overplot on the same plot.

Here is the intedend output

Either the data for each block is sent to a seperate file like this
file 1
Code:
  0.008   -165.2465     35.8109     40.6685     21.9148    121.1446     26.4629    -18.4976     33.8722   0.017   -165.2243     48.2201     40.6908     21.9058    120.8975     26.4179    -18.4794     33.8951   0.025   -165.1857     41.5258     40.7293     21.9056    120.6503     26.3729    -18.4611     33.9179   0.033   -165.1438     33.9471     40.7713     21.9072    120.4031     26.3278    -18.4427     33.9407   0.042   -165.1106     40.8982     40.8045     21.9039    120.1559     26.2826    -18.4242     33.9635   0.050   -165.0913     38.5970     40.8238     21.8932    119.9087     26.2375    -18.4056     33.9862   0.058   -165.0968     43.7509     40.8183     21.8692    119.6616     26.1922    -18.3868     34.0089   0.067   -165.0654     40.7461     40.8497     21.8649    119.4144     26.1470    -18.3680     34.0316   0.075   -165.0817     46.2518     40.8334     21.8350    119.1672     26.1016    -18.3490     34.0542   0.083   -165.0412     40.4513     40.8739     21.8355    118.9200     26.0563    -18.3299     34.0768   0.092   -165.0533     44.7589     40.8617     21.8078    118.6728     26.0108    -18.3107     34.0994   0.100   -165.0425     43.8650     40.8726     21.7924    118.4256     25.9653    -18.2913     34.1219   0.108   -165.0378     39.0059     40.8773     21.7736    118.1784     25.9198    -18.2719     34.1444   0.117   -165.0049     44.6828     40.9102     21.7699    117.9312     25.8741    -18.2523     34.1668

file 2
Code:
 17.900   -171.3423     62.8450     63.7259     26.5775    325.4868     15.0105     -7.3923     22.8404  17.908   -171.5057     55.5991     63.5625     26.6247    325.4057     15.2054     -7.4669     22.8738  17.917   -171.7347     53.3075     63.3335     26.6444    325.3236     15.4003     -7.5407     22.9067  17.925   -172.0204     57.9098     63.0478     26.6402    325.2407     15.5952     -7.6138     22.9393  17.933   -172.2141     50.7400     62.8541     26.6748    325.1568     15.7902     -7.6860     22.9714  17.942   -172.4885     48.0680     62.5797     26.6751    325.0720     15.9851     -7.7575     23.0031

file 3
Code:
 0.008   -148.7547     75.6631      76.7972     65.5288     89.3165     56.3463    -15.3538     30.6348   0.017   -148.7668     73.5426     76.7851     65.5199     89.7731     56.3486    -15.3715     30.6350   0.025   -148.7836     80.0087     76.7683     65.5057     90.2296     56.3488    -15.3892     30.6352   0.033   -148.7821     76.5570     76.7698     65.5058     90.6859     56.3469    -15.4069     30.6354   0.042   -148.7916     78.7440     76.7603     65.4952     91.1420     56.3430    -15.4246     30.6356   0.050   -148.8006     74.6552     76.7513     65.4836     91.5978     56.3370    -15.4423     30.6359   0.058   -148.8115     73.8469     76.7404     65.4692     92.0532     56.3289    -15.4600     30.6362   0.067   -148.8211     75.2542     76.7308     65.4545     92.5083     56.3188    -15.4777     30.6365

so that I can then use gnuplot to plot column 1 vs colomn 7 for each file on the same plot.

If instead of sending the data to a file I can send it directly to gnuplot and produce one plot with lines for each block of data then that would be the perfect result I am looking for.

For example I have put this line into a script
Code:
#! /bin/bash
#
awk -F: '/#data/ { R=NR+$2} NR<=R { print $1, $7}' UNZA2250.txt > data.out

I expected that data.out should contain the data but without the headers for each block and thats not what is happening.

Thank you all the the time and assistance.... its much appreciated.

Regards

---------- Post updated at 08:35 AM ---------- Previous update was at 08:30 AM ----------

Sorry the output data didnt come out correctly in my post above here is the disired outpot

file 1
Code:
  0.008   -165.2465     35.8109     40.6685     21.9148    121.1446     26.4629    -18.4976     33.8722
  0.017   -165.2243     48.2201     40.6908     21.9058    120.8975     26.4179    -18.4794     33.8951
  0.025   -165.1857     41.5258     40.7293     21.9056    120.6503     26.3729    -18.4611     33.9179
  0.033   -165.1438     33.9471     40.7713     21.9072    120.4031     26.3278    -18.4427     33.9407
  0.042   -165.1106     40.8982     40.8045     21.9039    120.1559     26.2826    -18.4242     33.9635
  0.050   -165.0913     38.5970     40.8238     21.8932    119.9087     26.2375    -18.4056     33.9862
  0.058   -165.0968     43.7509     40.8183     21.8692    119.6616     26.1922    -18.3868     34.0089
  0.067   -165.0654     40.7461     40.8497     21.8649    119.4144     26.1470    -18.3680     34.0316
  0.075   -165.0817     46.2518     40.8334     21.8350    119.1672     26.1016    -18.3490     34.0542
  0.083   -165.0412     40.4513     40.8739     21.8355    118.9200     26.0563    -18.3299     34.0768
  0.092   -165.0533     44.7589     40.8617     21.8078    118.6728     26.0108    -18.3107     34.0994
  0.100   -165.0425     43.8650     40.8726     21.7924    118.4256     25.9653    -18.2913     34.1219
  0.108   -165.0378     39.0059     40.8773     21.7736    118.1784     25.9198    -18.2719     34.1444
  0.117   -165.0049     44.6828     40.9102     21.7699    117.9312     25.8741    -18.2523     34.1668

file 2
Code:
 17.900   -171.3423     62.8450     63.7259     26.5775    325.4868     15.0105     -7.3923     22.8404
 17.908   -171.5057     55.5991     63.5625     26.6247    325.4057     15.2054     -7.4669     22.8738
 17.917   -171.7347     53.3075     63.3335     26.6444    325.3236     15.4003     -7.5407     22.9067
 17.925   -172.0204     57.9098     63.0478     26.6402    325.2407     15.5952     -7.6138     22.9393
 17.933   -172.2141     50.7400     62.8541     26.6748    325.1568     15.7902     -7.6860     22.9714
 17.942   -172.4885     48.0680     62.5797     26.6751    325.0720     15.9851     -7.7575     23.0031

file 3
Code:
  0.008   -148.7547     75.6631     76.7972     65.5288     89.3165     56.3463    -15.3538     30.6348
  0.017   -148.7668     73.5426     76.7851     65.5199     89.7731     56.3486    -15.3715     30.6350
  0.025   -148.7836     80.0087     76.7683     65.5057     90.2296     56.3488    -15.3892     30.6352
  0.033   -148.7821     76.5570     76.7698     65.5058     90.6859     56.3469    -15.4069     30.6354
  0.042   -148.7916     78.7440     76.7603     65.4952     91.1420     56.3430    -15.4246     30.6356
  0.050   -148.8006     74.6552     76.7513     65.4836     91.5978     56.3370    -15.4423     30.6359
  0.058   -148.8115     73.8469     76.7404     65.4692     92.0532     56.3289    -15.4600     30.6362
  0.067   -148.8211     75.2542     76.7308     65.4545     92.5083     56.3188    -15.4777     30.6365

# 7  
Old 09-13-2013
I repeat: "Note that there are only 14 lines following the 1st line, but the 1st line contains #data: 15 ;". The script you're using assumes that the 15 in the 1st line of your input file is correct and your output is wrong because that 15 should be 14 for the data in your input file.

If your input data is invalid, you will not get valid output. Or more commonly: GIGO (Garbage in; garbage out).

You can get the output you said want from that input with the following awk script (although I STRONGLY suggest that you not put a space in your filenames):
Code:
awk '
$1 == ">" {
        if(f) close(f)
        f="file " ++n
        next
}
{       print > f
}' UNZA2250.txt

This User Gave Thanks to Don Cragun For This Post:

Previous Thread | Next Thread
Test Your Knowledge in Computers #654
Difficulty: Easy
DOS was initially released on the IBM System/360.
True or False?

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Match text to lines in a file, iterate backwards until text or text substring matches, print to file

hi all, trying this using shell/bash with sed/awk/grep I have two files, one containing one column, the other containing multiple columns (comma delimited). file1.txt abc12345 def12345 ghi54321 ... file2.txt abc1,text1,texta abc,text2,textb def123,text3,textc gh,text4,textd... (6 Replies)
Discussion started by: shogun1970
6 Replies

2. Shell Programming and Scripting

Read text between regexps and write into files based on a field in the text

Hi, I have a huge file that has data something like shown below: huge_file.txt start regexp Name=Name1 Title=Analyst Address=Address1 Department=Finance end regexp some text some text start regexp Name=Name2 Title=Controller Address=Address2 Department=Finance end regexp (7 Replies)
Discussion started by: r3d3
7 Replies

3. Shell Programming and Scripting

Extracting lines from text files in folder based on the numbers in another file

Hello, I have a file ff.txt that looks as follows *ABNA.txt 356 24 36 112 *AC24.txt 457 458 321 2 ABNA.txt and AC24.txt are the files in the folder named foo1. Based on the numbers in the ff.txt file, I want to extract the lines from the corresponding files in the foo1 folder and... (2 Replies)
Discussion started by: mohamad
2 Replies

4. Shell Programming and Scripting

Comparing 2 text files & downloading a file if the last lines are different

Hello I'm having a little difficulty in writing a shell script for a few simple tasks. First I have two files "file1.txt" and "file2.txt" and I want to read and compare the last line of each file. The files look like this. File1.txt File2.txt After comparing the two lines I would... (2 Replies)
Discussion started by: RustikGaming
2 Replies

5. Shell Programming and Scripting

Read text between two lines containing a string

Hi, I have text file like the following: Start a b 121 c d End Start a 31 e f End Start p o i k (5 Replies)
Discussion started by: ysrini
5 Replies

6. UNIX for Dummies Questions & Answers

Extracting lines from a text file based on another text file with line numbers

Hi, I am trying to extract lines from a text file given a text file containing line numbers to be extracted from the first file. How do I go about doing this? Thanks! (1 Reply)
Discussion started by: evelibertine
1 Replies

7. Shell Programming and Scripting

[bash help]Adding multiple lines of text into a specific spot into a text file

I am attempting to insert multiple lines of text into a specific place in a text file based on the lines above or below it. For example, Here is a portion of a zone file. IN NS ns1.domain.tld. IN NS ns2.domain.tld. IN ... (2 Replies)
Discussion started by: cdn_humbucker
2 Replies

8. Shell Programming and Scripting

Shell script to read lines in a text file and filter user data

hi all, I have this file with some user data. example: $cat myfile.txt FName|LName|Gender|Company|Branch|Bday|Salary|Age aaaa|bbbb|male|cccc|dddd|19900814|15000|20| eeee|asdg|male|gggg|ksgu|19911216||| aara|bdbm|male|kkkk|acke|19931018||23| asad|kfjg|male|kkkc|gkgg|19921213|14000|24|... (4 Replies)
Discussion started by: srimal
4 Replies

9. Shell Programming and Scripting

Read any lines of text from file

Witam wszystkich , Jest to moj pierwszy post i już prośba ale gdybym potrafił zaradzić problemowi to nie zawracałbym nikomu głowy . mianowicie : Mam jakis 'plik' w ktorym są osadzone pojedyncze i zmienne słowa po jednym w lini czyli : test1 tekttw resst .... itd. Moje... (6 Replies)
Discussion started by: versace
6 Replies

10. Shell Programming and Scripting

read list of filenames from text file and remove these files in multiple directories

I have a large list of filenames from an Excel sheet, which I then translate into a simple text file. I'd like to use this list, which contains various file extensions , to archive these files and then remove them recursively through multiple directories and subdirectories. So far, it looks like... (5 Replies)
Discussion started by: fxvisions
5 Replies

Featured Tech Videos