Extracting data between specific lines, multiple times


 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers Extracting data between specific lines, multiple times
# 8  
Old 12-30-2012
Quote:
Originally Posted by captainalright
I tried saving this and using chmod +x so I could run it (I'm not great with Unix so forgive me if I'm saying this wrong), and it ran, but there was no output.
When I was testing it, I had the script I supplied in a filed named tester with an additional line at the beginning of the file:
Code:
#!/bin/ksh

You should add this line to whatever file you used but change /bin/ksh to an absolute pathname of the Korn shell (or any other shell that meets POSIX shell utility requirements) on your system.

I tested it with inputfile containing:
Code:
**Lots of data I don't need before this***

POSITION                                       TOTAL-FORCE (eV/Angst)
 -----------------------------------------------------------------------------------
        Start data from 1st section
      1.86126      1.86973      1.86972         0.000006      0.000006      0.000006
      1.83963      5.61435      5.60302        -0.000013      0.000019      0.000006
      ****A BUNCH MORE OF THESE NUMBERS****
      7.30218      7.59778     13.09455        -0.000012      0.000011      0.000016
     11.04119     11.33677     13.08985        -0.000021      0.000014      0.000014
        End data from 1st section
 -----------------------------------------------------------------------------------
**Lots of data I don't need between these sections****

There's about 50 of these sections, with tons of information before and after the sections I don't need.
I just want the numbers in between the dashes and nothing else outside it.
Again, these sections start with POSITION (etc.) and the dashes and end with the dashes.

I want the output to look like:


Code:
      1.86126      1.86973      1.86972         0.000006      0.000006      0.000006
      1.83963      5.61435      5.60302        -0.000013      0.000019      0.000006
      ****A BUNCH MORE OF THESE NUMBERS****
      7.30218      7.59778     13.09455        -0.000012      0.000011      0.000016
     11.04119     11.33677     13.08985        -0.000021      0.000014      0.000014

with all the data for all 50 sections in a row with no spaces between the data.

**Lots of data I don't need before this***

POSITION                                       TOTAL-FORCE (eV/Angst)
 -----------------------------------------------------------------------------------
        Start data from 2nd section
      1.86126      1.86973      1.86972         0.000006      0.000006      0.000006
      1.83963      5.61435      5.60302        -0.000013      0.000019      0.000006
      ****A BUNCH MORE OF THESE NUMBERS****
      7.30218      7.59778     13.09455        -0.000012      0.000011      0.000016
     11.04119     11.33677     13.08985        -0.000021      0.000014      0.000014
        End data from 2nd section
 -----------------------------------------------------------------------------------
**Lots of data I don't need between these sections****

 -----------------------------------------------------------------------------------
        Start data from 3rd section Should not print
      1.86126      1.86973      1.86972         0.000006      0.000006      0.000006
      1.83963      5.61435      5.60302        -0.000013      0.000019      0.000006
      ****A BUNCH MORE OF THESE NUMBERS****
      7.30218      7.59778     13.09455        -0.000012      0.000011      0.000016
     11.04119     11.33677     13.08985        -0.000021      0.000014      0.000014
        End data from 3rd section
 -----------------------------------------------------------------------------------
**Lots of data I don't need between these sections****

There's about 50 of these sections, with tons of information before and after the sections I don't need.
I just want the numbers in between the dashes and nothing else outside it.
Again, these sections start with POSITION (etc.) and the dashes and end with the dashes.

I want the output to look like:


Code:
      1.86126      1.86973      1.86972         0.000006      0.000006      0.000006
      1.83963      5.61435      5.60302        -0.000013      0.000019      0.000006
      ****A BUNCH MORE OF THESE NUMBERS****
      7.30218      7.59778     13.09455        -0.000012      0.000011      0.000016
     11.04119     11.33677     13.08985        -0.000021      0.000014      0.000014

with all the data for all 50 sections in a row with no spaces between the data.

producing the output:
Code:
	Start data from 1st section
      1.86126      1.86973      1.86972         0.000006      0.000006      0.000006
      1.83963      5.61435      5.60302        -0.000013      0.000019      0.000006
      ****A BUNCH MORE OF THESE NUMBERS****
      7.30218      7.59778     13.09455        -0.000012      0.000011      0.000016
     11.04119     11.33677     13.08985        -0.000021      0.000014      0.000014
	End data from 1st section
	Start data from 2nd section
      1.86126      1.86973      1.86972         0.000006      0.000006      0.000006
      1.83963      5.61435      5.60302        -0.000013      0.000019      0.000006
      ****A BUNCH MORE OF THESE NUMBERS****
      7.30218      7.59778     13.09455        -0.000012      0.000011      0.000016
     11.04119     11.33677     13.08985        -0.000021      0.000014      0.000014
	End data from 2nd section

# 9  
Old 12-30-2012
Code:
You should add this line to whatever file you used but change  /bin/ksh  to an absolute pathname of the Korn shell (or any other shell that meets POSIX shell utility requirements) on your system.

I have no clue what any of this means Smilie. I just had the script you gave me in the same directory as my input file and ran it there. Will that not work?
# 10  
Old 12-30-2012
Quote:
Originally Posted by captainalright
Code:
You should add this line to whatever file you used but change  /bin/ksh  to an absolute pathname of the Korn shell (or any other shell that meets POSIX shell utility requirements) on your system.

I have no clue what any of this means Smilie. I just had the script you gave me in the same directory as my input file and ran it there. Will that not work?
That depends on what operating system and shell you're using. Since you said this script produces no output on your system, I have to assume it is not working with the shell you're using on the operating system you're using.

Or it could be that your input file is not named inputfile, but if that was the case you should have gotten an error message saying that inputfile was not found or couldn't be opened.

If the name of your file containing your input is not inputfile, edit the file containing my script and change inputfile on the last line of the script to the name of your input file. Then try executing the script as follows:
Code:
ksh script

where script is the name of the file in which you saved my script.
# 11  
Old 01-02-2013
I checked that the input name is correct, and it is. I also tried running it with the example input that you tried, instead of using the real file just in case, but using ksh on the script didn't work for either the example input or the real file; both returned nothing.
# 12  
Old 01-02-2013
I've been digging back through your postings on this thread. In the first message in this thread, you said that your input file looked like:
Code:
 POSITION                                       TOTAL-FORCE (eV/Angst)         
 -----------------------------------------------------------------------------------
      1.86126      1.86973      1.86972         0.000006      0.000006      0.000006
      1.83963      5.61435      5.60302        -0.000013      0.000019      0.000006
      ****A BUNCH MORE OF THESE NUMBERS****
      7.30218      7.59778     13.09455        -0.000012      0.000011      0.000016
     11.04119     11.33677     13.08985        -0.000021      0.000014      0.000014
 -----------------------------------------------------------------------------------

Note that there is a leading space on every line.

In the 4th message in this thread you showed us the output you wanted and again showed the input file format, but it had changed to:
Code:
POSITION                                       TOTAL-FORCE (eV/Angst)         
 -----------------------------------------------------------------------------------
      1.86126      1.86973      1.86972         0.000006      0.000006      0.000006
      1.83963      5.61435      5.60302        -0.000013      0.000019      0.000006
      ****A BUNCH MORE OF THESE NUMBERS****
      7.30218      7.59778     13.09455        -0.000012      0.000011      0.000016
     11.04119     11.33677     13.08985        -0.000021      0.000014      0.000014
 -----------------------------------------------------------------------------------
**Lots of data I don't need between these sections****

Note that there is no space before the POSITION on the first line.

The script I gave you is trying to match against the input you showed us in the 4th message in this thread.

Try changing the line in my script that is currently:
Code:
awk '/^POSITION/ {

to:
Code:
awk '/^ POSITION/ {

and trying running the script again. If it still doesn't work, post the output from the following command:
Code:
head -n 15 | od -c

Note change the 15 to a larger number if the start of an entry you want processed doesn't occur in the 1st 15 lines in input file.
This User Gave Thanks to Don Cragun For This Post:
# 13  
Old 01-02-2013
That was the problem! Thank you so much for sticking with me!

One small thing, is there a way to output this to a text file instead of just printing it on the screen?
# 14  
Old 01-02-2013
Quote:
Originally Posted by captainalright
That was the problem! Thank you so much for sticking with me!

One small thing, is there a way to output this to a text file instead of just printing it on the screen?
I'm glad it is working. Smilie

Change the last line of the script from:
Code:
}' inputfile

to:
Code:
}' inputfile > outputfile

changing outputfile to whatever name you want for your output file.
This User Gave Thanks to Don Cragun For This Post:
 
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Extracting data from specific rows and columns from multiple csv files

I have a series of csv files in the following format eg file1 Experiment Name,XYZ_07/28/15, Specimen Name,Specimen_001, Tube Name, Control, Record Date,7/28/2015 14:50, $OP,XYZYZ, GUID,abc, Population,#Events,%Parent All Events,10500, P1,10071,95.9 Early Apoptosis,1113,11.1 Late... (6 Replies)
Discussion started by: pawannoel
6 Replies

2. Shell Programming and Scripting

Extracting data from multiple lines

Hi All, I am stuck in one step.. I have one file named file.txt having content: And SGMT.perd_id = (SELECT cal.fiscal_perd_id FROM $ODS_TARGT.TIM_DT_CAL_D CAL FROM $ODS_TARGT.GL_COA_SEGMNT_XREF_A SGMT SGMT.COA_XREF_TYP_IDN In (SEL COA_XREF_TYP_IDN From... (4 Replies)
Discussion started by: Shilpi Gupta
4 Replies

3. UNIX for Advanced & Expert Users

Extracting specific lines from data file

Hello, Is there a quick awk one-liner for this extraction?: file1 49389 text55 52211 text66 file2 59302 text1 49389 text2 85939 text3 52211 text4 13948 text5 Desired output 49389 text2 52211 text4 Thanks!! (5 Replies)
Discussion started by: palex
5 Replies

4. UNIX for Dummies Questions & Answers

Filtering data -extracting specific lines

I have a table to data which one of the columns include string of text from within that, I am searching to include few lines but not others for example I want to to include some combination of word address such as (address.| address? |the address | your address) but not (ip address | email... (17 Replies)
Discussion started by: A-V
17 Replies

5. Shell Programming and Scripting

Extracting specific lines of data from a file and related lines of data based on a grep value range?

Hi, I have one file, say file 1, that has data like below where 19900107 is the date, 19900107 12 144 129 0.7380047 19900108 12 168 129 0.3149017 19900109 12 192 129 3.2766666E-02 ... (3 Replies)
Discussion started by: Wynner
3 Replies

6. Shell Programming and Scripting

extracting specific text from lines

Hello, i've got this output text: and i need it to look something like this: which means that there won't be absolute path of each directory, just it's size and the last word after last '/' in each line, and i also don't need last line '1.7M /tmp' Looks like there is a simple... (5 Replies)
Discussion started by: krater559
5 Replies

7. Shell Programming and Scripting

extracting specific lines from a file

hi all, i searched in unix.com and accquired the following commands for extracting specific lines from a file .. sed -n '16482,16482p' in.sql > out.sql awk 'NR>=10&&NR<=20' in.sql > out.sql.... these commands are working fine if i give the line numbers as such .. but if i pass a... (2 Replies)
Discussion started by: sais
2 Replies

8. Shell Programming and Scripting

Extracting text out of specific lines

Hi, I have a file like LAHORE 2009-04-16 16:04:19 THU S5830 FAULT MESSAGE SUPPRESS STATUS LOC : ASP00 STS : SUPPRESSING CONTINUE INF : F6201 TRUNK. DATA FAULT REPORT COMPLETED LAHORE 2009-04-16 16:04:20 THU S8400 ISUP SIGNALLING TRACE -... (3 Replies)
Discussion started by: krabu
3 Replies

9. Shell Programming and Scripting

Trying to read data multiple times

I am developing a script to automate Global Mirroring on IBM DS8100's. Part of the process is to establish a global copy and wait until the paired LUN's Out of Sync tracks goes to zero. I can issue a command to display the ouput and am trying to use AWK to read the appropriate field. I am... (1 Reply)
Discussion started by: coachr
1 Replies

10. Shell Programming and Scripting

Trying to read data multiple times

I am developing a script to automate Global Mirroring on IBM DS8100's. Part of the process is to establish a global copy and wait until the paired LUN's Out of Sync tracks goes to zero. I can issue a command to display the ouput and am trying to use AWK to read the appropriate field. I am... (0 Replies)
Discussion started by: coachr
0 Replies
Login or Register to Ask a Question