Extracting data between specific lines, multiple times


 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers Extracting data between specific lines, multiple times
# 1  
Old 12-29-2012
Extracting data between specific lines, multiple times

I need help extracting specific lines in a text file. The file looks like this:

Code:
 POSITION                                       TOTAL-FORCE (eV/Angst)         
 -----------------------------------------------------------------------------------
      1.86126      1.86973      1.86972         0.000006      0.000006      0.000006
      1.83963      5.61435      5.60302        -0.000013      0.000019      0.000006
      ****A BUNCH MORE OF THESE NUMBERS****
      7.30218      7.59778     13.09455        -0.000012      0.000011      0.000016
     11.04119     11.33677     13.08985        -0.000021      0.000014      0.000014
 -----------------------------------------------------------------------------------

with about 50 of these sections. I want just the numbers between all the dashes for all the sections.

The trick is that the file has lots of these dashes (same number of dashes) separating a lot of data I don't need. I specifically need the data between the dashes with the first line of dashes preceded by that first line in the text (POSITION etc.)

Thanks!

Last edited by Scrutinizer; 12-29-2012 at 02:37 PM.. Reason: code tags
# 2  
Old 12-29-2012
I don't understand. Are you saying that you have ~50 sections in a file with each section looking like what you posted and you just want the last line of dashes to be removed from each section?

Is there something else at the start of the file or between sections that is supposed to be ignored?

Please show us more input data and show us what you want the output to be. (And, please use CODE tags.)
# 3  
Old 12-29-2012
'I just want the numbers':
Code:
 awk '$1 ~ /^[0-9]/'  inputfile > outputfile

print only lines where the first non-white character is a number.
# 4  
Old 12-29-2012
Code:
**Lots of data I don't need before this***

POSITION                                       TOTAL-FORCE (eV/Angst)         
 -----------------------------------------------------------------------------------
      1.86126      1.86973      1.86972         0.000006      0.000006      0.000006
      1.83963      5.61435      5.60302        -0.000013      0.000019      0.000006
      ****A BUNCH MORE OF THESE NUMBERS****
      7.30218      7.59778     13.09455        -0.000012      0.000011      0.000016
     11.04119     11.33677     13.08985        -0.000021      0.000014      0.000014
 -----------------------------------------------------------------------------------
**Lots of data I don't need between these sections****

There's about 50 of these sections, with tons of information before and after the sections I don't need. I just want the numbers in between the dashes and nothing else outside it. Again, these sections start with POSITION (etc.) and the dashes and end with the dashes.

I want the output to look like:

Code:
      1.86126      1.86973      1.86972         0.000006      0.000006      0.000006
      1.83963      5.61435      5.60302        -0.000013      0.000019      0.000006
      ****A BUNCH MORE OF THESE NUMBERS****
      7.30218      7.59778     13.09455        -0.000012      0.000011      0.000016
     11.04119     11.33677     13.08985        -0.000021      0.000014      0.000014

with all the data for all 50 sections in a row with no spaces between the data.
# 5  
Old 12-29-2012
Code:
perl -lne "if(/-----/.../-----/ and $_!~/----/){print}" joinl.txt


Last edited by Scott; 12-30-2012 at 01:40 AM.. Reason: Removed links
# 6  
Old 12-30-2012
If I understand the requirements correctly: print all lines between two lines that consist of exactly one space character followed by 83 hyphens if and only if the starting line of hyphens immediately follows a line starting with POSITION, and skip all other lines (including the hyphen lines and the POSITION line); the following awk script should do what you want:
Code:
awk '/^POSITION/ {
        dn = 1  # We expect a line of dashes next.
        next    # Skip to next line.
}
/^ -----------------------------------------------------------------------------------$/ {
        if(dn) {
                # This is a dashes line immediately after a POSITION line.
                dn = 0          # We are no longer looking for starting dashes.
                copy = 1        # Turn on copy mode.
        } else  copy = 0        # We found another line of dashes. Stop copying.
        next    # Skip to next line.
}       
dn {    dn = 0 # We expected a line of dashes but did not find it.  Reset.
        next    # Skip to next line.
}
copy {  print   # If we are in copy mode, print the line.
}' inputfile

If you are using a Solaris system, use /usr/xpg4/bin/awk or nawk instead of awk.
# 7  
Old 12-30-2012
I tried saving this and using chmod +x so I could run it (I'm not great with Unix so forgive me if I'm saying this wrong), and it ran, but there was no output.
 
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Extracting data from specific rows and columns from multiple csv files

I have a series of csv files in the following format eg file1 Experiment Name,XYZ_07/28/15, Specimen Name,Specimen_001, Tube Name, Control, Record Date,7/28/2015 14:50, $OP,XYZYZ, GUID,abc, Population,#Events,%Parent All Events,10500, P1,10071,95.9 Early Apoptosis,1113,11.1 Late... (6 Replies)
Discussion started by: pawannoel
6 Replies

2. Shell Programming and Scripting

Extracting data from multiple lines

Hi All, I am stuck in one step.. I have one file named file.txt having content: And SGMT.perd_id = (SELECT cal.fiscal_perd_id FROM $ODS_TARGT.TIM_DT_CAL_D CAL FROM $ODS_TARGT.GL_COA_SEGMNT_XREF_A SGMT SGMT.COA_XREF_TYP_IDN In (SEL COA_XREF_TYP_IDN From... (4 Replies)
Discussion started by: Shilpi Gupta
4 Replies

3. UNIX for Advanced & Expert Users

Extracting specific lines from data file

Hello, Is there a quick awk one-liner for this extraction?: file1 49389 text55 52211 text66 file2 59302 text1 49389 text2 85939 text3 52211 text4 13948 text5 Desired output 49389 text2 52211 text4 Thanks!! (5 Replies)
Discussion started by: palex
5 Replies

4. UNIX for Dummies Questions & Answers

Filtering data -extracting specific lines

I have a table to data which one of the columns include string of text from within that, I am searching to include few lines but not others for example I want to to include some combination of word address such as (address.| address? |the address | your address) but not (ip address | email... (17 Replies)
Discussion started by: A-V
17 Replies

5. Shell Programming and Scripting

Extracting specific lines of data from a file and related lines of data based on a grep value range?

Hi, I have one file, say file 1, that has data like below where 19900107 is the date, 19900107 12 144 129 0.7380047 19900108 12 168 129 0.3149017 19900109 12 192 129 3.2766666E-02 ... (3 Replies)
Discussion started by: Wynner
3 Replies

6. Shell Programming and Scripting

extracting specific text from lines

Hello, i've got this output text: and i need it to look something like this: which means that there won't be absolute path of each directory, just it's size and the last word after last '/' in each line, and i also don't need last line '1.7M /tmp' Looks like there is a simple... (5 Replies)
Discussion started by: krater559
5 Replies

7. Shell Programming and Scripting

extracting specific lines from a file

hi all, i searched in unix.com and accquired the following commands for extracting specific lines from a file .. sed -n '16482,16482p' in.sql > out.sql awk 'NR>=10&&NR<=20' in.sql > out.sql.... these commands are working fine if i give the line numbers as such .. but if i pass a... (2 Replies)
Discussion started by: sais
2 Replies

8. Shell Programming and Scripting

Extracting text out of specific lines

Hi, I have a file like LAHORE 2009-04-16 16:04:19 THU S5830 FAULT MESSAGE SUPPRESS STATUS LOC : ASP00 STS : SUPPRESSING CONTINUE INF : F6201 TRUNK. DATA FAULT REPORT COMPLETED LAHORE 2009-04-16 16:04:20 THU S8400 ISUP SIGNALLING TRACE -... (3 Replies)
Discussion started by: krabu
3 Replies

9. Shell Programming and Scripting

Trying to read data multiple times

I am developing a script to automate Global Mirroring on IBM DS8100's. Part of the process is to establish a global copy and wait until the paired LUN's Out of Sync tracks goes to zero. I can issue a command to display the ouput and am trying to use AWK to read the appropriate field. I am... (1 Reply)
Discussion started by: coachr
1 Replies

10. Shell Programming and Scripting

Trying to read data multiple times

I am developing a script to automate Global Mirroring on IBM DS8100's. Part of the process is to establish a global copy and wait until the paired LUN's Out of Sync tracks goes to zero. I can issue a command to display the ouput and am trying to use AWK to read the appropriate field. I am... (0 Replies)
Discussion started by: coachr
0 Replies
Login or Register to Ask a Question