Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

awk line extraction question

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting awk line extraction question
# 1  
Old 04-16-2010
awk line extraction question
Hi,

I am trying to extract the first and last lines for each unique item in column 2 of a large text file and then concatenate all extracted lines together in a new text file.

So ... I want to go from this format:

Code:
NEW 0088-BPM 1.042700e+04 877168.19 9718360.00 1496.00
NEW 0088-BPM 1.042800e+04 877143.31 9718359.00 1497.00
NEW 0088-BPM 1.042900e+04 877093.50 9718358.00 1501.00
NEW 0088-BPM 1.042900e+04 877081.06 9718358.00 1502.00
NEW 0088-BPM 1.043000e+04 877056.12 9718358.00 1503.00
NEW 0088-BPM 1.043000e+04 877043.69 9718358.00 1504.00
NEW 0088-BPM 1.043100e+04 877031.25 9718358.00 1505.00
NEW 7493-BPM 4.890000e+03 681176.38 9977126.00 11514.00
NEW 7493-BPM 4.891000e+03 681178.19 9977141.00 11515.00
NEW 7493-BPM 4.892000e+03 681210.62 9977179.00 11519.00
NEW 7493-BPM 4.892000e+03 681220.19 9977187.00 11520.00
NEW 7493-BPM 4.893000e+03 681223.94 9977201.00 11521.00
NEW 7493-BPM 4.893000e+03 681233.50 9977209.00 11522.00
NEW 7493-BPM 4.893000e+03 681243.00 9977217.00 11523.00
......

to this format:
Code:
NEW 0088-BPM 1.042700e+04 877168.19 9718360.00 1496.00
NEW 0088-BPM 1.043100e+04 877031.25 9718358.00 1505.00
NEW 7493-BPM 4.890000e+03 681176.38 9977126.00 11514.00
NEW 7493-BPM 4.893000e+03 681243.00 9977217.00 11523.00
......

Probably very simple!

Thanks,

jazthedribbler
Last edited by Scott; 04-16-2010 at 11:46 AM.. Reason: Please use code tags
# 2  
Old 04-16-2010
Can you try this?
Code:
#!/bin/ksh

sort -k2 inp.txt -o inp.txt
Pre_val="" ;
while read line
do
 Sec_val=`echo $line | awk '{ print $2 }'` ;
 if [[ "${Pre_val}" != "${Sec_val}" ]]
 then
  if [ ! -z $Pre_val ]
  then
   echo $Pre_line ;
  fi
  echo $line ;
 fi
 Pre_line=$line ;
 Pre_val=$Sec_val ;
done < inp.txt
tail -1 inp.txt ;

# 3  
Old 04-16-2010
Code:
awk 'END{print y}NR==1||$2!=x{print ((y)?y RS:z)$0;x=$2}{y=$0}' file

# 4  
Old 04-16-2010
Code:
nawk '$2!=two{if(FNR>1)print prev;print ;two=$2}{prev=$0}' myFile

# 5  
Old 04-16-2010
Quote:
Originally Posted by vgersh99
Code:
nawk '$2!=two{if(FNR>1)print prev;print ;two=$2}{prev=$0}' myFile

You have to print after the END otherwise you miss the last line.
Code:
 nawk '$2!=two{if(NR>1)print prev;print ;two=$2}{prev=$0}' file
NEW 0088-BPM 1.042700e+04 877168.19 9718360.00 1496.00
NEW 0088-BPM 1.043100e+04 877031.25 9718358.00 1505.00
NEW 7493-BPM 4.890000e+03 681176.38 9977126.00 11514.00

 nawk 'END{print prev}$2!=two{print ((NR-1)?prev RS:x)$0;two=$2}{prev=$0}' file
NEW 0088-BPM 1.042700e+04 877168.19 9718360.00 1496.00
NEW 0088-BPM 1.043100e+04 877031.25 9718358.00 1505.00
NEW 7493-BPM 4.890000e+03 681176.38 9977126.00 11514.00
NEW 7493-BPM 4.893000e+03 681243.00 9977217.00 11523.00

# 6  
Old 04-16-2010
Many thanks to danmero, vgersh99 and and bsnithin.

First post to this board. Problem solved. Case closed. Genii the lot of you.

jazthedribbler
Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Multi line extraction based on condition

Hi I have some data in a file as below ****************************** Class 1A Students absent are : 1. ABC 2. CDE 3. CPE ****************************** Class 2A Students absent are : ****************************** Class 3A Students absent are : (6 Replies)
Discussion started by: reldb
6 Replies

2. Shell Programming and Scripting

awk :quick question removing empty line.

How to write in awk to remove lines starting with "#" and then process the file: This is not working: cat file|awk '{if ($0 ~ /^#/) $0="";print NF>0}' When I just give cat file|awk '{if ($0 ~ /^#/) $0="";print }' it prints the blank lines . I don't wnat the blank lines along with the... (15 Replies)
Discussion started by: rveri
15 Replies

3. Shell Programming and Scripting

File extraction without awk

Hello everybody, Here is my problem : I cannot find a way to extract data from a particular file and more precisely I cannot extract the result of my awk script to an external file because I am currently working on HP-UX. I would like a simple script (without awk) which asks for a date like... (4 Replies)
Discussion started by: freyr
4 Replies

4. Shell Programming and Scripting

Another data extraction question

Hi, I have a tmp file like below: <ADATA> ANUM=900 ADESC=Saving ATYP=0 TXREGD=0 </ADATA> <ADATA> ANUM=890 ADESC=Saving ATYP=0 ABAL=9000 TXREGD=1 </ADATA> <ADATA> (6 Replies)
Discussion started by: kunigirib
6 Replies

5. Shell Programming and Scripting

extraction of last but one char in a line

I need to extract the character before the last "|" in the following lines, which are 'N' and 'U'. The last "|" shouldn't be extracted. Also the no.s of "|" may vary in a line, but I need only the character before the last one. ... (1 Reply)
Discussion started by: hidnana
1 Replies

6. Shell Programming and Scripting

AWK extraction

Hi all, I have a data file from which i would like to extract only certain fields, which are not adjacent to each other. Following is the format of data file (data.txt) that i have, which has about 6 fields delimited by "|" HARRIS|23|IT|PROGRAMMER|CHICAGO|EMP JOHN|35|IT|JAVA|NY|CON... (2 Replies)
Discussion started by: harris2107
2 Replies

7. Shell Programming and Scripting

AWK end of line question

Any clues on how to parse a line returned from an ls command that allows for the filename to be fully passed even if it includes spaces? What I got close with is: ls -ltra | awk '{print $1 "|" $3 "|" $4 "|" $5 "|" $6 "|" $7 "|" $8 "|" $9 $10 $11 $12 ... (etc)}' However this clears the spaces in... (6 Replies)
Discussion started by: jpport123
6 Replies

8. Shell Programming and Scripting

AWK extraction

Hi all, Can anyone please help me in parsing the following file. Suppose the file is called, example.lst, and has the following content in it. (DESCRIPTION = (ADDRESS_LIST = (ADDRESS = (PROTOCOL = TCP) (Host = 192.168.2.2) ... (3 Replies)
Discussion started by: harris2107
3 Replies

9. Shell Programming and Scripting

SED scipt help - line extraction

Forgive me if this is a dumb question...I'm a Windows sys admin with little programming knowledge. I have files containing anywhere from 3 to 200 lines. Using SED, I want to extract only lines containing a specific instance of "ISS." It is possible that "ISS" will occur several times in a... (10 Replies)
Discussion started by: thuston22
10 Replies
Login or Register to Ask a Question