Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Remove certain lines from file based on start of line except beginning and ending

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Remove certain lines from file based on start of line except beginning and ending
# 8  
Old 02-20-2013
Quote:
Originally Posted by nwalsh88
Same again pamu

Everything is perfect apart from the final line of the file which should be the 80 record but is the 70 record
Please check below

1) I am assuming we need to print only first occurrence of a 00 entry and remove remaining all.
2) Remove all instances of an entry starting with 80 except last one
3) Keep other entries as it is.

check..

Code:
$ cat file
00..................
06..................
06..................
06..................
06..................
70..................
80..................
00..................
06..................
06..................
06..................
06..................
70..................
80..................
00..................
06..................
06..................
06..................
06..................
70..................
80..................

Code:
$ awk '!s && /00/{s=1;print}
/80/ && a{K=$0;for(i=1;i<=a;i++){print X[i]};a=0}
!/^00|80/{X[++a]=$0}
END{if(K){print K}
for(i=1;i<=a;i++){print X[i]}
}' file

00..................
06..................
06..................
06..................
06..................
70..................
06..................
06..................
06..................
06..................
70..................
06..................
06..................
06..................
06..................
70..................
80..................

Please let me know if i need to correct anything..

pamu

---------- Post updated at 05:32 PM ---------- Previous update was at 05:29 PM ----------

For more harder input

Code:
$ cat file
06..................
00..................
06..................
06..................
06..................
06..................
70..................
80..................
00..................
06..................
06..................
06..................
06..................
70..................
80..................
00..................
06..................
06..................
06..................
06..................
70..................
80..................
06..................

Code:
$ awk '!s && /00/{for(i=1;i<=a;i++){print X[i]};s=1;a=0;print}
/80/ && a{K=$0;for(i=1;i<=a;i++){print X[i]};a=0}
!/^00|80/{X[++a]=$0}
END{if(K){print K}
for(i=1;i<=a;i++){print X[i]}
}' file

06..................
00..................
06..................
06..................
06..................
06..................
70..................
06..................
06..................
06..................
06..................
70..................
06..................
06..................
06..................
06..................
70..................
80..................
06..................

This User Gave Thanks to pamu For This Post:
nwalsh88
# 9  
Old 02-20-2013
Hi.

An alternative awk solution:
Code:
#!/usr/bin/env bash

# @(#) s1	Demonstrate filter for first and last matches.

# Utility functions: print-as-echo, print-line-with-visual-space, debug.
# export PATH="/usr/local/bin:/usr/bin:/bin"
pe() { for _i;do printf "%s" "$_i";done; printf "\n"; }
pl() { pe;pe "-----" ;pe "$*"; }
db() { ( printf " db, ";for _i;do printf "%s" "$_i";done;printf "\n" ) >&2 ; }
db() { : ; }
C=$HOME/bin/context && [ -f $C ] && $C awk

FILE=${1-data1}

lines=$( wc -l < $FILE )
pl " Input data file edges of $lines lines in $FILE:"
head -3 $FILE ; pe "..." ; tail -3 $FILE

awk '
BEGIN	{ zz = ""; ee = "" }
$0 ~ /^00/ && zz == ""	{ zz = $0 ; print ; next }
$0 ~ /^80/	{ ee = $0 ; next }
$0 !~ /^00/	{ print }
END	{ print ee }
' $FILE > f1
lines=$( wc -l < f1 )
pl " Output data file edges of $lines lines in f1:"
head -3 f1 ; pe "..." ; tail -3 f1

exit 0

producing:
Code:
% ./s1

Environment: LC_ALL = C, LANG = C
(Versions displayed with local utility "version")
OS, ker|rel, machine: Linux, 2.6.26-2-amd64, x86_64
Distribution        : Debian GNU/Linux 5.0.8 (lenny) 
bash GNU bash 3.2.39
awk GNU Awk 3.1.5

-----
 Input data file edges of 23 lines in data1:
00.first............
06..................
00..................
...
80..................
70..................
80.last.............

-----
 Output data file edges of 17 lines in f1:
00.first............
06..................
06..................
...
06..................
70..................
80.last.............

For production, just extract the awk code. The other code is support for data and version display, etc.

Best wishes ... cheers, drl
This User Gave Thanks to drl For This Post:
nwalsh88
# 10  
Old 02-20-2013
Thanks Guys,

Both of the above solved my issue.

Best Regards to both of you
# 11  
Old 02-20-2013
Compared to awk and sed, ed's a much more suitable tool for this task (memory permitting).
Code:
printf '%s\n' '/^00/+1,$ g//d' 1 '1,?^80?-1 g//d' w q | ed -s infile >/dev/null

Regards,
Alister
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Remove lines ending with a certain character

I have a file of a content like this: abc_bla -def 800 abc_bla -def 802 abc_bla -def 804 abc_bla -def 806 abc_bla -def 808 abc_bla -def 810 abc_bla -def 812 abc_bla -def 814 ... abc_bla -def 898 abc_bla -def 900 abc_bla -def 902 abc_bla -def 904 ... abc_bla -def 990 abc_bla -def... (7 Replies)
Discussion started by: maya3
7 Replies

2. Shell Programming and Scripting

awk to remove lines that do not start with digit and combine line or lines

I have been searching and trying to come up with an awk that will perform the following on a converted text file (original is a pdf). 1. Since the first two lines are (begin with) text they are removed 2. if $1 is a number then all text is merged (combined) into one line until the next... (3 Replies)
Discussion started by: cmccabe
3 Replies

3. Shell Programming and Scripting

Add text at start and ending of every line

Hi all, Is there other way to Add text at start and ending of every line? here my script: cat file.txt |awk '{print "<p align=\"justify\">"$0"</p>"}' but the problem they put including white spaces, I only need those line have a sentence or text not an skip all have empty string or have... (7 Replies)
Discussion started by: lxdorney
7 Replies

4. Shell Programming and Scripting

Remove duplicate lines from file based on fields

Dear community, I have to remove duplicate lines from a file contains a very big ammount of rows (milions?) based on 1st and 3rd columns The data are like this: Region 23/11/2014 09:11:36 41752 Medio 23/11/2014 03:11:38 4132 Info 23/11/2014 05:11:09 4323... (2 Replies)
Discussion started by: Lord Spectre
2 Replies

5. Shell Programming and Scripting

Reading line by line from live log file using while loop and considering only those lines start from

Hi, I want to read a live log file line by line and considering those line which start from time stamp; Below code I am using, which read line but throws an exception when comparing line that does not contain error code tail -F /logs/COMMON-ERROR.log | while read myline; do... (2 Replies)
Discussion started by: ketanraut
2 Replies

6. UNIX for Dummies Questions & Answers

Remove lines in a positional file based on string value

Gurus, I am relatively new to Unix scripting and am struck with a problem in my script. I have positional input file which has a FLAG indicator in at position 11 in every record of the file. If the Flag has value =Y, then the record from the input needs to be written to a new file.However if... (3 Replies)
Discussion started by: gsam
3 Replies

7. UNIX for Dummies Questions & Answers

how to remove lines ending with '*'

I have a file where some lines end with '*'. I would like to remove those lines ending with '*'. inFile: a b* c d*outFile: a cThank you (7 Replies)
Discussion started by: jdhahbi
7 Replies

8. Shell Programming and Scripting

How to fetch rows based on line numbers or based on the beginning of a word?

I have a file which will have rows like shown below, ST*820*316054716 RMR*IV*11333331009*PO*40.31 REF*IV*22234441009*xsss471-2762 DTM*003*091016 ENT*000006 RMR*IV*2222234444*PO*239.91 REF*IV*1234445451009*LJhjlkhkj471-2762 </SPAN> DTM*003* 091016 RMR*IV*2223344441009*PO*40.31... (18 Replies)
Discussion started by: Muthuraj K
18 Replies

9. Shell Programming and Scripting

Remove lines based on contents of another file

So, this issue is driving me nuts! I was hoping to get a lending hand here... I have 2 files: file1.txt contains: this is example1 this is example2 this is example3 this is example4 this is example5 file2.txt contains: example3 example5 Basically, I need a script or command to... (4 Replies)
Discussion started by: bashshadow1979
4 Replies

10. Shell Programming and Scripting

Remove white space at the beginning of lines

Hi! I store some data obtained with grep or awk in a file. The problem is that some lines have white space at the begining : line1 line2 line3 I use something like grep WORD INFILE >> OUTFILE awk >> OUTFILE I would love if it were possible to remove the white whitout parsing the... (4 Replies)
Discussion started by: tipi
4 Replies
Login or Register to Ask a Question