Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

remove extra lines in the file

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting remove extra lines in the file
# 1  
Old 03-29-2005
remove extra lines in the file
Hi,

I have some files, with some extra lines in weird characters on the top and bottom of the. I want to get rid of those line. Is there a way I can do that?

example of the input file. I want to get rid of those lines in bold
Code:
(B[?25h[?4l[?1l[?3h[?3h
                                                                                                
                                                       07:46 AM Tue Mar 29 2005 
                                                                      Page:   1 
                                                                                
                                     SCAD                                       
                                 sale report                                    
                            Location Sales Report                               
                    03/28/05 12:00AM Thru 03/28/05 11:59PM                      






                           Blackboard Inc.                                  End  



[?3l 
in pick_p 1
in pick_p 1
in pick_p 1
in pick_p 1
in pick_p 1
in pick_p 2
in pick_p 3
in pick_p 4
in pick_p 5
in pick_p 6
in pick_p 7
in pick_p 8
in pick_p 9
in pick_p 0
in pick_p 11
in pick_p -2
our pick_p

Thanks

CT
# 2  
Old 03-29-2005
do 'od -c yourFile' and post the results, pls.
# 3  
Old 03-29-2005
here is the results from "od -c myfile"

Code:
0000000 033   (   B 033   [   ?   2   5   h 033   [   2   J 033   [   ?
0000020   4   l 033   [   ?   1   l 033   [   0   m 033   [   1   ;   2
0000040   4   r 033   [   ?   3   h 033   [   ?   3   h  \n            
0000060                                                                
*
0000200                                                      \n        
0000220                                                                
*
0000300                       0   7   :   4   5       A   M       T   u
0000320   e       M   a   r       2   9       2   0   0   5      \n    
0000340                                                                
*
0000440                       P   a   g   e   :               1      \n
0000460                                                                
*
0000600  \n                                                            
0000620                                                                
0000640                           S   C   A   D                        
0000660                                                                
*
0000720      \n                                                        
0000740                                                                
0000760               s   a   l   e       r   e   p   o   r   t        
0001000                                                                
*
0001040          \n                                                    
0001060                                                               L
0001100   o   c   a   t   i   o   n       S   a   l   e   s       R   e
0001120   p   o   r   t                                                
0001140                                                                
0001160              \n                                                
0001200                                   0   3   /   2   8   /   0   5
0001220       1   2   :   0   0   A   M       T   h   r   u       0   3
0001240   /   2   8   /   0   5       1   1   :   5   9   P   M        
0001260                                                                
0001300                  \n  \n      \n  \n  \n                        
0001320                                                                
0001340                       B   l   a   c   k   b   o   a   r   d    
0001360   I   n   c   .                                                
0001400                                                                
0001420                           E   n   d      \f      \n  \n  \r 033
0001440   [   ?   3   l  \n   i   n       p   i   c   k   _   p       1
0001460  \n   i   n       p   i   c   k   _   p       1  \n   i   n    
0001500   p   i   c   k   _   p       1  \n   i   n       p   i   c   k
0001520   _   p       1  \n   i   n       p   i   c   k   _   p       1
0001540  \n   i   n       p   i   c   k   _   p       2  \n   i   n    
0001560   p   i   c   k   _   p       3  \n   i   n       p   i   c   k
0001600   _   p       4  \n   i   n       p   i   c   k   _   p       5
0001620  \n   i   n       p   i   c   k   _   p       6  \n   i   n    
0001640   p   i   c   k   _   p       7  \n   i   n       p   i   c   k
0001660   _   p       8  \n   i   n       p   i   c   k   _   p       9
0001700  \n   i   n       p   i   c   k   _   p       0  \n   i   n    
0001720   p   i   c   k   _   p       1   1  \n   i   n       p   i   c
0001740   k   _   p       -   2  \n   o   u   r       p   i   c   k   _
0001760   p  \n
0001762

Thanks

CT
# 4  
Old 03-29-2005
Code:
nawk 'BEGIN{chr=sprintf("\033")} $0 !~ chr' myFile

# 5  
Old 03-29-2005
vgersh99,

That works for only the first line in the file, but those last lines still there. I want those to be remove also (those that in bold in the first post):

Code:
0001340                       B   l   a   c   k   b   o   a   r   d    
0001360   I   n   c   .                                                
0001400                                                                
0001420                           E   n   d      \f      \n  \n  \r 033
0001440   [   ?   3   l  \n   i   n       p   i   c   k   _   p       1
0001460  \n   i   n       p   i   c   k   _   p       1  \n   i   n    
0001500   p   i   c   k   _   p       1  \n   i   n       p   i   c   k
0001520   _   p       1  \n   i   n       p   i   c   k   _   p       1
0001540  \n   i   n       p   i   c   k   _   p       2  \n   i   n    
0001560   p   i   c   k   _   p       3  \n   i   n       p   i   c   k
0001600   _   p       4  \n   i   n       p   i   c   k   _   p       5
0001620  \n   i   n       p   i   c   k   _   p       6  \n   i   n    
0001640   p   i   c   k   _   p       7  \n   i   n       p   i   c   k
0001660   _   p       8  \n   i   n       p   i   c   k   _   p       9
0001700  \n   i   n       p   i   c   k   _   p       0  \n   i   n    
0001720   p   i   c   k   _   p       1   1  \n   i   n       p   i   c
0001740   k   _   p       -   2  \n   o   u   r       p   i   c   k   _
0001760   p  \n
0001762

Thanks

CT
# 6  
Old 03-29-2005
Quote:
Originally Posted by CamTu
vgersh99,

That works for only the first line in the file, but those last lines still there. I want those to be remove also (those that in bold in the first post):

Thanks

CT
this is NOT entire correct - the code removes ANY line containing '033' octal character [ESC]. You have the FIRST line AND line number 30 with '033' octal.

All other lines contain no 'escape' character - therefore they're NOT removed.

You can remove the 'trailers' this way - but it's NOT a generic solution:
Code:
nawk 'BEGIN{chr=sprintf("\033")} $0 !~ chr && !/^in pick/ && !/^our pick/' myFile

Last edited by vgersh99; 03-29-2005 at 12:56 PM..
# 7  
Old 03-29-2005
so there is no way i can remove those?

CT
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Removing extra lines from file

I have a file where data looks like this: === <?xml version="1.0" encoding="utf-8"?> <xml xmlns:s='uuid:XYZ' xmlns:dt='uuid:ABC' xmlns:rs='urn:schemas-microsoft-com:rowset' xmlns:z='#RowsetSchema'> <s:Schema id='RowsetSchema'> <s:ElementType name='row'... (7 Replies)
Discussion started by: vx04
7 Replies

2. UNIX for Dummies Questions & Answers

Want to remove all lines but not latest 50 lines from a file

Hi, I have a huge file which has Lacs of lines. File system got full. I want your guys help to suggest me a solution so that I can remove all lines from that file but not last 50,000 lines. I want solution which can remove lines from existing file so that I can have some space left with. (28 Replies)
Discussion started by: prashant2507198
28 Replies

3. UNIX Desktop Questions & Answers

To remove the extra spaces at the end of each line in a file

I have a file of about 10k records and eace line is having an extra space of 5 byte at the end.. Iwant to remove the extra spaces at the end of each line.. Can someone please help me out.. I tried using sed command and its not working... can someone please help me out. (3 Replies)
Discussion started by: rammohan
3 Replies

4. Shell Programming and Scripting

[Solved] Howto remove extra space in the file

Hi Gurus, I have a file which contains some special char or space. when using cat -evt I can see the file as following: 0,"0000","abc/def aaa ... (6 Replies)
Discussion started by: ken6503
6 Replies

5. UNIX for Dummies Questions & Answers

Remove Extra Delimiter

Hi , I have file like this.. aaa|bbbb|cccc|dddd|fff|dsaaFFDFD| Adsads|sas|sa|as|asa|saddas|dsasd|sdad| dsas|dss|sss|sss|ddd|dssd|rrr|fddf| www|fff|refd|dads|fsdf|00sd| 5fgdg|dfs00|d55f|sfds55|445fsd|55ds|sdf| so I do no have any fix pattern and I want to remove extra... (11 Replies)
Discussion started by: pankajchaudhari
11 Replies

6. UNIX for Dummies Questions & Answers

how to remove extra characters

Guys, I have a file that contains entries like this: LaxOrdID=19220288<8> LaxOrdID=19220288 I would like to remove <> and the values inside it anywhere its found. How? (2 Replies)
Discussion started by: DallasT
2 Replies

7. Shell Programming and Scripting

How to remove extra double quotes from string in a delimited file??

Hi Unix Gurus.. I am new to Unix. Please help me. The file I am getting is as follows: Input File "2011-07-06 03:53:23","0","I","NOT SET ",,,,"123985","SAW CUT CONCRETE SLAB 20"THICK",,"98.57","","EACH","N" "2011-07-06 03:53:23","0","I","NOT SET ",,,,"204312","ARMAFLEX-1 3/8 X... (2 Replies)
Discussion started by: BICC
2 Replies

8. Shell Programming and Scripting

Remove extra character

Hi I am using cat <filename> command in one of my datastage job(Command Activity). It is giving actual value but giving extra line. Eg: Displayed Output: 1 and showing extraline(Eg: 1 ) I had checked even wc -c it is giving one character extra. If the file contains 11. wc -c says 3. ... (3 Replies)
Discussion started by: cnrj
3 Replies

9. Shell Programming and Scripting

shell script to remove extra commas from CSV outp file

Name,,,,,,,,,,,,,,,,,,,,Domain,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,Contact,Phone,Email,,,,,,,,,,,,,,,,,,,,,,,,,,,,,Location -----------------------,------------------------------------------------,-------,-----,---------------------------------,------------------------------------ ----... (1 Reply)
Discussion started by: sreenath1037
1 Replies

10. Shell Programming and Scripting

Extra/parse lines from a file between unque lines through the file

I need help to parse a file where there are many records, all of which are consistently separated by lines containing “^=============” and "^ End of Report". Example: ============= 1 2 3 4 End of record ============= 1 3 4 End of record Etc.... I only need specific lines... (5 Replies)
Discussion started by: jouuu
5 Replies
Login or Register to Ask a Question