Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Omit Blank Lines while comparing two files.

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Omit Blank Lines while comparing two files.
# 1  
Old 07-13-2009
Omit Blank Lines while comparing two files.
Hello All,

I am writting file comparison Utility and I have encountered such a senario where there are 2 files such as follows-

Code:
1#!/usr/local/bin/python
2 import gsd.scripts.admin.control.gsdPageEscalate
3.gsd.scripts.admin.control.gsdPageEscalate.main()

Code:
1 #!/usr/local/bin/python
2
3 import gsd.scripts.admin.control.gsdPageEscalate
4
5 gsd.scripts.admin.control.gsdPageEscalate.main()

here the lines of code are exactly the same except for the mismatch in extra blank lines in the 2nd file. I have tried both diff and cmp commands with various options and also tried the sum command , but both these files are never shown identical. Can anyone please tell me if its possible to consider them equal by ingoring the blank lines/white spaces while comparing them?
# 2  
Old 07-13-2009
I can give you a suggestion...
strip off all the blank lines using grep or sed or awk
and then compare.
# 3  
Old 07-13-2009
Try this

comm -13 a1 a2 | sed '/^$/ d'
# 4  
Old 07-13-2009
Guys, I just noticed all the files in 2nd directory have control-M characters ^M in them which is causing all files in 2nd directory to be shown as different. Is there a way I can write a script that will loop through all the files in the 2nd directory and remove the control characters from the file if they exits and save the file and then do the comparison.
# 5  
Old 07-13-2009
Try:
Code:
for i in *
do
   sed 's/^M//g' $i > _tmp."$i"
   mv _tmp."$i" $i
done

# 6  
Old 07-13-2009
Try the below to remove ^M characters:

Code:
perl -i -ne ' s/^M//g; print; *

or,

Code:
perl -e 'while(<>) { chop; print; }'  filename

# 7  
Old 07-13-2009
If you have real carriage return characters (Octal 015), the sed from rakeshawasthi will not work.

Code:
One way of removing spurious carriage return characters:

cat filename | tr -d "\r" > newfilename

BTW: The problem you see usually comes from transferring text files from MSDOS to Unix with binary mode FTP (not text mode ftp). The line terminator in MSDOS is cr/lf whereas in unix it is just lf. A text mode ftp will translate the line terminator but in some versions of ftp will also expand tabs and corrupt extended ascii characters (which you may not want)

Code:
To see the line terminators:

sed -n l filename

cr displays as a $
lf displays as \r
tab displays as \t
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

To check Blank Lines, Blank Records and Junk Characters in a File

Hi All Need Help I have a file with the below format (ABC.TXT) : ®¿¿ABCDHEJJSJJ|XCBJSKK01|M|7348974982790 HDFLJDKJSKJ|KJALKSD02|M|7378439274898 KJHSAJKHHJJ|LJDSAJKK03|F|9898982039999 (cont......) I need to write a script where it will check for : blank lines (between rows,before... (6 Replies)
Discussion started by: chatwithsaurav
6 Replies

2. UNIX for Advanced & Expert Users

Delete blank spaces and blank lines in a file

Hi Gurus, Somebody can say me how to delete blank spaces and blank lines in a file unix, please. Thank you for advanced. (10 Replies)
Discussion started by: systemoper
10 Replies

3. Shell Programming and Scripting

Comparing lines of two different files

Hello, Please help me with this problem if you have a solution. I have two files: <file1> : In each line, first word is an Id and then other words that belong to this Id piMN-1 abc pqr xyz py12 niLM y12 FY4 pqs fiRLym F12 kite red <file2> : same as file1, but can have extra lds... (3 Replies)
Discussion started by: mira
3 Replies

4. Shell Programming and Scripting

Delete blank lines, if blank lines are more than one using shell

Hi, Consider a file named "testfile" The contents of file are as below first line added for test second line added for test third line added for test fourth line added for test fifth line added for test (5 Replies)
Discussion started by: anil8103
5 Replies

5. UNIX for Dummies Questions & Answers

Comparing two files and count number of lines that match

Hello all, I always found help for my problems using the search option, but this time my request is too specific. I have two files that I want to compare. File1 is the index and File2 contains the data: File1: chr1 protein_coding exon 500 600 . + . gene_id "20532";... (0 Replies)
Discussion started by: DerSeb
0 Replies

6. Shell Programming and Scripting

PERL: removing blank lines from multiple files

Hi Guru's , I have a whole bunch of files in /var/tmp that i need to strip any blank lines from, so ive written the following script to identify the lines (which works perfectly).. but i wanted to know, how can I actually strip the identified lines from the actual source files ?? my... (11 Replies)
Discussion started by: hcclnoodles
11 Replies

7. Shell Programming and Scripting

Comparing 2 files and return the unique lines in first file

Hi, I have 2 files file1 ******** 01-05-09|java.xls| 02-05-08|c.txt| 08-01-09|perl.txt| 01-01-09|oracle.txt| ******** file2 ******** 01-02-09|windows.xls| 02-05-08|c.txt| 01-05-09|java.xls| 08-02-09|perl.txt| 01-01-09|oracle.txt| ******** (8 Replies)
Discussion started by: shekhar_v4
8 Replies

8. Shell Programming and Scripting

Comparing two files and appending only missing lines.

Hi All, I am a newbie to Shell scripting. Please help me with the Following problem, 1. I have two files with the same name in different locations in the same machine. Eg: /root/testfolder/a ---- location 1 /tmp/testfolder/a ----- location 2 2. I want to compare the files in... (5 Replies)
Discussion started by: Karthick333031
5 Replies

9. Shell Programming and Scripting

comparing lines from 2 files

Hi Friends, I have 2 files A and B . I want to compare the 3rd line of file A and B . (I dont want to compare the 2 files, using diff or cmp). I just want to know whether 3rd line of A matches the 3 rd line of B. Can anybody share their knowledge on the same? Thanks , Vijaya (12 Replies)
Discussion started by: vijaya2006
12 Replies

10. UNIX for Dummies Questions & Answers

Omit repeating lines

Can someone help me with the following 2 objectives? 1) The following command is just an example. It gets a list of all print jobs. From there I am trying to extract the printer name. It works with the following command: lpstat -W "completed" -o | awk -F- '{ print $1}' Problem is, I want... (6 Replies)
Discussion started by: TheCrunge
6 Replies
Login or Register to Ask a Question