Visit Our UNIX and Linux User Community


Compare two text files and output difference


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Compare two text files and output difference
# 1  
Old 08-23-2012
Compare two text files and output difference

Hi experts,

I am trying to compare two text files and output the difference to another file.
I'm not strictly looking for differences in text but additional text at the end of one file that isn't in another, so basically comparing the file 2 against file 1 and printing any additional text to file 3.

Code
Code:
# Tool number
toolnr=`uname -n | cut -c2-`

# Log Directory Path
LOG_path="/usr/asm/atl.1001/user_data/error_logs"

# Input file names
file1="ASML_LOGBOOK.TXT"
file2="ASML_LOGBOOK_2.TXT

diff ${file1} ${file2} |grep "^<" > ${LOG_path}/LOG${toolnr}.TXT

the following is an error output when I run the script:
Error code:
> /usr/asm/atl.1001/user_data/error_logs/LOG1001.TXT^J^J^J: cannot open


Any help greatly appreciated!
# 2  
Old 08-23-2012
Quote:
Originally Posted by martin0852
Hi experts,

I am trying to compare two text files and output the difference to another file.
I'm not strictly looking for differences in text but additional text at the end of one file that isn't in another, so basically comparing the file 2 against file 1 and printing any additional text to file 3.

Code
Code:
# Tool number
toolnr=`uname -n | cut -c2-`

# Log Directory Path
LOG_path="/usr/asm/atl.1001/user_data/error_logs"

# Input file names
file1="ASML_LOGBOOK.TXT"
file2="ASML_LOGBOOK_2.TXT

diff ${file1} ${file2} |grep "^<" > ${LOG_path}/LOG${toolnr}.TXT

the following is an error output when I run the script:
Error code:
> /usr/asm/atl.1001/user_data/error_logs/LOG1001.TXT^J^J^J: cannot open


Any help greatly appreciated!
It appears that you do not have permission to create /usr/asm/atl.1001/user_data/error_logs/LOG1001.TXT if it doesn't already exist, or you do not have permission to erase the current contents and write new data to that file if it does already exist. What is the output of the command:
Code:
ls -l /usr/asm/atl.1001/user_data/error_logs/LOG1001.TXT

after you get that error message?

Just out of curiosity, why do you want to remove the first character of the system's node name when creating a log file for that system?
# 3  
Old 08-23-2012
Hi Don,that code I used was used from another thread on here about comparing two text files. So I'm not sure what that bit is doing, what I thought that was doing was taking the differences in file 2 compared to file 1 and printing them to another file??

I dont have access to the machine at the moment but I should have privileges to read and write files as I do this all the time, after the first error I opened and saved LOGtoolnr.txt so the file actually exists now but it still can't be accessed.

How could i clean up my code to just print the differences to another file?

Thanks
# 4  
Old 08-23-2012
In the error msg the filename that cannot be opened appears to have three trailing <newline> chars. Are you sure that this is what you want? You better double check the filename composing.
# 5  
Old 08-23-2012
Quote:
Originally Posted by martin0852
Hi Don,that code I used was used from another thread on here about comparing two text files. So I'm not sure what that bit is doing, what I thought that was doing was taking the differences in file 2 compared to file 1 and printing them to another file??

I dont have access to the machine at the moment but I should have privileges to read and write files as I do this all the time, after the first error I opened and saved LOGtoolnr.txt so the file actually exists now but it still can't be accessed.

How could i clean up my code to just print the differences to another file?

Thanks
OK. I didn't realize how little experience you have writing scripts. When I saw that you were selecting specific pieces of the output of diff, I assumed you had more knowledge about shell programming than you seem to have.

I went back to your first message in this thread and looked at your script again. The immediate problem is that you are missing a closing quote on the line:
Code:
file2="ASML_LOGBOOK_2.TXT

Therefore, the value you assigned to the file2 variable is:
Code:
"ASML_LOGBOOK_2.TXT

diff ASML_LOGBOOK.TXT ${file2} |grep "

and ending up trying to read input from a file named:
Code:
 " > ${LOG_path}/LOG${toolnr}.TXT

with the name presumably being terminated by a " on a line you didn't show, or by your shell closing the string when it hit the end of file of your script.

If you just copied this code from another thread without understanding what you're doing, I strongly suggest that just replace what you showed us with the command:
Code:
diff ASML_LOGBOOK.TXT ASML_LOGBOOK_2.TXT

to see if the output is what you're expecting. If it is, then change it to:

Code:
diff ASML_LOGBOOK.TXT ASML_LOGBOOK_2.TXT > LOG1001.TXT

If it isn't what you want, read the diff(1) man page to see if there is an option that will give you what you want. If not, explain in detail what you really want in LOG1001.TXT and how that is different from what the diff utility produces by default.
# 6  
Old 08-24-2012
Hi again don,

Thanks again for your reply and input. I'll have access to the UNIX machine later on so I'll try what you suggested.

But to explain what exactly I am looking for in log1001.txt.

A logbook is kept on tool which is used to document work carried out on the tool. At 0700 and 1900 there is a pass down where the actions from last 12 hours need to be documented.

What I am trying to do is save a copy of the logbook (asml_logbook.txt) at 0700 and then at 1900 compare that to the current logbook(asml_logbook_2.txt) to see if there is anything added (ie any actions performed in last 12 hours). If there is it will be added to the end of the current file and so I want to copy this extra text to the third file, log1001.txt.

I have tried to find out if there is a way to search between two date + time stamps depending on current time but this doesn't seem to be possible.

M

---------- Post updated 08-24-12 at 08:40 AM ---------- Previous update was 08-23-12 at 11:26 PM ----------

Quote:
Originally Posted by Don Cragun
OK. I didn't realize how little experience you have writing scripts. When I saw that you were selecting specific pieces of the output of diff, I assumed you had more knowledge about shell programming than you seem to have.

I went back to your first message in this thread and looked at your script again. The immediate problem is that you are missing a closing quote on the line:
Code:
file2="ASML_LOGBOOK_2.TXT

Therefore, the value you assigned to the file2 variable is:
Code:
"ASML_LOGBOOK_2.TXT

diff ASML_LOGBOOK.TXT ${file2} |grep "

and ending up trying to read input from a file named:
Code:
 " > ${LOG_path}/LOG${toolnr}.TXT

with the name presumably being terminated by a " on a line you didn't show, or by your shell closing the string when it hit the end of file of your script.

If you just copied this code from another thread without understanding what you're doing, I strongly suggest that just replace what you showed us with the command:
Code:
diff ASML_LOGBOOK.TXT ASML_LOGBOOK_2.TXT

to see if the output is what you're expecting. If it is, then change it to:

Code:
diff ASML_LOGBOOK.TXT ASML_LOGBOOK_2.TXT > LOG1001.TXT

If it isn't what you want, read the diff(1) man page to see if there is an option that will give you what you want. If not, explain in detail what you really want in LOG1001.TXT and how that is different from what the diff utility produces by default.

Hi Again Don,

I just added the quote at the end of the file name that I was missing and now it seems to have worked, one small thing though, is there a way to get the output without a "<" at the start of each line?

Thanks,

M.
# 7  
Old 08-24-2012
The pipeline that you're running is:
Code:
diff ${file1} ${file2} |grep "^<" > ${LOG_path}/LOG${toolnr}.TXT

Please just enter the command:
Code:
diff ASML_LOGBOOK.TXT ASML_LOGBOOK_2.TXT

and look at the output produced. Note that this command is the first command in your pipeline after the shell expands the file1 and file2 variables.
Note that there are lines starting with < which identify lines that are in file2, but not in file1; lines starting with > which identify lines that are in file1, but not in file2; and lines not starting with < or > that indicate where the lines starting with < and > appear in the files. The second stage in your pipeline:
Code:
grep "^<"

says that you want to discard all of the output produced by diff except for the lines that have < as the first character on the line. This means that you are throwing away all of the information that tells you whether the lines you have selected are lines that are different in file1 than they are in file2 or are only present in file2. You are also throwing away all of the information that shows lines that are in file2, but not present in file1; lines that show how a line that changed appears in file2; and lines that were in file1, but are not present in file2; and all of the lines that indicate where lines that were added, deleted, or changed appear in these files.

If this is what you want and you also want to throw away the first two characters (< and space) from the remaining lines, replace the
Code:
grep "^<"

in your pipeline with:
Code:
awk  '!/^</ {next} {sub(/^../,"");print}'


Previous Thread | Next Thread
Test Your Knowledge in Computers #383
Difficulty: Medium
Solaris 2.5.1 was registered as UNIX 95 compliant on the PReP PowerPC platform.
True or False?

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Script to compare 2 files and prints difference as output sidebyside

Hi All, Am trying script to compare 2 files and print the difference found from old file to new file on line by line basis on side by side display. Basically line by line comparision and files may contain blank line as well I know we have compare/diff commands but i don't how to make... (10 Replies)
Discussion started by: Optimus81
10 Replies

2. Shell Programming and Scripting

Comparing text in 2 files and output difference in another file.

I have 2 files of almost same text apart from 2,3 ending lines. Now I want to get that difference in another file. e.g file1.txt is Filesystem Size Used Avail Use% Mounted on /dev/mapper/vg_livecd-lv_root 18G 2.4G 15G 14% / tmpfs 504M ... (12 Replies)
Discussion started by: kashif.live
12 Replies

3. Shell Programming and Scripting

compare multiple files and get the difference

Hi all, i have 50 files .data should be same in these 50 files , so my task is to find the difference. i need a logic , which finds difference between all files and print in output file with file name where it found that difference . i tried below logic , but its not giving me what i want. let... (2 Replies)
Discussion started by: deepakiniimt
2 Replies

4. Shell Programming and Scripting

Compare two files and output difference, by first field using awk.

It seems like a common task, but I haven't been able to find the solution. vitallog.txt 1310,John,Hancock 13211,Steven,Mills 122,Jane,Doe 138,Thoms,Doe 1500,Micheal,May vitalinfo.txt 12122,Jane,Thomas 122,Janes,Does 123,Paul,Kite **OUTPUT** vitalfiltered.txt 12122,Jane,Thomas... (2 Replies)
Discussion started by: charles33
2 Replies

5. Shell Programming and Scripting

Compare 2 files and output only the different text.

I know the diff does this but it does output more info than just the different text (e.g. $ diff file1 file2 29a30 > /home/alex/Pictures/hello.jpg 1694a1696 > /home/alex/Pictures/hi.jpg ) How can I make it output only /home/alex/Pictures/hello.jpg /home/alex/Pictures/hi.jpg ? thank... (2 Replies)
Discussion started by: hakermania
2 Replies

6. UNIX for Dummies Questions & Answers

compare / difference between sub-sections of files

Hi there, I'm sure this question has been asked many times but I can't find any posts with information. How can I check the differences between say lines 20 - 200 in file1 and lines 420 - 600 in file2? Thanks in advance for any help! js (2 Replies)
Discussion started by: js8765
2 Replies

7. Shell Programming and Scripting

Compare two columns in two files and print the difference

one file . . importing table employee 119 . . importing table jobs 1 2nd file . . importing table employee 120 . . importing table jobs 1 and would like... (2 Replies)
Discussion started by: jhonnyrip
2 Replies

8. Shell Programming and Scripting

Compare two files and print the two lines with difference

I have two files like this: #FILE 1 ABCD 4322 26485 JMTJ 5311 97248 XMPJ 4321 58978 #FILE 2 ABCD 4321 26485 JMTJ 5311 97248 XMPJ 4321 68978 What to do: Compare the two files and find those lines that doesn't match. And have a new file like this: #FILE 3 "from file 1" ABCD 4322 26485... (11 Replies)
Discussion started by: kingpeejay
11 Replies

9. UNIX for Dummies Questions & Answers

Compare Files and Output Difference

I have to compare two files for any differences, then output the lab and question number for any differences. This is what I currently have: diff lab2.txt lab2answer.txt > lab2compare.txt Though the output doesn't have to be sent to a .txt (or any sort of log), I found that easier, at least... (2 Replies)
Discussion started by: Joesgrrrl
2 Replies

10. Shell Programming and Scripting

to compare two files and to print the difference

suppose one file P1168S P2150L P85L Q597R R1097C Another file P2150L P85L Q597R R1097C R1379C R1587K Then output shud be R1379C R1587K thanks (5 Replies)
Discussion started by: cdfd123
5 Replies

Featured Tech Videos