Sponsored Content
Top Forums Shell Programming and Scripting Difference of two data files & writing to an outfile. Post 302530764 by filter on Wednesday 15th of June 2011 02:29:48 AM
Old 06-15-2011
Hi rdcwayx,

Thanks a lot for providing the awk script.
As you said the awk looks simpler and shorter.

I have ran the two awk scripts that you have provided to me for the two different files and have generated the two same.txt and same1.txt files.

Code:
awk '-F\t' 'NR==FNR && /\t/ {a[$38  FS $82]=$0} NR>FNR && /\t/ {if (a[$38 FS $82]=="") {print  > FILENAME ".diff"} else {print > "same.txt"}}' File1 File2

File1.diff --> Records that exists in File1 but not in File2
same.txt --> Records that exists in both the files


Code:
awk '-F\t' 'NR==FNR && /\t/ {a[$38  FS $82]=$0} NR>FNR && /\t/ {if (a[$38 FS $82]=="") {print  > FILENAME ".diff"} else {print > "same1.txt"}}' File1 File2


File2.diff --> Records that exists in File2 but not in File1
same1.txt --> Records that exists in both the files'

Here,
The file size of the same.txt and same1.txt is different.

--> 506108009 Jun 15 01:50 same.txt


--> 505878904 Jun 15 01:52 same1.txt

So, is there anyway that we can capture the records if the Primary Keys ($38 and $82) of both the files matches but the data in the other columns is not the same.

To solve the above issue, I thought once we have the files same.txt and same1.txt we should compare the number of characters in the a line in File1 with number of characters in a line in file2 , if there is a change then write to a file else discard (do nothing or leave it)

Could you please share any ideas to solve the above issue. It would be really grateful.
 

10 More Discussions You Might Find Interesting

1. UNIX Desktop Questions & Answers

what is the difference between Unix & linux, what are the advantages & disadvantages

ehe may i know what are the difference between Unix & Linux, and what are the advantages of having Unix as well as disadvantages of having Unix or if u dun mind i am dumb do pls tell me what are the advantages as well as the disadvantages of having linux as well. thanks (1 Reply)
Discussion started by: cybertechmkteo
1 Replies

2. Shell Programming and Scripting

Need help in writing a script to create a new text file with specific data from existing two files

Hi, I have two text files. Need to create a third text file extracting specific data from first two existing files.. Text File 1: Format contains: SQL*Loader: Release 10.2.0.1.0 - Production on Wed Aug 4 21:06:34 2010 some text ............so on...and somwhere text like: Record 1:... (1 Reply)
Discussion started by: shashi143ibm
1 Replies

3. UNIX for Dummies Questions & Answers

Reading and writing data to and from multiple files

Hi, I have several text files. One main file contains the detail data, other have some information to extract data from the main file, and some are empty files. Examples are shown below: The main file look like: MainFile.txt >Header1 data1...data1... >Header2 data2...data2... ... ...... (2 Replies)
Discussion started by: Fahmida
2 Replies

4. Shell Programming and Scripting

How to combine 2 files and output the unique & difference?

Hi Guys, I have two input files and I want to combine them and get the unique values and differences and put them into one file. See below desired output file. Inputfile1: 1111111 2222222 3333333 7860068 7860069 7860071 7860072 Inputfile2: 4444444 (4 Replies)
Discussion started by: pinpe
4 Replies

5. Shell Programming and Scripting

Sort a the file & refine data column & row format

cat file1.txt field1 "user1": field2:"data-cde" field3:"data-pqr" field4:"data-mno" field1 "user1": field2:"data-dcb" field3:"data-mxz" field4:"data-zul" field1 "user2": field2:"data-cqz" field3:"data-xoq" field4:"data-pos" Now i need to have the date like below. i have just... (7 Replies)
Discussion started by: ckaramsetty
7 Replies

6. Shell Programming and Scripting

Copying the Header & footer Information to the Outfile.

Hi I am writing a perl script which checks for the specific column values from a file and writes to the OUT file. So the feed file has a header information and footer information. I header information isaround107 lines i.e. Starts with START-OF-FILE ....... so on .... ... (11 Replies)
Discussion started by: filter
11 Replies

7. Shell Programming and Scripting

search & merg data from 3 files

i have 3 files which contains as below (example): yy-mm-dd hh:mm:sec lat lon depth mag 2006-01-01 23:17:26.80 39.8405 41.8795 2.0 3.3 2006-01-06 00:10:26.80 39.9570 41.2130 5.0 3.3 2006-01-06 06:59:02.10 39.4099 44.6065 10.0 3.7 2006-01-06 13:49:52.70... (4 Replies)
Discussion started by: oreka18
4 Replies

8. Shell Programming and Scripting

awk help: Match data fields from 2 files & output results from both into 1 file

I need to take 2 input files and create 1 output based on matches from each file. I am looking to match field #1 in both files (Userid) and create an output file that will be a combination of fields from both file1 and file2 if there are any differences in the fields 2,3,4,5,or 6. Below is an... (5 Replies)
Discussion started by: ambroze
5 Replies

9. Shell Programming and Scripting

Help on writing data from 2 different files to one based on a common factor

Hello all, I have 2 text files. For example: File1.txt contains data A B C D ****NEXT**** X Y Z ****NEXT**** L M N and File2.txt contains data (13 Replies)
Discussion started by: vat1kor
13 Replies

10. UNIX for Dummies Questions & Answers

Difference between & and nohup &

Hi All, Can anyone please help me understanding what the difference between the below two? 1. script.sh & 2. nohup script.sh & (2 Replies)
Discussion started by: Anupam_Halder
2 Replies
DIFF(1) 						      General Commands Manual							   DIFF(1)

NAME
diff - print differences between two files SYNOPSIS
diff [-c | -e | -C n] [-br]file1 file2 OPTIONS
-C n Produce output that contains n lines of context -b Ignore white space when comparing -c Produce output that contains three lines of context -e Produce an ed-script to convert file1 into file2 -r Apply diff recursively to files and directories of EXAMPLES
diff file1 file2 # Print differences between 2 files diff -C 0 file1 file2 # Same as above diff -C 3 file1 file2 # Output three lines of context with every diff -c file1 file2 # Same diff /etc /dev # Compares recursively the directories /etc and /dev diff passwd /etc # Compares ./passwd to /etc/passwd DESCRIPTION
the same name, when file1 and file2 are both directories" difference encountered" Diff compares two files and generates a list of lines telling how the two files differ. Lines may not be longer than 128 characters. If the two arguments on the command line are both directories, diff recursively steps through all subdirectories comparing files of the same name. If a file name is found only in one directory, a diagnostic message is written to stdout. A file that is of either block special, character special or FIFO special type, cannot be compared to any other file. On the other hand, if there is one directory and one file given on the command line, diff tries to compare the file with the same name as file in the directory directory. SEE ALSO
cdiff(1), cmp(1), comm(1), patch(1). DIFF(1)
All times are GMT -4. The time now is 01:18 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy