Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Difference of two data files & writing to an outfile.
Top Forums Shell Programming and Scripting Difference of two data files & writing to an outfile. Post 302530764 by filter on Wednesday 15th of June 2011 02:29:48 AM
Old 06-15-2011
Hi rdcwayx,

Thanks a lot for providing the awk script.
As you said the awk looks simpler and shorter.

I have ran the two awk scripts that you have provided to me for the two different files and have generated the two same.txt and same1.txt files.

Code:
awk '-F\t' 'NR==FNR && /\t/ {a[$38  FS $82]=$0} NR>FNR && /\t/ {if (a[$38 FS $82]=="") {print  > FILENAME ".diff"} else {print > "same.txt"}}' File1 File2

File1.diff --> Records that exists in File1 but not in File2
same.txt --> Records that exists in both the files


Code:
awk '-F\t' 'NR==FNR && /\t/ {a[$38  FS $82]=$0} NR>FNR && /\t/ {if (a[$38 FS $82]=="") {print  > FILENAME ".diff"} else {print > "same1.txt"}}' File1 File2


File2.diff --> Records that exists in File2 but not in File1
same1.txt --> Records that exists in both the files'

Here,
The file size of the same.txt and same1.txt is different.

--> 506108009 Jun 15 01:50 same.txt
--> 505878904 Jun 15 01:52 same1.txt

So, is there anyway that we can capture the records if the Primary Keys ($38 and $82) of both the files matches but the data in the other columns is not the same.

To solve the above issue, I thought once we have the files same.txt and same1.txt we should compare the number of characters in the a line in File1 with number of characters in a line in file2 , if there is a change then write to a file else discard (do nothing or leave it)

Could you please share any ideas to solve the above issue. It would be really grateful.
 

10 More Discussions You Might Find Interesting

1. UNIX Desktop Questions & Answers

what is the difference between Unix & linux, what are the advantages & disadvantages

ehe may i know what are the difference between Unix & Linux, and what are the advantages of having Unix as well as disadvantages of having Unix or if u dun mind i am dumb do pls tell me what are the advantages as well as the disadvantages of having linux as well. thanks (1 Reply)
Discussion started by: cybertechmkteo
1 Replies

2. Shell Programming and Scripting

Need help in writing a script to create a new text file with specific data from existing two files

Hi, I have two text files. Need to create a third text file extracting specific data from first two existing files.. Text File 1: Format contains: SQL*Loader: Release 10.2.0.1.0 - Production on Wed Aug 4 21:06:34 2010 some text ............so on...and somwhere text like: Record 1:... (1 Reply)
Discussion started by: shashi143ibm
1 Replies

3. UNIX for Dummies Questions & Answers

Reading and writing data to and from multiple files

Hi, I have several text files. One main file contains the detail data, other have some information to extract data from the main file, and some are empty files. Examples are shown below: The main file look like: MainFile.txt >Header1 data1...data1... >Header2 data2...data2... ... ...... (2 Replies)
Discussion started by: Fahmida
2 Replies

4. Shell Programming and Scripting

How to combine 2 files and output the unique & difference?

Hi Guys, I have two input files and I want to combine them and get the unique values and differences and put them into one file. See below desired output file. Inputfile1: 1111111 2222222 3333333 7860068 7860069 7860071 7860072 Inputfile2: 4444444 (4 Replies)
Discussion started by: pinpe
4 Replies

5. Shell Programming and Scripting

Sort a the file & refine data column & row format

cat file1.txt field1 "user1": field2:"data-cde" field3:"data-pqr" field4:"data-mno" field1 "user1": field2:"data-dcb" field3:"data-mxz" field4:"data-zul" field1 "user2": field2:"data-cqz" field3:"data-xoq" field4:"data-pos" Now i need to have the date like below. i have just... (7 Replies)
Discussion started by: ckaramsetty
7 Replies

6. Shell Programming and Scripting

Copying the Header & footer Information to the Outfile.

Hi I am writing a perl script which checks for the specific column values from a file and writes to the OUT file. So the feed file has a header information and footer information. I header information isaround107 lines i.e. Starts with START-OF-FILE ....... so on .... ... (11 Replies)
Discussion started by: filter
11 Replies

7. Shell Programming and Scripting

search & merg data from 3 files

i have 3 files which contains as below (example): yy-mm-dd hh:mm:sec lat lon depth mag 2006-01-01 23:17:26.80 39.8405 41.8795 2.0 3.3 2006-01-06 00:10:26.80 39.9570 41.2130 5.0 3.3 2006-01-06 06:59:02.10 39.4099 44.6065 10.0 3.7 2006-01-06 13:49:52.70... (4 Replies)
Discussion started by: oreka18
4 Replies

8. Shell Programming and Scripting

awk help: Match data fields from 2 files & output results from both into 1 file

I need to take 2 input files and create 1 output based on matches from each file. I am looking to match field #1 in both files (Userid) and create an output file that will be a combination of fields from both file1 and file2 if there are any differences in the fields 2,3,4,5,or 6. Below is an... (5 Replies)
Discussion started by: ambroze
5 Replies

9. Shell Programming and Scripting

Help on writing data from 2 different files to one based on a common factor

Hello all, I have 2 text files. For example: File1.txt contains data A B C D ****NEXT**** X Y Z ****NEXT**** L M N and File2.txt contains data (13 Replies)
Discussion started by: vat1kor
13 Replies

10. UNIX for Dummies Questions & Answers

Difference between & and nohup &

Hi All, Can anyone please help me understanding what the difference between the below two? 1. script.sh & 2. nohup script.sh & (2 Replies)
Discussion started by: Anupam_Halder
2 Replies

LEARN ABOUT REDHAT

unix2dos

unix2dos(1)						      General Commands Manual						       unix2dos(1)

NAME

       unix2dos - UNIX to DOS text file format converter

SYNOPSYS

       unix2dos [options] [-c convmode] [-o file ...] [-n infile outfile ...]

       Options:

       [-hkqV] [--help] [--keepdate] [--quiet] [--version]

DESCRIPTION

       This manual page documents unix2dos, the program that converts text files in UNIX format to DOS format.

OPTIONS

       The following options are available:

       -h --help
	      Print online help.

       -k --keepdate
	      Keep the date stamp of output file same as input file.

       -q --quiet
	      Quiet mode. Suppress all warning and messages.

       -V --version
	      Prints version information.

       -c --convmode convmode
	      Sets conversion mode. Simulates unix2dos under SunOS.

       -o --oldfile file ...
	      Old file mode. Convert the file and write output to it. The program default to run in this mode. Wildcard names may be used.

       -n --newfile infile outfile ...
	      New  file  mode.	Convert the infile and write output to outfile. File names must be given in pairs and wildcard names should NOT be
	      used or you WILL lost your files.

EXAMPLES

       Get input from stdin and write output to stdout.

	      unix2dos

       Convert and replace a.txt. Convert and replace b.txt.

	      unix2dos a.txt b.txt

	      unix2dos -o a.txt b.txt

       Convert and replace a.txt in ASCII conversion mode.  Convert and replace b.txt in ISO conversion mode.

	      unix2dos a.txt -c iso b.txt

	      unix2dos -c ascii a.txt -c iso b.txt

       Convert and replace a.txt while keeping original date stamp.

	      unix2dos -k a.txt

	      unix2dos -k -o a.txt

       Convert a.txt and write to e.txt.

	      unix2dos -n a.txt e.txt

       Convert a.txt and write to e.txt, keep date stamp of e.txt same as a.txt.

	      unix2dos -k -n a.txt e.txt

       Convert and replace a.txt. Convert b.txt and write to e.txt.

	      unix2dos a.txt -n b.txt e.txt

	      unix2dos -o a.txt -n b.txt e.txt

       Convert c.txt and write to e.txt. Convert and replace a.txt.  Convert and replace b.txt. Convert d.txt and write to f.txt.

	      unix2dos -n c.txt e.txt -o a.txt b.txt -n d.txt f.txt

DIAGNOSTICS

BUGS

       The program does not work properly under MSDOS in stdio processing mode.  If you know why is that so, please tell me.

AUTHOR

       Benjamin Lin - ( blin@socs.uts.edu.au )

MISCELLANY

       Tested environment:

	      Linux 1.2.0 with GNU C 2.5.8

	      SunOS 4.1.3 with GNU C 2.6.3

	      MS-DOS 6.20 with Borland C++ 4.02

       Suggestions and bug reports are welcome.

SEE ALSO

       dos2unix(1)

1995.03.31							   unix2dos v2.2						       unix2dos(1)
All times are GMT -4. The time now is 07:34 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy