Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: awk, associative array, compare files
Top Forums Shell Programming and Scripting awk, associative array, compare files Post 302565688 by shruthi123 on Tuesday 18th of October 2011 01:26:44 PM
Old 10-18-2011
Network
I had following two files file1.txt and file2.txt
File1.txt
Code:
Field1|Filed2|Filed3
'393200103001';'TIM';'20080205'
'393200103017';'TIM';'20040521'
'393200103025';'OPI';'20041025'
'393200103032';'OPI';'20080218'
'393200103048';'OPI';'20101122'
'393200103052';'H3G';'20081204'
'393200103059';'TIM';'20110111'
'393200103061';'TIM';'20060206'
'393200103064';'OPI';'20110623'

File2.txt
Code:
Field1|Filed2|Filed3
'393200103001';'TIM';'20080205'
'393200103017';'TIM';'20040521'
'393200103025';'OPI';'20041025'
'393200103032';'OPI';'20080218'
'393200103048';'OPI';'20101122'
'393200103052';'HKG';'20081204'
'393200103056';'TIM';'20110111'
'393200103088';'TIM';'20060206'

My requirement is that i have to generate three more files like
missed_file1.txt-which should have the records missing that are in file2 but not in file1.
missed_file2.txt--which should have the records missing that are in file1 but not in file2.
common.txt- which should have like this
For '393200103052' filed2 differs file1:'H3G' file2:'HKG'

I think its more clear.I had done the thing but problem was my script is working fine for small files but original files have millions of records.so its taking very much time.

---------- Post updated at 12:26 PM ---------- Previous update was at 12:22 PM ----------
Code:
awk -F '[;$]+' '
        # load first file into array indexed by field 1
        NR == FNR {
                for (i=2; i<=NF; i++) {
                         file1[$1,$2,$3,i] = $i 
                }
                # store the number of fields for this index
                file1nf[$1] = NF
                next
        }
        {
                if (!file1nf[$1]) {
                             print ""$1";"$2";"$3"" >> "missed_file1.txt"
                            next
                }
                    }
' file1.txt file2.txt

same script i had used to generate missed_file2.txt by changing the last line like thsi
file2.txt file1.txt
Last edited by Scott; 10-18-2011 at 02:30 PM.. Reason: Use code tags, please...
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Associative Array

Hi, I am trying to make an associative array to use in a popup_menu on a website. Here is what i have: foreach $entr ( @entries ) { $temp_uid = $entr->get_value(uid); $temp_naam = $entr->get_value(sn); $s++; } This is the popup_menu i want to use it in. popup_menu(-name=>'modcon',... (4 Replies)
Discussion started by: tine
4 Replies

2. Shell Programming and Scripting

Problem with lookup values on AWK associative array

I'm at wits end with this issue and my troubleshooting leads me to believe it is a problem with the file formatting of the array referenced by my script: awk -F, '{if (NR==FNR) {a=$4","$3","$2}\ else {print a "," $0}}' WBTSassignments1.txt RNCalarms.tmp On the WBTSassignments1.txt file... (2 Replies)
Discussion started by: JasonHamm
2 Replies

3. Shell Programming and Scripting

Help needed on Associative array in awk

Hi All, I got stuck up with shell script where i use awk. The scenario which i am working on is as below. I have a file text.txt with contents COL1 COL2 COL3 COL4 1 A 500 400 1 B 500 400 1 A 500 200 2 A 290 300 2 B 290 280 3 C 100 100 I could able to sum col 3 and col4 based on... (3 Replies)
Discussion started by: imsularif
3 Replies

4. Shell Programming and Scripting

Associative array

I have an associative array named table declare -A table table="fruit" table="veggie" table="GT" table="eminem" Now say I have a variable returning the value highway How do I find corresponding value GT ?? (this value that I find (GT in this case) is supposed to be the name of a mysql... (1 Reply)
Discussion started by: leghorn
1 Replies

5. Shell Programming and Scripting

Bash 3d associative array with bash3 AND multiple files

Hello again guru’s (big apologies for wall of text) Still working on that DNS updater for my production team and while there is a ton of hit in searches i can't seem to find the answer to this. Context: We have apps that switch from let’s say host1 to host2. REAL basic DNS clustering... (5 Replies)
Discussion started by: maverick72
5 Replies

6. Shell Programming and Scripting

Morse Code with Associative Array

Continuing my quest to learn BASH, Bourne, Awk, Grep, etc. on my own through the use of a few books. I've come to an exercise that has me absolutely stumped. The specifics: 1. Using ONLY BASH scripting commands (not sed, awk, etc.), write a script to convert a string on the command line to... (22 Replies)
Discussion started by: ksmarine1980
22 Replies

7. Shell Programming and Scripting

Awk: Dealing with whitespace in associative array indicies

Is there a reliable way to deal with whitespace in array indicies? I am trying to annotate fails in a database using a table of known fails. In a begin block I have code like this: # Read in Known Fail List getline < "'"$failListFile"'"; getline < "'"$failListFile"'"; getline <... (6 Replies)
Discussion started by: Michael Stora
6 Replies

8. Shell Programming and Scripting

Using associative array for comparison

Hello together, i make something wrong... I want an array that contains information to associate it for further processing. Here is something from my bash... You will know, what I'm trying to do. I have to point out in advance, that the variable $SYSOS is changing and not as static as in my... (2 Replies)
Discussion started by: Decstasy
2 Replies

9. Shell Programming and Scripting

Associative array index question

I am trying to assign indexes to an associative array in a for loop but I have to use an eval command to make it work, this doesn't seem correct I don't have to do this with regular arrays For example, the following assignment fails without the eval command: #! /bin/bash read -d "\0" -a... (19 Replies)
Discussion started by: Riker1204
19 Replies

10. UNIX for Beginners Questions & Answers

awk Associative Array and/or Referring to Field by String (Nonconstant String Value)

I will start with an example of what I'm trying to do and then describe how I am approaching the issue. File PS028,005 Lexeme HRS # M # PhraseType 1(1:1) 7(7) PhraseLab 501 503 ClauseType ZYq0 PS028,005 Lexeme W # L> # BNH # M #... (17 Replies)
Discussion started by: jvoot
17 Replies

LEARN ABOUT REDHAT

unix2dos

unix2dos(1)						      General Commands Manual						       unix2dos(1)

NAME

       unix2dos - UNIX to DOS text file format converter

SYNOPSYS

       unix2dos [options] [-c convmode] [-o file ...] [-n infile outfile ...]

       Options:

       [-hkqV] [--help] [--keepdate] [--quiet] [--version]

DESCRIPTION

       This manual page documents unix2dos, the program that converts text files in UNIX format to DOS format.

OPTIONS

       The following options are available:

       -h --help
	      Print online help.

       -k --keepdate
	      Keep the date stamp of output file same as input file.

       -q --quiet
	      Quiet mode. Suppress all warning and messages.

       -V --version
	      Prints version information.

       -c --convmode convmode
	      Sets conversion mode. Simulates unix2dos under SunOS.

       -o --oldfile file ...
	      Old file mode. Convert the file and write output to it. The program default to run in this mode. Wildcard names may be used.

       -n --newfile infile outfile ...
	      New  file  mode.	Convert the infile and write output to outfile. File names must be given in pairs and wildcard names should NOT be
	      used or you WILL lost your files.

EXAMPLES

       Get input from stdin and write output to stdout.

	      unix2dos

       Convert and replace a.txt. Convert and replace b.txt.

	      unix2dos a.txt b.txt

	      unix2dos -o a.txt b.txt

       Convert and replace a.txt in ASCII conversion mode.  Convert and replace b.txt in ISO conversion mode.

	      unix2dos a.txt -c iso b.txt

	      unix2dos -c ascii a.txt -c iso b.txt

       Convert and replace a.txt while keeping original date stamp.

	      unix2dos -k a.txt

	      unix2dos -k -o a.txt

       Convert a.txt and write to e.txt.

	      unix2dos -n a.txt e.txt

       Convert a.txt and write to e.txt, keep date stamp of e.txt same as a.txt.

	      unix2dos -k -n a.txt e.txt

       Convert and replace a.txt. Convert b.txt and write to e.txt.

	      unix2dos a.txt -n b.txt e.txt

	      unix2dos -o a.txt -n b.txt e.txt

       Convert c.txt and write to e.txt. Convert and replace a.txt.  Convert and replace b.txt. Convert d.txt and write to f.txt.

	      unix2dos -n c.txt e.txt -o a.txt b.txt -n d.txt f.txt

DIAGNOSTICS

BUGS

       The program does not work properly under MSDOS in stdio processing mode.  If you know why is that so, please tell me.

AUTHOR

       Benjamin Lin - ( blin@socs.uts.edu.au )

MISCELLANY

       Tested environment:

	      Linux 1.2.0 with GNU C 2.5.8

	      SunOS 4.1.3 with GNU C 2.6.3

	      MS-DOS 6.20 with Borland C++ 4.02

       Suggestions and bug reports are welcome.

SEE ALSO

       dos2unix(1)

1995.03.31							   unix2dos v2.2						       unix2dos(1)
All times are GMT -4. The time now is 01:31 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy