Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Comparing two huge files on field basis.
Top Forums Shell Programming and Scripting Comparing two huge files on field basis. Post 302523143 by ygemici on Wednesday 18th of May 2011 04:56:05 AM
Old 05-18-2011
Code:
# ./justdoit file1 file2
Record no.8 is different and the field is ( field no 2 / field value MADHAVI ) in file2
Record no.11 is different and the field is ( field no 3 / field value PAWAR ) in file2

Code:
# ./justdoit file2 file1
Record no.8 is different and the field is ( field no 2 / field value MADHURA ) in file1
Record no.11 is different and the field is ( field no 3 / field value PAWAN ) in file1

Code:
# cat justdoit
#!/bin/bash
ix=0
while read -r linef
do
compar[ix]=$linef
((ix++))
done<$1
ix=0
while read -r lineff
do
comparr[ix]=$lineff
((ix++))
done<$2
x=0;ix=0;c=0
for i in "${compar[@]}"
do
 ((c++))
if [ "$i" != "${comparr[x]}" ] ; then
 chk=(${compar[x]}) ; chk2=(${comparr[x]})
 for j in ${chk[@]}
  do
  ((fno++))
  if [ "$j" != "${chk2[ix]}" ] ; then
  echo "Record no.$c is different and the field is ( field no $fno / field value ${chk2[ix]} ) in $2"
  fi
  ((ix++))
  done
fi
((x++)) ; ix=0;fno=0
done

regards
ygemici
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

comparing Huge Files - Performance is very bad

Hi All, Can you please help me in resolving the following problem? My requirement is like this: 1) I have two files YESTERDAY_FILE and TODAY_FILE. Each one is having nearly two million data. 2) I need to check each record of TODAY_FILE in YESTERDAY_FILE. If exists we can skip that by... (5 Replies)
Discussion started by: madhukalyan
5 Replies

2. Shell Programming and Scripting

Last field problem while comparing two csv files

Hi All, I've two .csv files as below file1.csv abc, tdf, 223, tpx jgsd, tex, 342, rpy a, jdjdsd, 423, djfkld Where as file2.csv is the new version of file1.csv with some added fields in the end of each line and some additional lines. lfj, eru, 98, jkldj, 39, jdkj9 abc, tdf, 223, tpx,... (3 Replies)
Discussion started by: ganapati
3 Replies

3. Shell Programming and Scripting

Field comparing in files

Guys trying to compare field in two files. For Ex: demo.txt 23.33.4.2 hostname 3.2.4.2 hostname12 demo1.txt 3.3.3.3 hostname23 45.23.23.23 hostname 323 I would like to compare the ips b/w these two files.any... (2 Replies)
Discussion started by: coolkid
2 Replies

4. Shell Programming and Scripting

Comparing two huge files

Hi, I have two files file A and File B. File A is a error file and File B is source file. In the error file. First line is the actual error and second line gives the information about the record (client ID) that throws error. I need to compare the first field (which doesnt start with '//') of... (11 Replies)
Discussion started by: kmkbuddy_1983
11 Replies

5. Shell Programming and Scripting

comparing files with field using awk

hi, i have 1 files a.csv temp.out a.cvs looks like add,16390,180,674X,HALIFAX_COMMONS_X,902,497,902-209 add,16390,180,674X,HALIFAX_COMMONS_X,902,497,902-219 add,16390,180,674X,HALIFAX_COMMONS_X,902,497,902-220 add,16390,180,674X,HALIFAX_COMMONS_X,902,497,902-221 and temp.out looks... (1 Reply)
Discussion started by: raghavendra.cse
1 Replies

6. UNIX for Advanced & Expert Users

Sort on basis of particular field

05.50.25:AIRE.S:RESTRICTED:S2: 05.50.25:ANDR.VI:RESTRICTED:S2: 05.50.25:BASF.MI:RESTRICTED:N: 05.50.25:BMWG.DE:RESTRICTED:N: 05.50.25:BORE.ST:RESTRICTED:N: I can sort of the basis of second field using " sort -t: -k2,2 " but what i want to sort the data is on the basis its extension e.g.:... (9 Replies)
Discussion started by: manishma71
9 Replies

7. Shell Programming and Scripting

Comparing 2 huge text files

I have this 2 files: k5login sanwar@systems.nyfix.com jjamnik@systems.nyfix.com nisha@SYSTEMS.NYFIX.COM rdpena@SYSTEMS.NYFIX.COM service/backups-ora@SYSTEMS.NYFIX.COM ivanr@SYSTEMS.NYFIX.COM nasapova@SYSTEMS.NYFIX.COM tpulay@SYSTEMS.NYFIX.COM rsueno@SYSTEMS.NYFIX.COM... (11 Replies)
Discussion started by: linuxgeek
11 Replies

8. Shell Programming and Scripting

Comparing the matches in two files using awk when both files have their own field separators

I've two files with data like below: file1.txt: AAA,Apples,123 BBB,Bananas,124 CCC,Carrot,125 file2.txt: Store1|AAA|123|11 Store2|BBB|124|23 Store3|CCC|125|57 Store4|DDD|126|38 So,the field separator in file1.txt is a comma and in file2.txt,it is | Now,the output should be... (2 Replies)
Discussion started by: asyed
2 Replies

9. Shell Programming and Scripting

Perl: Need help comparing huge files

What do i need to do have the below perl program load 205 million record files into the hash. It currently works on smaller files, but not working on huge files. Any idea what i need to do to modify to make it work with huge files: #!/usr/bin/perl $ot1=$ARGV; $ot2=$ARGV; open(mfileot1,... (12 Replies)
Discussion started by: mrn6430
12 Replies

10. Shell Programming and Scripting

Comparing two files field and position

I have two files : file1 and file2 file1 format: 7026-H70 7026-1017685 7026-H70 7026-1017687 7026-B80 7026-108D65A 7026-B80 7026-108D67A 7046-B50 7026-1034B4A File2 format : mt01cp01 7026-B80 01108D69A mt01cp02 7026-B80 01108D68A mt01sv01 7046-B50... (7 Replies)
Discussion started by: amir07
7 Replies

LEARN ABOUT DEBIAN

mencal

mencal(1)								 1								 mencal(1)

NAME

       mencal - menstruation calendar

SYNOPSIS

       mencal [options] [file1 file2 ... -c CONF1 -c CONF2 ...]

DESCRIPTION

   Display options:
       -m, --monday
	      draw monday as first weekday (sunday is default)

       -1     current month (default)

       -3     previous, current and next month

       -y [YYYY]
	      all-year calendar (default YYYY is current year)

       -q, --quiet
	      no top information will be printed

       -n, --nocolor
	      noncolored output

       -i, --icolor COLOR
	      intersection color (default red)

	      available colors: red, green, blue, yellow, violet, cyan, shiny, bold

   mencal configuration:
       -c, --config
	      s=[YYYY]MMDD,l=LL,d=DD,n=NAME,f=FILE,c=COLOR

	      The  second argument is a comma separated list of options. No spaces are allowed in this list. If no name is specified, 'Unknown' is
	      used.  Various -c options or filenames can be set.

       s,start=[YYYY]MMDD
	      start day of period (default current day)

       l,length=LL
	      length of period in days (default 28)

       d,duration=D
	      duration of menstruation in days (default 4)

       n,name=NAME
	      name of subject

       f,file=FILE
	      filename to save configuration to (see section FILES)

	      only menstruation related variables will be saved

       c,color=COLOR
	      color used for menstruation days

	      available colors: red, green, blue, yellow, violet, cyan, shiny, bold default color is red, with	'-n'  switch  color  settings  are
	      ignored

   Info options:
       -h, --help
	      print help

       -V, --version
	      print version information

FILES
:
       $HOME/.mencalrc - the default configuration file that is automatically loaded

AUTHOR
:
       (C) 2002 C. McCohy
       e-mail: <mccohy@kyberdigi.cz>
       Word-Wide-Web: http://www.kyberdigi.cz/projects/mencal/english

       This  manual  page  was	written for the Debian GNU/Linux distribution because the original program does not have a manual page (but may be
       used by others).

HISTORY

       22 March 2002 - Originally written by Amaya Rodrigo <amaya@debian.org>.
       24 Apr 2002 - New manpage contributed by Pablo S. Torralba <psanchez@skyrealms.org>.
       Amaya Rodrigo <amaya@debian.org> is now responsible for developing and maintaining this manual page. Comments and suggestions  are  greatly
       wellcome.

mencal							       Menstruation calendar							 mencal(1)
All times are GMT -4. The time now is 08:58 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy