Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Difference between two huge .csv files
Top Forums Shell Programming and Scripting Difference between two huge .csv files Post 302711603 by Dimple on Sunday 7th of October 2012 08:00:54 AM
Old 10-07-2012
Difference between two huge .csv files
Hi all,

I need help on getting difference between 2 .csv files.
I have 2 large . csv files which has equal number of columns. I nned to compare them and get output in new file which will have difference olny.

E.g.
File1.csv
Code:
Name, Date, age,number 
Sakshi, 16-12-2011, 22, 56
Akash, 14-12-2011, 23, 76

File2.csv
Code:
Name,Date,age,number
Sakshi, 14-12-2011,22,56
Akash,18-12-2011,23,76

then output should be like
Code:
16-12-2011                      14-12-2011
14-12-2011                       18-12-2011

It's just an example. What I am trying to say is I should get only the values of columns where we have the difference. Not the whole line.
Assuming File will be in sorted order.
There can be m number of columns but for sure in both the files, we will get same columns. If values are different then those values should be given in output.
It can also work if we can get difference in comma separated file like
wherver values matches between 2 files we get blank
,16-12-2011,,
Hope I am able to explain the issue.
Last edited by Franklin52; 10-08-2012 at 04:07 AM.. Reason: Please use code tags for data and code samples
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Difference between two huge files

Hi, As per my requirement, I need to take difference between two big files(around 6.5 GB) and get the difference to a output file without any line numbers or '<' or '>' in front of each new line. As DIFF command wont work for big files, i tried to use BDIFF instead. I am getting incorrect... (13 Replies)
Discussion started by: pyaranoid
13 Replies

2. AIX

Huge difference in reported Disk usage between ls,df and du

IBM RS6000 F50 AIX 4.3.2 i am having trouble in calculating the actual size of a set of directories and reconciling the results with the actual Hard Disk space used I have 33GB disk which is showing 7.8GB used, a byte count of the files in the directory/sub-dirs i`m interested in is 48GB,... (4 Replies)
Discussion started by: cooperuf
4 Replies

3. Programming

Huge difference between _POSIX_OPEN_MAX and sysconf(_SC_OPEN_MAX).

On my Linux system there seems to be a massive difference between the value of _POSIX_OPEN_MAX and what sysconf(_SC_OPEN_MAX) returns and also what I'd expect from the table of examples of configuration limits from Advanced Programming In The UNIX Environment, 2nd Ed. _POSIX_OPEN_MAX: 16... (5 Replies)
Discussion started by: gencon
5 Replies

4. Shell Programming and Scripting

Counting difference in two CSV files

Hi, I am new to awk and trying to count the difference between the first columns of two CSV files. -------- Sample input (header is:name, id1,id2): file1.csv name, id1,id2 sss,34,56 yyy,3,56 www,56,78 pppp,43,12 file2.csv name,id1,id2 sss,32,56 yyy,12,7 ttt,4,8 uuu,7,9 (0 Replies)
Discussion started by: sam40
0 Replies

5. Shell Programming and Scripting

Three Difference File Huge Data Comparison Problem.

I got three different file: Part of File 1 ARTPHDFGAA . . Part of File 2 ARTGHHYESA . . Part of File 3 ARTPOLYWEA . . (4 Replies)
Discussion started by: patrick87
4 Replies

6. Shell Programming and Scripting

Format & Compare two huge CSV files

I have two csv files having 90K records each & each row has around 50 columns.Lets say the file names are FILE1 and FILE2. I have to compare both the files and generate a new file that has rows from FILE2 if it differs. FILE1 ----- 2001,"John",25,19901130,21211.41,Unix Forum... (3 Replies)
Discussion started by: Sheel
3 Replies

7. Shell Programming and Scripting

Comparing 2 difference csv files

Hello, I have about 10 csv files which range from csv1 - csv10. Each csv file has same type/set of tabs and we have around 5-6 tabs for each of the csv file which have slightly different content(data). A sample of CSV1 is shown below: Joins: Data related to Joins, it can be any number of... (2 Replies)
Discussion started by: bobby1015
2 Replies

8. Shell Programming and Scripting

Compare two CSV files and put the difference in third file with line no,field no and diff value.

I am having two csv files i need to compare these files and the output file should have the information of the differences at the field level. For Example, File 1: A,B,C,D,E,F 1,2,3,4,5,6 File 2: A,C,B,D,E,F 1,2,4,5,5,6 out put file: (12 Replies)
Discussion started by: karingulanagara
12 Replies

9. Shell Programming and Scripting

Comparing 2 CSV files and sending the difference to a new csv file

(say) I have 2 csv files - file1.csv & file2.csv as mentioned below: file1.csv ID,version,cost 1000,1,30 2000,2,40 3000,3,50 4000,4,60 file2.csv ID,version,cost 1000,1,30 2000,2,45 3000,4,55 6000,5,70 ... (1 Reply)
Discussion started by: Naresh101
1 Replies

10. Shell Programming and Scripting

Compare 2 csv files in ksh and o/p the difference in a new csv file

(say) I have 2 csv files - file1.csv & file2.csv as mentioned below: file1.csv ID,version,cost 1000,1,30 2000,2,40 3000,3,50 4000,4,60 file2.csv ID,version,cost 1000,1,30 2000,2,45 3000,4,55 6000,5,70 The... (7 Replies)
Discussion started by: Naresh101
7 Replies

LEARN ABOUT DEBIAN

locale::language

Locale::Language(3perl) 				 Perl Programmers Reference Guide				   Locale::Language(3perl)

NAME

       Locale::Language - standard codes for language identification

SYNOPSIS

	  use Locale::Language;

	  $lang = code2language('en');	      # $lang gets 'English'
	  $code = language2code('French');    # $code gets 'fr'

	  @codes   = all_language_codes();
	  @names   = all_language_names();

DESCRIPTION

       The "Locale::Language" module provides access to standard codes used for identifying languages, such as those as defined in ISO 639.

       Most of the routines take an optional additional argument which specifies the code set to use. If not specified, the default ISO 639 two-
       letter codes will be used.

SUPPORTED CODE SETS

       There are several different code sets you can use for identifying languages. The ones currently supported are:

       alpha-2
	   This is the set of two-letter (lowercase) codes from ISO 639, such as 'he' for Hebrew.

	   This code set is identified with the symbol "LOCALE_LANG_ALPHA_2".

	   This is the default code set.

       alpha-3
	   This is the set of three-letter (lowercase) bibliographic codes from ISO 639, such as 'heb' for Hebrew.

	   This code set is identified with the symbol "LOCALE_LANG_ALPHA_3".

       term
	   This is the set of three-letter (lowercase) terminologic codes from ISO 639.

	   This code set is identified with the symbol "LOCALE_LANG_TERM".

ROUTINES

       code2language ( CODE [,CODESET] )
       language2code ( NAME [,CODESET] )
       language_code2code ( CODE ,CODESET ,CODESET2 )
       all_language_codes ( [CODESET] )
       all_language_names ( [CODESET] )
       Locale::Language::rename_language  ( CODE ,NEW_NAME [,CODESET] )
       Locale::Language::add_language  ( CODE ,NAME [,CODESET] )
       Locale::Language::delete_language  ( CODE [,CODESET] )
       Locale::Language::add_language_alias  ( NAME ,NEW_NAME )
       Locale::Language::delete_language_alias	( NAME )
       Locale::Language::rename_language_code  ( CODE ,NEW_CODE [,CODESET] )
       Locale::Language::add_language_code_alias  ( CODE ,NEW_CODE [,CODESET] )
       Locale::Language::delete_language_code_alias  ( CODE [,CODESET] )
	   These routines are all documented in the Locale::Codes man page.

SEE ALSO

       Locale::Codes
       Locale::Constants
       http://www.loc.gov/standards/iso639-2/
	   Source of the ISO 639 codes.

AUTHOR

       See Locale::Codes for full author history.

       Currently maintained by Sullivan Beck (sbeck@cpan.org).

COPYRIGHT

	  Copyright (c) 1997-2001 Canon Research Centre Europe (CRE).
	  Copyright (c) 2001-2010 Neil Bowers
	  Copyright (c) 2010-2011 Sullivan Beck

       This module is free software; you can redistribute it and/or modify it under the same terms as Perl itself.

perl v5.14.2							    2011-09-26						   Locale::Language(3perl)
All times are GMT -4. The time now is 10:20 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy