|
|
Sponsored Content
Top Forums
Shell Programming and Scripting
Compare and merge two big CSV files
Post 303021868 by SirMannu on Sunday 19th of August 2018 05:54:33 AM
08-19-2018
I attach an example of the two files (only some rows...files are too big
)
in the second file, fields from 2 to 5 are with the same data.
PS: the real file1 is more big of file 2...is that a problem?
)in the second file, fields from 2 to 5 are with the same data.
PS: the real file1 is more big of file 2...is that a problem?
|
|
10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
Hi all,
Am new to scripting. So i just need your ideas to help me out. Here goes my requirement.
I have two csv files
1.csv 2.csv
abc,1.24 abc,1
def,2.13 def,1
I need to compare the first column of 1.csv with 2.csv and if matches then need to compare... (2 Replies)
Discussion started by: chinnahyd
2 Replies
2. UNIX for Advanced & Expert Users
Hi ,
I have a requirement to compare 2 files which can contain 40 million or more records and more than 20 fields to compare .
Currently I am using awk scripting , and since awk has a memory issue, I am not able to process file more than 10 million records.
Any suggestions or pointers to... (7 Replies)
Discussion started by: rashmisb
7 Replies
3. Shell Programming and Scripting
Hi All,
I have two different files as shown below separated by a "|". I need to compare the first column from both the files and if they match merge both the columns.
File 1
"S00172012"|"CHRONIC RENAL FAILURE"|""|"I"
"S00159962"|"SUBENDO INFRC-INIT EPISD"|""|"I"
"S00255303"|"BENIGN... (6 Replies)
Discussion started by: nua7
6 Replies
4. Shell Programming and Scripting
(say) I have 2 csv files - file1.csv & file2.csv as mentioned below:
file1.csv
ID,version,cost
1000,1,30
2000,2,40
3000,3,50
4000,4,60
file2.csv
ID,version,cost
1000,1,30
2000,2,45
3000,4,55
6000,5,70
The... (7 Replies)
Discussion started by: Naresh101
7 Replies
5. Shell Programming and Scripting
Hello:
Newbie with Awk. Trying to compare two files and merge data based on CID. Please see the input file format and desired output. Any help is appreciated.
TIA
Input File1
CID1 --- TYP1 --- DCN1 --- INDATE1 --- IN-DATA1
CID2 --- TYP2 --- DCN2 --- INDATE2 --- IN-DATA2
CID3 ---... (6 Replies)
Discussion started by: wincrazy
6 Replies
6. Shell Programming and Scripting
I have lot of csv file collected from script like below :
Name of files (some examples) there are thousands of it:
192.168.0.123_251_18796_1433144473.csv
192.168.0.123_251_18796_1433144772.csv
192.168.0.123_251_18796_1433145073.csv
192.168.0.123_251_18796_1433145372.csvContent of each... (5 Replies)
Discussion started by: rk4k
5 Replies
7. UNIX for Dummies Questions & Answers
Hi all,
I have two separate csv files(comma delimited) file 1 and file 2.
File 1 contains
PAN,NAME,Salary
AAAAA5467D,Raj,50000
AAFAC5467D,Ram,60000
BDCFA5677D,Kumar,90000
File 2 contains
PAN,NAME,Dept,Salary
ASDFG6756T,Karthik,ABC,450000
QWERT8765Y,JAX,CDR,780000... (5 Replies)
Discussion started by: Nivas
5 Replies
8. Shell Programming and Scripting
Hello everybody
Looking for help in comparing two files in Linux(files are big 800MB each).
Example:-
File1 has below data
$ cat file1
5,6,3
2.1.4
1,1,1
8,9,1
File2 has below data
$ cat file2
5,6,3
8,9,8
1,2,1
2,1,4 (1 Reply)
Discussion started by: shanul karim
1 Replies
9. UNIX for Beginners Questions & Answers
Hello everybody
Looking for help in comparing two files in Linux(files are big 800MB each).
Example:-
File1 has below data
$ cat file1
5,6,3
2.1.4
1,1,1
8,9,1
File2 has below data
$ cat file2
5,6,3
8,9,8
1,2,1
2,1,4 (8 Replies)
Discussion started by: shanul karim
8 Replies
10. UNIX for Beginners Questions & Answers
I have three files with similar pattern i need to merge all the coloumns side by side from all three files according to the first coloumn example as shown below
I mentioned 5 coloumns only in example but i have around 15 coloumns in each file.
file1:
Name,Samples,Error,95RT,90RT... (4 Replies)
Discussion started by: Raghuram717
4 Replies
LEARN ABOUT LINUX
xzdiff
XZDIFF(1) XZ Utils XZDIFF(1) NAME
xzcmp, xzdiff, lzcmp, lzdiff - compare compressed files SYNOPSIS
xzcmp [cmp_options] file1 [file2] xzdiff [diff_options] file1 [file2] lzcmp [cmp_options] file1 [file2] lzdiff [diff_options] file1 [file2] DESCRIPTION
xzcmp and xzdiff invoke cmp(1) or diff(1) on files compressed with xz(1), lzma(1), gzip(1), or bzip2(1). All options specified are passed directly to cmp(1) or diff(1). If only one file is specified, then the files compared are file1 (which must have a suffix of a supported compression format) and file1 from which the compression format suffix has been stripped. If two files are specified, then they are uncom- pressed if necessary and fed to cmp(1) or diff(1). The exit status from cmp(1) or diff(1) is preserved. The names lzcmp and lzdiff are provided for backward compatibility with LZMA Utils. SEE ALSO
cmp(1), diff(1), xz(1), gzip(1), bzip2(1), zdiff(1) BUGS
Messages from the cmp(1) or diff(1) programs refer to temporary filenames instead of those specified. Tukaani 2010-09-27 XZDIFF(1)