Sponsored Content
Full Discussion: Merging 2 .CSV files in Unix
Top Forums UNIX for Dummies Questions & Answers Merging 2 .CSV files in Unix Post 302191330 by chachabronson on Friday 2nd of May 2008 07:58:04 AM
Old 05-02-2008
Merging 2 .CSV files in Unix

I need a little help as I am a complete novice at scripting in unix. However, i am posed with an issue...Smilie i have two csv files in the following format@

FILE1.CSV:
HEADER
HEADER
Header
, , HEADER
001X ,,200
002X ,,300
003X ,,300
004X ,,300



FILE2.CSV:
HEADER
HEADER
Header
, , HEADER
001X ,,300
003X ,,500
005X ,,600

006X ,,100


I need the above to files to merged into on final file as below:

FINAL_MERGED.CSV:
HEADER
HEADER
Header
, , HEADER
001X ,,300
002X ,,300
003X ,,500
004X ,,300
005X ,,600
006X ,,100



I have tried using the join command but that only lets me to join the two files but as you can see from the above i need the ability to be able to overwrite values from file 1 whith those in file 2.

Any assistance would be greatly appreciated.
Thanks

Last edited by chachabronson; 05-06-2008 at 04:58 AM..
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Merging Non-Empty Columns within a CSV

I am trying to place all my data in a single row (order doesn't matter). Note I am a Unix novice, so please go easy on me. Here is an example Raw data: row# (1) 45 64 23 (2) 32 1 6 56 (3) 32 45 Needs to be like this: row# (1) 45 (2) 32 (3) 32 ... (2 Replies)
Discussion started by: mmann1123
2 Replies

2. Shell Programming and Scripting

Merging files to create CSV file

Hi, I have different files of the same type, as: Time: 100 snr: 88 perf: 10 other: 222 Each of these files are created periodically. What I need to do is to merge all of them into one but having the following form: (2 Replies)
Discussion started by: Ravendark
2 Replies

3. UNIX for Dummies Questions & Answers

Merging two CSV files by 3 primary keys (columns)

Hi there! I have the following problem: I have a set of files called rates_op_yyyyddmm with the format below (which corresponds to the file rates_op_20090130) 30-JAN-2009,ED,FEB09,C,96.375,,,0,,,,,,2.375,,,,,, 30-JAN-2009,ED,FEB09,C,96.5,,,0,,,,,,2.25,,,,,,... (2 Replies)
Discussion started by: Pep Puigvert
2 Replies

4. Shell Programming and Scripting

Matching lines across multiple csv files and merging a particular field

I have about 20 CSV's that all look like this: "","","","","","","","","","","","","","","",""What I've been told I need to produce is the exact same thing, but with each file now containing the start_code from every other file where the email matches. It doesn't matter if any of the other... (1 Reply)
Discussion started by: Demosthenes
1 Replies

5. Shell Programming and Scripting

Merging all (48) CSV files from a directory

I have 48 csv files in my directory that all have this form: Storm Speed (mph),43.0410781151 Storm motion direction (degrees),261.580774982 MLCAPE,2450.54098661 MLCIN,-9.85040520279 MLLCL,230 MLLFC,1070.39871 MLEL,207.194689294 MLCT,Not enough data Sbcape,2203.97617778... (3 Replies)
Discussion started by: RissaR
3 Replies

6. Shell Programming and Scripting

Merging Very large CSV files in Unix

Hi, I have two very large CSV files, which I want to merge (equi-join) based on a key (column). One of the file (say F1) would have ~30 MM records and 700 columns. The other file (~f2) would have same # of records and lesser columns (say 50). I want to create an output file joining on a... (3 Replies)
Discussion started by: student_007
3 Replies

7. Shell Programming and Scripting

Help with merging two CSV files

Hi, I have following 2 CSV files file1.txt A1,B1,C1,D1,E1 A2,B2,C2,D2,E2 A3,B3,C3,D3,E3 .... file2.txt A1,B1,P1,Q1,R1,S1,T1,U1 A1,B1,P2,Q2,R2,S2,T2,U2 A1,B1,P3,Q3,R3,S3,T3,U3 A2,B2,X1,Y1,Z1,I1,J1,K1 A2,B2,X2,Y2,Z2,I2,J2,K2 A2,B2,X3,Y3,Z3,I3,J3,K3 A2,B2,X4,Y4,Z4,I4,J4,K4... (2 Replies)
Discussion started by: learnoutmore99
2 Replies

8. Shell Programming and Scripting

Merging Files in UNIX shell script

I have the urge to merge some files using unix shell script but I'm very new using this language and I haven't succeeded yet. The requirement is to merge the header, body and footer into one file with the name "ANY-NAME" in below example. To identify which files should be merged, I have flagged... (9 Replies)
Discussion started by: black_soul
9 Replies

9. Shell Programming and Scripting

Merging two files in UNIX

Hi Experts, Need urgent solution for a problem. I have two files file1 and file2. file1 is tab separated and file2 is comma separated. I need to merge both the files into single file based on CUST_ID by retaining the headers of file1 Matching CUST_IDs should be placed one below the other in... (11 Replies)
Discussion started by: bharathbangalor
11 Replies

10. Shell Programming and Scripting

Merging fields in CSV

Hi experts, I have a csv file which has one field (ID) repeated multiple times with corresponding other field values. I need to convert this file in a format where for a ID all other values has to be present in single field. For Eg : Here in below file ID 1 is repeated 3 times with different... (7 Replies)
Discussion started by: bharathbangalor
7 Replies
SimpleX(3pm)						User Contributed Perl Documentation					      SimpleX(3pm)

NAME
Net::IMAP::SimpleX - Addons for Net::IMAP::Simple SYNOPSIS
use strict; use warnings; use Net::IMAP::SimpleX; Net::IMAP::SimpleX uses Net::IMAP::Simple as a base so the object creation is the same as it is for the ancestor: my $imap = Net::IMAP::SimpleX->new('imap.example.com') || die "Unable to connect to IMAP: $Net::IMAP::Simple::errstr "; $imap->select("INBOX"); Net::IMAP::SimpleX is a collection of handy methods that are not simple, require Parse::RecDescent, or are experimental. DESCRIPTION
This module adds some useful, yet not so simple, extensions on top of Net::IMAP::Simple. METHODS
new For details on the invocation, read Net::IMAP::Simple. body_summary Typical invocations will take this overall shape. # get an object representation of the message body my $summary = $imap->body_summary($message_number); # multipart message if ($summary->has_parts) { for my $subpart ($summary->parts) { if ($subpart->has_parts) { ... } # examine the message part my @attr = map { $subpart->$_ } qw/content_type encoding encoded_size/; # fetch the raw message part my $subpart_body = $imap->get($message_number, $subpart->part_number); } } else { my $body = $summary->body; my @attr = map { $body->$_ } qw/content_type encoding encoded_size/ } This method returns a simple object that contains a representation of the body of a message. The object is built by a Parse::RecDescent parser using the output of an IMAP fetch body command. The parser uses the formal syntax as defined by RFC3501 http://tools.ietf.org/html/rfc3501#section-9 <http://tools.ietf.org/html/rfc3501#section-9>. my $body = $summary->body; my @attr = map { $body->$_ } qw/ content_description encoded_size charset content_type part_number format id encoding /; For multipart messages, the object contains sub-objects for each message part, accessible via the parts() method and inspected via the has_parts() method. The type method describes the type of multipart (such as mixed or alternative). The parts method returns a list of sub parts, which themselves may have subparts, and so on. An example of a multipart, alternative message with a text body and an html version of the body would looke something like: if ($summary->has_parts) { if ($summary->type eq 'alternative') { my ($html) = grep { $_->content_type eq 'text/html' } $summary->parts; } } A really complex, multipart message could look something like this: if ($summary->has_parts && $summary->type eq 'mixed') { for my $part ($summary->parts) { if ($part->has_parts && $part->type eq 'mixed') { ... } ... } } fetch The fetch command returns the various parts of messages that users request. It is fairly complicated (following RFC3501 using a grammar/parser), but there are some basic patterns that it follows. my $res =$imap->fetch('30:32' => 'UID BODY.PEEK[HEADER.FIELDS (DATE)] FLAGS') # $res = { # 30 => { # "BODY[HEADER.FIELDS (DATE)]" => "Date: Sun, 18 Jul 2010 20:54:48 -0400 ", # "FLAGS" => ["\Flagged", "\Seen"], # "UID" => 58890, # }, # 31 => { # "BODY[HEADER.FIELDS (DATE)]" => "Date: Wed, 21 Jul 2010 09:09:04 -0400 ", # "FLAGS" => ["\Seen"], # "UID" => 58891, # }, # 32 => { # "BODY[HEADER.FIELDS (DATE)]" => "Date: Sat, 24 Jul 2010 05:12:06 -0700 ", # "FLAGS" => ["\Seen"], # "UID" => 58892, # }, # } So-called "parenthized" lists will be returned as an array (see "FLAGS") but nearly everything else will come back as strings. This includes parenthized queries. Take "BODY.PEAK[HEADER.FIELDS (DATE FROM SUBJECT)]"), for example. The result would come back as the RFC822 header lines (as the above "Date: Sun, ..." has done). For more information about the different types of queries, see RFC3501. There's a surprising number of things that can be queried. uidfetch This is roughly the same thing as the "fetch()" method above, but the query runs on UIDs instead of sequence numbers. The keys of the $res are still the sequence numbers though. my $res =$imap->fetch('58890' => 'UID BODY.PEEK[HEADER.FIELDS (DATE)] FLAGS') # $res = { # 30 => { # "BODY[HEADER.FIELDS (DATE)]" => "Date: Sun, 18 Jul 2010 20:54:48 -0400 ", # "FLAGS" => ["\Flagged", "\Seen"], # "UID" => 58890, # }, # ... AUTHOR
INITIAL AUTHOR Jason Woodward "<woodwardj@jaos.org>" ADDITIONAL CONTRIBUTIONS Paul Miller "<jettero@cpan.org>" [fetch()] COPYRIGHT
Copyright (c) 2010 Jason Woodward All rights reserved. This program is free software; you can redistribute it and/or modify it under the same terms as Perl itself. LICENSE
This module is free software. You can redistribute it and/or modify it under the terms of the Artistic License 2.0. This program is distributed in the hope that it will be useful, but without any warranty; without even the implied warranty of merchantability or fitness for a particular purpose. BUGS
https://rt.cpan.org/Dist/Display.html?Queue=Net-IMAP-Simple <https://rt.cpan.org/Dist/Display.html?Queue=Net-IMAP-Simple> SEE ALSO
perl, Net::IMAP::Simple, Parse::RecDescent perl v5.14.2 2012-02-11 SimpleX(3pm)
All times are GMT -4. The time now is 09:46 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy