Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Merging 2 text files when there is a common time stamp column in them
Top Forums Shell Programming and Scripting Merging 2 text files when there is a common time stamp column in them Post 302798673 by Skrynesaver on Thursday 25th of April 2013 03:39:51 AM
Old 04-25-2013
Taking the data as presented and outputting as a csv.
Code:
skrynesaver@busybox ~/tmp$ cat tmp.dat tmp_2.dat
Date_Time            Context  D1  D2
04/19/2013_23:48:54.819    ABCD   x     x
04/19/2013_23:48:55.307    ABCD   x     x
04/19/2013_23:48:55.823    ABCD   x     x
04/19/2013_23:48:56.309    ABCD   x     x
Date_Time       Context D3  D4
04/19/2013_23:48:54.819     ABCD   y      y
04/19/2013_23:48:55.821     ABCD   y      y
04/19/2013_23:48:56.799     ABCD   y      y
04/19/2013_23:48:57.819     ABCD   y      y

skrynesaver@busybox ~/tmp$ perl -Mstrict -e ' 
my %record;
open (DAT_1 , "<", $ARGV[0]);
while(<DAT_1>){
chomp;
  if (/^\d/){
    my @record=split/\s+/,$_;
    $record{reorder($record[0])}=join(",",@record," "," ");
  }
}
close(DAT_1);
open(DAT_2,"<",$ARGV[1]);
while(<DAT_2>){
  chomp;
  if(/^\d/){
    my @record=split/\s+/,$_;
    $record{reorder($record[0])}=$record{reorder($record[0])} 
                   ?sublast($record{reorder($record[0])},@record[2,3])
                   :join(",",@record[0,1]," "," ",@record[2,3]);
  }
}
END{
  for my $timestamp (sort keys %record){
    print $record{$timestamp},"\n";
  }
}
sub reorder{
  my $date=shift;
  my @date=split/\/|_/,$date;
  return join("_",@date[2,1,0,3]);
}
sub sublast{
  my ($record,@fields)=@_;
  my @record=split/,/,$record;
  @record[4,5]=@fields;
  return join",",@record
}' tmp.dat tmp_2.dat
04/19/2013_23:48:54.819,ABCD,x,x,y,y
04/19/2013_23:48:55.307,ABCD,x,x, ,
04/19/2013_23:48:55.821,ABCD, , ,y,y
04/19/2013_23:48:55.823,ABCD,x,x, ,
04/19/2013_23:48:56.309,ABCD,x,x, ,
04/19/2013_23:48:56.799,ABCD, , ,y,y
04/19/2013_23:48:57.819,ABCD, , ,y,y

 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Merging two files with a common column

Hi, I have two files file1 and file2. I have to merge the columns of those two files into file3 based on common column of two files. To be simple. file1: Row-id name1 13456 Rahul 16789 Vishal 18901 Karan file2 : Row-id place 18901 Mumbai ... (2 Replies)
Discussion started by: manneni prakash
2 Replies

2. Shell Programming and Scripting

Merging 2 files based on a common column

Hi All, I do have 2 files file 1 has 4 tab delimited columns 234 a c dfgyu 294 b g fih 302 c h jzh 328 z c san 597 f g son File 2 has 2 tab delimted columns 234 23 302 24 597 24 I want to merge file 2 with file 1 based on the data common in both files which is the first column so... (6 Replies)
Discussion started by: Lucky Ali
6 Replies

3. UNIX for Dummies Questions & Answers

Merging two text files by a column

So I have two text files. The first one looks like this: refsnp_id chr_name chrom_start 1 rs1000000 12 126890980 2 rs10000010 4 21618674 3 rs10000012 4 1357325 4 rs10000013 4 37225069 5 rs1000002 3 183635768 And the second one looks like this: AUC rs1000000 0.03 0.1240 AUC ... (4 Replies)
Discussion started by: evelibertine
4 Replies

4. UNIX for Dummies Questions & Answers

Merging two text files by a column

I have two text files. One has two columns and looks like below: rs# otherallele_freq rs10399749 0 rs4030303 0 rs4030300 0 rs940550 1.000 rs13328714 0 rs11490937 0 rs6683466 0 rs12025928 1.000 rs6650104 0 rs11240781 0... (5 Replies)
Discussion started by: evelibertine
5 Replies

5. Shell Programming and Scripting

Select files by time stamp

Hi, I need help to read file in a directory on basis of time stamp. e.g. If file access in last 2 minutes it should not be copy to remote directory. Below is my script. +++++++++++++++++++++++++ #!/bin/ksh DATE=`date +"%Y-%m-%d_%H%M"` SEPARATER=" " exec < out_interfaces.cfg... (1 Reply)
Discussion started by: qamar.alam
1 Replies

6. UNIX for Dummies Questions & Answers

Merging two text files by a column and filling in the missing values

Hi, I have to text files that I want to merge by the first column. The values in the first column pretty much match for the first part. However there are some values that are present in column 1 and not present in column 2 or vice versa. For such values I would like to substitute X for the... (9 Replies)
Discussion started by: evelibertine
9 Replies

7. Shell Programming and Scripting

Merging files with common IDs without JOIN

Hi, I am trying to merge information across 2 files. The first file is a "master" file, with all IDS. File 2 contains a subset of IDs of those in File 1. I would like to match up individuals in File 1 and File 2, and add information in File 2 to that of File 1 if they appear. However, if an... (3 Replies)
Discussion started by: hubleo
3 Replies

8. Shell Programming and Scripting

Merging two files without any common pattern

Hi I have file1 as IJU_NSOMOW; SOWWOD_TWUIQ; and file2 as how are you?; fine there; Now my problem is i need the output file as IJU_NSOMOW; how are you?; SOWWOD_TWUIQ; fine there; (2 Replies)
Discussion started by: Priya Amaresh
2 Replies

9. Shell Programming and Scripting

Files with date and time stamp

Hi Folks, Need a clarification on files with date and time stamp. Here is my requirement. There is a file created everyday with the following format "file.txt.YYYYMMDDHHMMSS". Now i need to check for this file and if it is available then i need to do some task to the file. I tried... (6 Replies)
Discussion started by: jayadanabalan
6 Replies

10. Shell Programming and Scripting

Add current time stamp column in existing csv file

Hi , I want to add a new column 'current_time stamp' in my existing csv file with current time stamp for all the records.I tried something this but this is printing 0 with date & time and printed date one line above header.Please help awk -F "," 'BEGIN{ OFS="," } {$6=system("date... (5 Replies)
Discussion started by: netdbaind
5 Replies

LEARN ABOUT OSX

dsmemberutil

dsmemberutil(1) 					    BSD General Commands Manual 					   dsmemberutil(1)

NAME

     dsmemberutil -- various operations for the membership APIs, including state dump, check memberships, UUIDs, etc.

SYNOPSIS

     dsmemberutil [-v] [-h] command [options]

DESCRIPTION

     dsmemberutil is a program that implements the membership API calls in a command line utility.

FLAGS

     A list of flags and their descriptions:

     -h      Lists the options for calling dsmemberutil

     -v      Causes dsmemberutil to operate in verbose mode.

COMMANDS

     The action of each command is described below:

     getuuid -ugUGsS value
		 Takes any of the options and returns the associated UUID.

     getid -UGsSX value
		 Takes any of the options and returns the associated UID or GID depending on option provided.

     getsid -ugUGX value
		 Takes any of the options and returns the associated SID.

     checkmembership -uUxs param -gGXS param
		 Returns if a user or group with the associated option is a member of the group.

     flushcache  Flushes the current membership cache.

     Legacy commands such as dumpstate and statistics are gone. See odutil(1) for show cache and statistics operations.

     A list of options available. In some cases -xX and -sS can be used synonymously due to nature of the value.

     -u uid	      Using user with UID

     -U name	      Using user with name

     -s sid	      Using user with SID

     -x uuid	      Using user with UUID

     -g gid	      Using group with GID

     -G name	      Using group with name

     -S sid	      Using group with SID

     -X uuid	      Using group with UUID

EXAMPLES

     Get a user's uuid:
		    % dsmemberutil getuuid -u 501
		    EEA4F2F6-B268-49E7-9C6F-E3C4A37DA4FD

     Get a group's uuid
		    % dsmemberutil getuuid -g 0
		    ABCDEFAB-CDEF-ABCD-EFAB-CDEF00000000

     Get a user's or group's id from a uuid
		    % dsmemberutil getid -X ABCDEFAB-CDEF-ABCD-EFAB-CDEF0000000C
		    gid: 12

     Check a user's membership in a group (using UID and GID)
		    % dsmemberutil checkmembership -u 501 -g 0
		    user is not a member of the group

     Check a user's membership in a group (using names)
		    % dsmemberutil checkmembership -U root -G wheel
		    user is a member of the group

SEE ALSO

     odutil(1), dseditgroup(1), dscacheutil(1)

Darwin								    Jan 1, 2007 							    Darwin
All times are GMT -4. The time now is 07:01 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy