Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

perl merge two files by the time sequence

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting perl merge two files by the time sequence
# 1  
Old 06-12-2009
Lightbulb perl merge two files by the time sequence
Hi Guys,
i have two files:
fileA:
20090611 00:00:11
20090611 00:00:11
20090611 00:00:24
20090611 00:01:10
20090611 07:13:00

fileB:
20090611 00:00:01
20090611 00:00:12
20090611 00:00:24
20090611 00:01:12
20090611 09:13:00

want to make two files into a single file, but follow the time sequence, the output fileC:
20090611 00:00:01
20090611 00:00:11
20090611 00:00:11
20090611 00:00:12
20090611 00:00:24
20090611 00:00:24
20090611 00:01:10
20090611 00:01:12
20090611 07:13:00
20090611 09:13:00

instead of one by one compare the time, any simple way to doing it?

Thanks
# 2  
Old 06-12-2009
Quote:
Originally Posted by jimmy_y
...
i have two files:
fileA:
20090611 00:00:11
20090611 00:00:11
20090611 00:00:24
20090611 00:01:10
20090611 07:13:00

fileB:
20090611 00:00:01
20090611 00:00:12
20090611 00:00:24
20090611 00:01:12
20090611 09:13:00

want to make two files into a single file, but follow the time sequence, the output fileC:
20090611 00:00:01
20090611 00:00:11
20090611 00:00:11
20090611 00:00:12
20090611 00:00:24
20090611 00:00:24
20090611 00:01:10
20090611 00:01:12
20090611 07:13:00
20090611 09:13:00

instead of one by one compare the time, any simple way to doing it?

...
You sure you want to use Perl to do that ?

One simple way could be:

Code:
$
$ sort filea fileb
20090611 00:00:01
20090611 00:00:11
20090611 00:00:11
20090611 00:00:12
20090611 00:00:24
20090611 00:00:24
20090611 00:01:10
20090611 00:01:12
20090611 07:13:00
20090611 09:13:00
$

Using perl -

Code:
$
$ perl -e '{open(F1,"filea"); @a1=<F1>; close(F1); open(F2,"fileb"); @a2=<F2>; close(F2); print sort(@a1,@a2)}'
20090611 00:00:01
20090611 00:00:11
20090611 00:00:11
20090611 00:00:12
20090611 00:00:24
20090611 00:00:24
20090611 00:01:10
20090611 00:01:12
20090611 07:13:00
20090611 09:13:00
$
$

tyler_durden
# 3  
Old 06-12-2009
Thanks tyler Smilie,

i like the sort so much, thanks for this.
but if my fileA:
4 20090611 00:00:11
5 20090611 00:00:11
3 0090611 00:00:24
3 20090611 00:01:10
10 20090611 07:13:00

fileB:
4 20090611 00:00:01
11 20090611 00:00:12
22 20090611 00:00:24
9 20090611 00:01:12
9 20090611 09:13:00

So for each file, there is another field in the head, so if still want to get the date sort, then seems cannot use 'sort filea fileb'. Smilie

-----Post Update-----

Oh, just saw can use "-k" with sort.

-----Post Update-----

Oh, just saw can use "-k" with sort.
# 4  
Old 06-13-2009
Hi Guys,

I know how to use the sort, -k to sort two files, but face the problem where need to sort differient fields in differient files.

FileA:
20090610 23:58:59 444
20090610 23:52:13 555
20090611 00:00:02 6

FileB:
1 63660 5807 Wed 10/06/2009 23:54:58
4 63660 5807 Wed 10/06/2009 23:59:58
33 63660 5807 Wed 11/06/2009 23:54:58

###################

So For FileA, field1 and field2 sort date, and for FileB, sort date by field date. As we can see fileA and B date are in differient fields, how to sort these two files?

Thanks
# 5  
Old 06-13-2009
- What do you want the output to look like ?
- Do you want to sort each file individually ?
- If you want to combine the contents of both files, the data won't align properly as the files have different number of fields.

Post the output that you expect, taking your files as the example.

tyler_durden
# 6  
Old 06-13-2009
Hi tyler,

Thanks. the output file will combine these two files sort by the date:
20090610 23:52:13 555
1 63660 5807 Wed 10/06/2009 23:54:58
20090610 23:58:59 444
4 63660 5807 Wed 10/06/2009 23:59:58
20090611 00:00:02 6
33 63660 5807 Wed 11/06/2009 23:54:58
###########
so sort by date. The reason for such two files differient date field and date format different, becuase those two files are from two differient billing.

Thanks.

-----Post Update-----

The output also can be below, means can remove FileB before data characters. or if possible change the fileB data format to fileA data format.
#########################
20090610 23:52:13 555
Wed 10/06/2009 23:54:58
20090610 23:58:59 444
Wed 10/06/2009 23:59:58
20090611 00:00:02 6
Wed 11/06/2009 23:54:58
######################

:--( see cut, paste, join, sort, but no luck.
Last edited by jimmy_y; 06-13-2009 at 03:20 PM..
# 7  
Old 06-13-2009
Jumping back to perl... Assuming the last field of FileA and the first field of FileB are not important for the sort...

Code:
use strict;
use warnings;
my @all_lines;
open(my $FA, 'filea.txt') or die "$!";
while(<$FA>) {
   chomp;
   push @all_lines,[join('',(split(/\s+/))[0,1]),$_];
}
close $FA;
open(my $FB, 'fileb.txt') or die "$!";
while(<$FB>) {
   chomp;
   my($d,$t) = (split(/\s+/))[4,5];
   my $d2 = join('',(split(/\//,$d))[2,1,0]);
   push @all_lines,["$d2$t",$_];
}
close $FB;
my @sorted = sort{$a->[0] cmp $b->[0]} @all_lines;  
print "$_->[1]\n" for @sorted;

Uses a 2D array to store the sort keys (YYYYMMDDHH::MM::SS) and the original lines. Then sorts the sort keys but returns the original lines.

If these are really big files you might consider using a database instead of a scripting langugae to store the data then you can do all sorts of wonderful things with it more efficiently and maybe easier in the long run.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Help with merge data with a reference sequence

I have two input file.: File 1 is a large reference sequence (A large Fasta sequence); File 1 (is a file which first line is the header description and line other ">" is its corresponding word and counting from 1 till end of file); >Data_1 ASWDADAQTWQQGSAAAAASDAFAFA . . File 2 is... (31 Replies)
Discussion started by: cpp_beginner
31 Replies

2. Shell Programming and Scripting

Perl script to Merge contents of 2 different excel files in a single excel file

All, I have an excel sheet Excel1.xls that has some entries. I have one more excel sheet Excel2.xls that has entries only in those cells which are blank in Excel1.xls These may be in different workbooks. They are totally independent made by 2 different users. I have placed them in a... (1 Reply)
Discussion started by: Anamika08
1 Replies

3. Shell Programming and Scripting

Awk or Perl - to selectively merge two files.

I have two files, these have to be selectively merged into two other files. In addition there will require to be a edit to the last field, where the date format is changed. The first file is a csv file with around 300k lines the data is now nearly 20 years old and I have been asked to move this... (7 Replies)
Discussion started by: gull04
7 Replies

4. Shell Programming and Scripting

Pick files after specified Time from a folder in Perl

Hi All I am stuck in a problem and wants help What i want to do is I have a directory(say name of that directory is outdir) In that directory(outdir) the files come after some gap of time by some processes. What i want to do is i want to list all the files in that directory after the given... (2 Replies)
Discussion started by: parthmittal2007
2 Replies

5. Shell Programming and Scripting

perl code-sequence of json format

Hi All , Below is the perl code. from below code want to confirm one thing that wahtever the sequence of data we are passing through json format which contains 3 tuples of different sequences Eg: ParentID,SystemID,SendingTime,Time,ClientLocation,ClientID, ... (1 Reply)
Discussion started by: aish11
1 Replies

6. Programming

find the missing sequence in hash perl

Dear Perl's Users, Could anyone help me how to solve my problem. I have data with details below. TTY NAME SEQUENCES U-0 UNIX 0 U-1 UNIX 1 U-2 UNIX 2 <-- From 2 jump to 5 U-5 UNIX 5 U-6 UNIX 6 <-- From 6 jump to 20 U-20 ... (2 Replies)
Discussion started by: askari
2 Replies

7. Shell Programming and Scripting

How to merge 2 files based on delimeter in perl?

Hi, I have 2 files. a.txt & b.txt # a.txt contains the following text. apple grapes # b.txt contains the following text. banana pine My question is. (1 Reply)
Discussion started by: vanitham
1 Replies

8. UNIX for Dummies Questions & Answers

create time sequence in bash

hi all, hope someone can assist in this. I'm trying to make a sequence of time in bash so that the output prints the following; 1950-01-01 1950-02-01 1950-03-01 ... 1999-11-01 1999-12-01 In R, i can issue the following command; t1 <- ISOdate(1950,1,1,0,0,0) t2 <-... (0 Replies)
Discussion started by: Muhammad Rahiz
0 Replies

9. Shell Programming and Scripting

generating a sequence depending on the time of the day

ould we generate a sequence number based on the time of the day? e.g. an application is scheduled to run from 0800 hrs to 1700 hrs with a frequency of Once every five minutes the intention is to "assign" a certain value to a variable when depending on the time of the day i.e. if the time... (6 Replies)
Discussion started by: zainravi
6 Replies

10. Shell Programming and Scripting

Merge two files in windows using perl script

Hi I want to merge two or more files using perl in windows only(Just like Paste command in Unix script) . How can i do this.Is ther any single command to do this? Thanks Kunal (1 Reply)
Discussion started by: kunal_dixit
1 Replies
Login or Register to Ask a Question