The UNIX and Linux Forums  


Go Back   The UNIX and Linux Forums > Top Forums > Shell Programming and Scripting
.
google unix.com



Shell Programming and Scripting Post questions about KSH, CSH, SH, BASH, PERL, PHP, SED, AWK and OTHER shell scripts and shell scripting languages here.

More UNIX and Linux Forum Topics You Might Find Helpful
Thread Thread Starter Forum Replies Last Post
generating a sequence depending on the time of the day zainravi Shell Programming and Scripting 6 04-17-2009 10:22 PM
Merge files of differrent size with one field common in both files using awk shashi1982 Shell Programming and Scripting 2 03-03-2009 07:12 AM
Rename Files in sequence ChicagoBlues UNIX for Dummies Questions & Answers 3 09-30-2008 01:26 PM
Merge two files in windows using perl script kunal_dixit Shell Programming and Scripting 1 07-28-2008 02:39 PM
Delete all files up to a sequence kamathg Shell Programming and Scripting 10 04-07-2008 09:42 AM

Reply
English Japanese Spanish French German Portuguese Italian Dutch Swedish Russian Norwegian Hungarian Hebrew Danish Bulgarian Greek Powered by Powered by Google
 
LinkBack Thread Tools Search this Thread Rate Thread Display Modes
  #1 (permalink)  
Old 06-12-2009
jimmy_y jimmy_y is offline
Registered User
  
 

Join Date: Jun 2009
Posts: 94
Lightbulb perl merge two files by the time sequence

Hi Guys,
i have two files:
fileA:
20090611 00:00:11
20090611 00:00:11
20090611 00:00:24
20090611 00:01:10
20090611 07:13:00

fileB:
20090611 00:00:01
20090611 00:00:12
20090611 00:00:24
20090611 00:01:12
20090611 09:13:00

want to make two files into a single file, but follow the time sequence, the output fileC:
20090611 00:00:01
20090611 00:00:11
20090611 00:00:11
20090611 00:00:12
20090611 00:00:24
20090611 00:00:24
20090611 00:01:10
20090611 00:01:12
20090611 07:13:00
20090611 09:13:00

instead of one by one compare the time, any simple way to doing it?

Thanks
  #2 (permalink)  
Old 06-12-2009
durden_tyler's Avatar
durden_tyler durden_tyler is offline Forum Advisor  
Registered User
  
 

Join Date: Apr 2009
Posts: 547
Quote:
Originally Posted by jimmy_y View Post
...
i have two files:
fileA:
20090611 00:00:11
20090611 00:00:11
20090611 00:00:24
20090611 00:01:10
20090611 07:13:00

fileB:
20090611 00:00:01
20090611 00:00:12
20090611 00:00:24
20090611 00:01:12
20090611 09:13:00

want to make two files into a single file, but follow the time sequence, the output fileC:
20090611 00:00:01
20090611 00:00:11
20090611 00:00:11
20090611 00:00:12
20090611 00:00:24
20090611 00:00:24
20090611 00:01:10
20090611 00:01:12
20090611 07:13:00
20090611 09:13:00

instead of one by one compare the time, any simple way to doing it?

...
You sure you want to use Perl to do that ?

One simple way could be:


Code:
$
$ sort filea fileb
20090611 00:00:01
20090611 00:00:11
20090611 00:00:11
20090611 00:00:12
20090611 00:00:24
20090611 00:00:24
20090611 00:01:10
20090611 00:01:12
20090611 07:13:00
20090611 09:13:00
$

Using perl -


Code:
$
$ perl -e '{open(F1,"filea"); @a1=<F1>; close(F1); open(F2,"fileb"); @a2=<F2>; close(F2); print sort(@a1,@a2)}'
20090611 00:00:01
20090611 00:00:11
20090611 00:00:11
20090611 00:00:12
20090611 00:00:24
20090611 00:00:24
20090611 00:01:10
20090611 00:01:12
20090611 07:13:00
20090611 09:13:00
$
$

tyler_durden
  #3 (permalink)  
Old 06-12-2009
jimmy_y jimmy_y is offline
Registered User
  
 

Join Date: Jun 2009
Posts: 94
Thanks tyler ,

i like the sort so much, thanks for this.
but if my fileA:
4 20090611 00:00:11
5 20090611 00:00:11
3 0090611 00:00:24
3 20090611 00:01:10
10 20090611 07:13:00

fileB:
4 20090611 00:00:01
11 20090611 00:00:12
22 20090611 00:00:24
9 20090611 00:01:12
9 20090611 09:13:00

So for each file, there is another field in the head, so if still want to get the date sort, then seems cannot use 'sort filea fileb'.

-----Post Update-----

Oh, just saw can use "-k" with sort.

-----Post Update-----

Oh, just saw can use "-k" with sort.
  #4 (permalink)  
Old 06-13-2009
jimmy_y jimmy_y is offline
Registered User
  
 

Join Date: Jun 2009
Posts: 94
Hi Guys,

I know how to use the sort, -k to sort two files, but face the problem where need to sort differient fields in differient files.

FileA:
20090610 23:58:59 444
20090610 23:52:13 555
20090611 00:00:02 6

FileB:
1 63660 5807 Wed 10/06/2009 23:54:58
4 63660 5807 Wed 10/06/2009 23:59:58
33 63660 5807 Wed 11/06/2009 23:54:58

###################

So For FileA, field1 and field2 sort date, and for FileB, sort date by field date. As we can see fileA and B date are in differient fields, how to sort these two files?

Thanks
  #5 (permalink)  
Old 06-13-2009
durden_tyler's Avatar
durden_tyler durden_tyler is offline Forum Advisor  
Registered User
  
 

Join Date: Apr 2009
Posts: 547
- What do you want the output to look like ?
- Do you want to sort each file individually ?
- If you want to combine the contents of both files, the data won't align properly as the files have different number of fields.

Post the output that you expect, taking your files as the example.

tyler_durden
  #6 (permalink)  
Old 06-13-2009
jimmy_y jimmy_y is offline
Registered User
  
 

Join Date: Jun 2009
Posts: 94
Hi tyler,

Thanks. the output file will combine these two files sort by the date:
20090610 23:52:13 555
1 63660 5807 Wed 10/06/2009 23:54:58
20090610 23:58:59 444
4 63660 5807 Wed 10/06/2009 23:59:58
20090611 00:00:02 6
33 63660 5807 Wed 11/06/2009 23:54:58
###########
so sort by date. The reason for such two files differient date field and date format different, becuase those two files are from two differient billing.

Thanks.

-----Post Update-----

The output also can be below, means can remove FileB before data characters. or if possible change the fileB data format to fileA data format.
#########################
20090610 23:52:13 555
Wed 10/06/2009 23:54:58
20090610 23:58:59 444
Wed 10/06/2009 23:59:58
20090611 00:00:02 6
Wed 11/06/2009 23:54:58
######################

:--( see cut, paste, join, sort, but no luck.

Last edited by jimmy_y; 06-13-2009 at 03:20 PM..
  #7 (permalink)  
Old 06-13-2009
KevinADC KevinADC is offline Forum Advisor  
Registered User
  
 

Join Date: Jan 2008
Posts: 731
Jumping back to perl... Assuming the last field of FileA and the first field of FileB are not important for the sort...


Code:
use strict;
use warnings;
my @all_lines;
open(my $FA, 'filea.txt') or die "$!";
while(<$FA>) {
   chomp;
   push @all_lines,[join('',(split(/\s+/))[0,1]),$_];
}
close $FA;
open(my $FB, 'fileb.txt') or die "$!";
while(<$FB>) {
   chomp;
   my($d,$t) = (split(/\s+/))[4,5];
   my $d2 = join('',(split(/\//,$d))[2,1,0]);
   push @all_lines,["$d2$t",$_];
}
close $FB;
my @sorted = sort{$a->[0] cmp $b->[0]} @all_lines;  
print "$_->[1]\n" for @sorted;

Uses a 2D array to store the sort keys (YYYYMMDDHH::MM::SS) and the original lines. Then sorts the sort keys but returns the original lines.

If these are really big files you might consider using a database instead of a scripting langugae to store the data then you can do all sorts of wonderful things with it more efficiently and maybe easier in the long run.
Bits Awarded / Charged to KevinADC for this Post
Date User Comment Amount
06-13-2009 jimmy_y Very nice script, so simple. 1,000
06-13-2009 Neo Great Job! 20,000
Reply

Bookmarks

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes Rate This Thread
Rate This Thread:

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On




All times are GMT -4. The time now is 03:30 PM.


Powered by: vBulletin, Copyright ©2000 - 2006, Jelsoft Enterprises Limited. Language Translations Powered by .
vBCredits v1.4 Copyright ©2007 - 2008, PixelFX Studios
The UNIX and Linux Forums Content Copyright ©1993-2009. All Rights Reserved.Ad Management by RedTyger

Content Relevant URLs by vBSEO 3.2.0