![]() |
Hello and Welcome from United States to the UNIX and Linux Forums! Thank You for Visiting and Joining Our Global Community.
|
|
google unix.com
|
|||||||
| Forums | Register | Forum Rules | Links | Albums | FAQ | Members List | Calendar | Search | Today's Posts | Mark Forums Read |
| Shell Programming and Scripting Post questions about KSH, CSH, SH, BASH, PERL, PHP, SED, AWK and OTHER shell scripts and shell scripting languages here. |
More UNIX and Linux Forum Topics You Might Find Helpful
|
||||
| Thread | Thread Starter | Forum | Replies | Last Post |
| Unix Arithmatic operation issue , datatype issue | thambi | Shell Programming and Scripting | 23 | 02-19-2008 07:19 AM |
| Unix command mmin issue | Mohee | UNIX for Dummies Questions & Answers | 1 | 08-30-2007 11:41 PM |
| cp command issue. | sai_nj | UNIX for Dummies Questions & Answers | 4 | 06-12-2007 01:15 PM |
| Tar command issue | ajayyadavmca | HP-UX | 2 | 05-02-2007 11:42 AM |
| AIX 5.3 - Issue with using at command | vigsgb | AIX | 2 | 08-03-2005 09:43 AM |
![]() |
|
|
LinkBack | Thread Tools | Search this Thread | Rate Thread | Display Modes |
|
|
|
||||
|
Hi Experts,
I am finding the performance of cat command is very wierd, it is taking more time to merge the files into a single file. We have a situation where we would be merging more than 100 files into a single file, but with cat command it is running slow. I tried doing with paste, join and cat, but cat is working faster than any of these. Please guide me if any one has faced similar issue, please help. How about sed and awk commands usage for merging bigger files. Thanks, Rajiv |
|
||||
|
your statements are contradicting each other.
Could you please clarify that ? ![]() Quote:
Quote:
Quote:
|
|
||||
|
Matrixmadan,
We are currently using cat command to combine files, basically appending files together into a single file. cat file1 file2 file3 .....file 100 > final_file I tried creating a file list and ran cat command this way.... cat filelist | xargs cat >> final_file There is no performance gain. The file1, file2 ...file 100 are of varying size, and the final_file is very big, somewhere around 15 GB. I wanted to know is there a better way to combine multiple files into one. Thanks, Rajiv |
|
||||
|
Code:
#! /opt/third-party/bin/perl
use strict;
my @contents;
my @files_arr;
my $DIR1 = "/source/directory/";
my $big = "thebigfile.txt";
opendir(DIR, $DIR1) || die "Unable to open files in Dir : $DIR1 <$!> \n";
@files_arr = readdir(DIR);
closedir(DIR);
open(BIG, ">", $big) or die "Unable to open file $big <$!> \n";
foreach (@files_arr) {
if( ! /^\./ && ! /^\.\./ ) {
print "$_\n";
$_ = $DIR1 . $_;
open(FILE, "<", $_) or die "Unable to open file $_ <$!> \n";
@contents = <FILE>;
close(FILE);
print BIG "@contents";
}
}
close(BIG);
exit 0
|
![]() |
| Bookmarks |
| Thread Tools | Search this Thread |
| Display Modes | Rate This Thread |
|
|