Read multiple files, parse data and append to a file


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Read multiple files, parse data and append to a file
# 8  
Old 09-27-2012
Code:
#! /bin/perl

open(IDX,"<Index") || die "Couldn't open index file";
@index=<IDX>;
close IDX;
$header{POSITION}="Position";
for($i=0;$i<$#ARGV+1;$i++)
{
$FH="F${i}";
open($FH,"< $ARGV[${i}]") || die "couldn't open $ARGV[${i}]";
$header{POSITION}="$header{POSITION} $ARGV[${i}]";
while(<$FH>){
chomp($_);
@array=split(/  */);
foreach $index (@index){
chomp($index);
if($index==$array[1]){
if((!$hash{$array[1]} || ($count{$array[1]}++ !=$i)) && $i){
$dem=($i-$count{$array[1]});
$tail="    "x$dem;
$hash{$array[1]}="$hash{$array[1]} $tail";
$count{$array[1]}++;
}
$hash{$index}="$hash{$index} $array[3] $array[5]";
#$count{$array[1]}++;
next;
}
}
}
close $FH;
}
print $header{POSITION}."\n";
foreach $item (keys %hash){
if($hash{$item}){
print "$item $hash{$item}"."\n";
}
}


A dying attempt from this perl greenhorn...Smilie
# 9  
Old 09-27-2012
Quote:
Originally Posted by empyrean
@Don Cragun... hank you so much.. It just worked perfect. Exactly the way i wanted. As you said, perfect alignment too.. thank you again

Only one small thing is that its not printing the filenames instead its printing File1, File2 like that. Is that an easy fix?
I printed the cross reference to the filename as they were encountered since I have no idea whether your *.csv files will fit in the 16 characters used to print each file's reference and alternate values. If your replace
Code:
FNR==1 {# Print the file key entry for another file and update the header lines.
        printf("File%03d is %s\n", ++fc, FILENAME)
        hdr1 = sprintf("%s\t --File%03d--", hdr1, fc)
        hdr2 = sprintf("%s\t Ref\t Alt", hdr2)
}

in my script with:
Code:
FNR==1 {# Update the header lines and the file count..
        ++fc
        if((dw = (14 - length(FILENAME)) / 2) > 0)
                ds = sprintf("%*.*s", dw, dw, "--------")
        else    ds = ""
        hdr1 = sprintf("%s\t%s%s%s", hdr1, ds, FILENAME, ds)
        hdr2 = sprintf("%s\t Ref\t Alt", hdr2)
}

it should work as long as none of your *.csv files have names with more than 15 characters.

I could use variable length output fields based on the filename lengths, but if you want me to make modifications that extensive, you'll need to hire me to do the work. Smilie
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Programming

Python script to run multiple command and append data in output csv file

Experts, I am writing a script and able to write only small piece of code and not able to collect logic to complete this task. In input file have to look for name like like this (BGL_HSR_901_1AG_A_CR9KTR10) before sh iss neors. Record this (BGL_HSR_901_1AG_A_CR9KTR10) in csv file Now have to... (0 Replies)
Discussion started by: as7951
0 Replies

2. UNIX for Beginners Questions & Answers

UNIX script to append multiple text files into one file based on pattern present in filaname

Hi All-I am new to Unix , I need to write a script. Can someone help me with a requirement where I have list of files in a directory, I want to Merge the files if a pattern of string matches in filenames? AAAL_555A_ORANGE1_F190404.TXT AAAL_555A_ORANGE2_F190404.TXT AAAL_555A_ORANGE3_F190404.TXT... (6 Replies)
Discussion started by: Shankar455
6 Replies

3. Shell Programming and Scripting

In PErl script: need to read the data one file and generate multiple files based on the data

We have the data looks like below in a log file. I want to generat files based on the string between two hash(#) symbol like below Source: #ext1#test1.tale2 drop #ext1#test11.tale21 drop #ext1#test123.tale21 drop #ext2#test1.tale21 drop #ext2#test12.tale21 drop #ext3#test11.tale21 drop... (5 Replies)
Discussion started by: Sanjeev G
5 Replies

4. Shell Programming and Scripting

Read multiple text files and copy data to csv

hi i need to extract lines from multiple files to a csv file. for example, i have these 3 files file1.txt date:29dec1980 caller:91245824255 called:8127766 file2.txt date:11apr2014 caller:9155584558 called:8115478 file3.txt date:25jun2015 caller:445225552 called:8117485 (30 Replies)
Discussion started by: lp.descamps
30 Replies

5. Shell Programming and Scripting

Append data by looking up 2 tables for multiple files

I want to lookup values from two different tables based on common columns and append. The trick is the column to be looked up is not fixed and varies , so it has to be detected from the header. How can I achieve this at once, for multiple data files, but lookup tables fixed. The two lookup... (5 Replies)
Discussion started by: ritakadm
5 Replies

6. Shell Programming and Scripting

Append Multiple files with file name in the beginning of line

Hi, I have multiple files having many lines like as bvelow: file Name a.txt abc def def xyz 123 5678 file Name b.txt abc def def xyz 123 5678 I would like to append files in the below format to a new file: file Name c.txt (7 Replies)
Discussion started by: rramkrishnas
7 Replies

7. Shell Programming and Scripting

awk : Filter a set of data to parse header line and last field of multiple same match.

Hi Experts, I have a data with multiple entry , I want to filter PKG= & the last column "00060110" or "00088150" in the output file: ############################################################################################### PKG= P8SDB :: VGS = vgP8SOra vgP8SDB1 vgP8S001... (5 Replies)
Discussion started by: rveri
5 Replies

8. Programming

C program to read n files and append it to a variable.

I am struck in the code of handling multiple files reading and appending it into a variable. My First file's entire content I have concatenated using strcat(<char pointer>,char array) In the Second file I extract using strtstr() function and below is the code. if( fp2 != NULL){ ... (2 Replies)
Discussion started by: gameboy87
2 Replies

9. Shell Programming and Scripting

How to read and append certain files form directory

Hi ,i have a question ,if I am in sertain directory and I have 4 files *.c how can I read and append this files in file backup.bac Thanks find ./ -name "*.csh" | wc -l (2 Replies)
Discussion started by: lio123
2 Replies

10. Shell Programming and Scripting

Read the data from multiple files and sum the value

Hi all, I have a requirement where i have to read multiple files using Shell Script in Korn Shell. each file will have the 3rd line as the amount field, i have to read this amount field and sum it for all the files. any idea on how to achieve this?? (i think i can achieve it using a loop,... (9 Replies)
Discussion started by: nvuradi
9 Replies
Login or Register to Ask a Question