Sponsored Content
Top Forums Shell Programming and Scripting Efficiently altering and merging files in perl Post 302905511 by sam05121988 on Thursday 12th of June 2014 03:06:51 AM
Old 06-12-2014
Lightbulb Efficiently altering and merging files in perl

I have two files

Code:
fileA
HEADER LINE A
CommentLine A
Content A
....
....
....
TAILER A

Code:
fileB
HEADER LINE B
CommentLine B
Content B
....
....
....
TAILER B

I want to merge these two files as
Code:
HEADER LINE A
CommentLine A
Content A
....
....
....
Content B
....
....
....
TAILER B

i.e. skip the TAILER line of file A and skip the HEADER and Comment Line of fileB

I am able to do it using the below perl code
Code:
        open ( FA, "$fileA" ) || die("can't open fileA $!");
        open ( FB, "$fileB" ) || die("can't open fileB $!");
        open ( TMP, ">> tmp_file" ) || die("can't open tmp_file $!");

        #reading both files in array
        my @fileA = <FA>;
        my @fileB = <FB>;

        #getting rid of HEADER, Comment line, in fileB
        shift @fileB;
        shift @fileB;

        #getting rid of TAILER in fileA
        pop @fileA;

        my @tmp_file=(@fileA,@fileB);

        foreach ( @tmp_file ){
            print TMP $_;
        }

        close(FA);
        close(FB);
        close(TMP);
        
        rename tmp_file, fileA || die("can't rename tmp_file to fileA);

This code works fine, however I doubt it's efficiency if fileA and fileB are going to be millions of lines (which is the case)
i.e. why read whole file in arrays just to get rid of three lines (will end up using lots of memory)

Can someone suggest a more efficient way of doing this
(answers in perl only)

Last edited by sam05121988; 06-12-2014 at 04:15 AM..
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Issue altering end data

I have an inventory program that I would like to have the ability to go and change or alter the field data based on the item number as a key. I have the menu option set but at the end of the script process it just appends the changed data to the database rather than what I would like; which is to... (5 Replies)
Discussion started by: stlitguru
5 Replies

2. UNIX Desktop Questions & Answers

how to search files efficiently using patterns

hi friens, :) if i need to find files with extension .c++,.C++,.cpp,.Cpp,.CPp,.cPP,.CpP,.cpP,.c,.C wat is the pattern for finding them :confused: (2 Replies)
Discussion started by: arunsubbhian
2 Replies

3. Shell Programming and Scripting

altering numbers in files

I want to change a number in a file into number -1.. for instance file_input is fdisdlf_s35 fdjsk_s27 fsdf_s42 jkljllljkkl_s57 ... etc now i want the output to be fdisdlf_s34 fdjsk_s26 fdsf_s41 jkljllljkkl_s56 ... etc I was think of using "sed -e 's/2/1/g' -e 's/3/2/g' -e... (4 Replies)
Discussion started by: bigboizvince
4 Replies

4. Shell Programming and Scripting

Scripting question: Altering 2 field.

Hi Experts, I want to alter two filed of my data file: The _new should come to 2nd column, and _new to be removed from 4rth column, please advise, datafile.txt aa /dev/vgAA/lvol1 bb /dev/vgAA_new/lvol1 aa /dev/vgAA1/lvol2 bb /dev/vgAA1_new/lvol2 aa /dev/vgAC/lvol1 bb... (5 Replies)
Discussion started by: rveri
5 Replies

5. Shell Programming and Scripting

perl : merging two arrays on basis of common parameter

I have 2 arrays, @array1 contains records in the format 1|_|X|_|ssd|_| 4|_|H|_|hbd|_| 9|_|Y|_|u8gjdfg|_| @array2 contains records in the format X|_|asdf|_| Y|_|qwer|_| A|_|9kdkf|_| @array3 should contain records in the PLz X|_|ssd|_|asdf|_| Y|_|hdb|_|qwer|_| PLZ dont use... (2 Replies)
Discussion started by: centurion_13
2 Replies

6. Shell Programming and Scripting

Algorithm to load files efficiently without missing or accidently archiving....

We have a requirement where we get the Delta Files in every one hour and we need to load them into Oracle database every one hour using Powercenter. To efficiently do this we need to build an File management system. Here is our process: we get 6 files for 6 tables with a timestamp appended... (2 Replies)
Discussion started by: okkadu
2 Replies

7. Shell Programming and Scripting

merging two files

file1.txt 1 2 10 11 56 57 7 8 43 44 and let's suppose that there is a file called file2.txt with 100 columns I want to produce a file3.txt with columns specified in file1.txt in that order (1,2,10,11,56,57,7,8,43,44) Thanks! (2 Replies)
Discussion started by: johnkim0806
2 Replies

8. Shell Programming and Scripting

Perl - multiple keys and merging two files

Hi, I'm not a regular coder but some times I write some basic perl script, hence Perl is bit difficult for me :). I'm merging two files a.txt and b.txt into c.txt: a.txt ------ x001;frtb70;xyz;109 x001;frvt65;sec;239 x003;wqax34;jul;659 x004;yhud43;yhn;760 b.txt ------... (8 Replies)
Discussion started by: Lokesha
8 Replies

9. Shell Programming and Scripting

Altering a variable

Can I take an argument input, lets say it's, hg0000_xy1_v2, in the script it becomes f ... then hack off the end of the filename to change the variable to hg0000 only. I tried using sed but can't figure it out. f="$f" | sed 's/_fg_v//' I could change the variable label if necessary to... (4 Replies)
Discussion started by: scribling
4 Replies

10. Programming

Altering a jar file

I have a script I am trying to test and run but it runs against a jar file. I wrote an external property file so it would redirect with my script, but it keeps going in search of the previous property file. Is there any way to externally over write the jar file and if not how do you go about... (7 Replies)
Discussion started by: risarose87
7 Replies
Net::SIP::Packet(3pm)					User Contributed Perl Documentation				     Net::SIP::Packet(3pm)

NAME
Net::SIP::Packet - handling of SIP packets SYNOPSIS
use Net::SIP::Packet; my $pkt = eval { Net::SIP::Packet->new( $sip_string ) } or die "invalid SIP packet"; $pkt->get_header( 'call-id' ) || die "no call-id"; $pkt->set_header( via => @via ); print $pkt->as_string; DESCRIPTION
This module implements the parsing, manipulation and creation of SIP packets according to RFC3261. NET::SIP::Packet's objects can be created by parsing a string containing the SIP packet or by constructing it from parts, e.g. header keys and values, body, method+URI (requests) or code+text (responses). All parts can be manipulated and finally the string representation of the manipulated packet can be (re)created. For dealing with requests and responses directly usually the subclasses Net::SIP::Request or Net::SIP::Response will be used instead. EXAMPLES
# create packet from string my $invite = Net::SIP::Packet->new( <<'EOS' ); INVITE sip:you@example.com SIP/2.0 From: <sip:me@example.com> To: <sip:you@example.com> ... EOS # show and manipulate some header print "callid=".$invite->get_header( 'call-id' )." "; print "route=".join( ",", $invite->get_header( 'route' ))." "; $invite->set_header( 'via' => [ $via1,$via2,.. ] ); # get resulting string representation print $invite->as_string; # create packet from parts my $resp = Net::SIP::Packet->new( 200, 'Ok', { to => '<sip:you@example.com>', from => '<sip:me@example.com>',.. } Net::SIP::SDP->new(...) ); # and get the packet as string print $resp->as_string; CONSTRUCTOR
new ( STRING | @PARTS ) This is the default constructor. Depending on the number of arguments branches into new_from_string or new_from_parts. new_from_string ( STRING ) Interprets STRING as a SIP request or response and creates Net::SIP::Request or Net::SIP::Response object accordingly. Will die() if it cannot parse the string as a SIP packet. new_from_parts ( CODE|METHOD, TEXT|URI, \%HEADER|@HEADER, [ BODY ] ) If CODE|METHOD is numeric a Net::SIP::Response object will be created with the response code CODE and the text TEXT. Otherwise a Net::SIP::Request object will be created with the method METHOD and the uri URI. Header data can be given as a hash %HEADER or array @HEADER reference. In case of a hash the key is the SIP field name and the value as either a string or a @list of strings. The fields on the resulting SIP packet will be sorted by name of the fields and fields with multiple values will be created as seperat lines. If the header is given as an array the elements of the array are "[ key => value ]" pairs where the keys are the field names and the values are strings or @list of strings. Each pair will result in a single line in the SIP header. If the value was a list reference the values in the list will be concatened by ','. The order of the fields in the resulting SIP packet will be the same as in the array. The BODY is optional and can be given either as a string or as an reference to an object which has a method as_string, like Net::SIP::SDP. If the BODY is an object which has a method content_type it will set the "content-type" header of the SIP object based on the result of "BODY->content_type" unless a "content-type" header was explicitly given. METHODS
is_request Returns TRUE if the SIP packet is a request, otherwise FALSE. is_response Returns TRUE if the SIP packet is a response, otherwise FALSE. tid Returns a transaction ID created from the sequence number in the "CSeq" header and the "Call-Id" header. All packets with the same tid belong to the same transaction. cseq Returns "CSeq" header. Short for "$self->get_header( 'cseq' )". callid Returns "Call-Id" header. Short for "$self->get_header( 'call-id' )". get_header ( [ NAME ] ) If NAME is given it returns the SIP header for NAME. If no header exists returns (). If there is only one value for the header returns this value. In case of multiple values it returns a @list of all values, but if "wantarray" says, that the caller expects only a single value it will "croak()". If no NAME is given it will return a reference to a hash wich contains all fields and has the format described in new_from_parts. add_header ( NAME, VAL ) Adds the header at the end of the SIP header. VAL can be a string or a reference to a list of strings. insert_header ( NAME, VAL ) Like add_header, but the lines will be added on top of the header. del_header ( NAME ) Delete all lines from header where the field name is NAME. set_header ( NAME, VAL ) Replaces an existing header, like del_header followed by add_header. set_body ( VAL ) Sets body to VAL, which can be string or object. The handling for body objects see new_from_parts. as_string Returns string representation of SIP packet. dump ( [ LEVEL ] ) Returns dump of packet as string for debugging. The higher LEVEL is the more details one gets. At the moment a LEVEL of 0 gets a one- line summary and the rest the result from as_string. as_parts Returns Array with CODE|METHOD, TEXT|URI, @HEADER and BODY like used in new_from_parts. sdp_body Returns body as Net::SIP::SDP object if there is a body and the content-type is 'application/sdp' or empty. If body contains invalid SDP it raises an exception (e.g. die()). UNDOCUMENTED METHODS get_header_hashval ( [ NAME ] ) scan_header ( @ARG ) clone perl v5.14.2 2011-08-26 Net::SIP::Packet(3pm)
All times are GMT -4. The time now is 10:02 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy