Sponsored Content
Top Forums Shell Programming and Scripting Perl : Large amount of data put into an array Post 302817289 by sumguy on Wednesday 5th of June 2013 01:53:58 PM
Old 06-05-2013
Perl : Large amount of data put into an array

This basic code works.

I have a very long list, almost 10000 lines that I am building into the array. Each line has either 2 or 3 fields as shown in the code snippit. The array elements are static (for a few reasons that out of scope of this question) the list has to be "built in".

It runs very well in the Nix environment but slows down in the Windows environment.

Question is what can I do to speed things up ?


Code:
@MYARRAY = ("field1 field2 field3", "filed1 field2" ........ 10000 lines);
foreach $eachline (@MYARRAY) {
   if ($eachline =~ /\b$ARGV[0]\b/) {
     print "$eachline\n";
   } 
}

 

10 More Discussions You Might Find Interesting

1. Linux

shmat() Failure While Using a Large Amount of Shared Memory

Hi, I'm developing a data processing pipeline with multiple stages, with data being moved between the stages using shared memory segments. The size of the data is typically of the order of hundreds of megabytes, and there are typically a few tens of main shared memory segments each of size... (2 Replies)
Discussion started by: theicarusagenda
2 Replies

2. Programming

Read/Write a fairly large amount of data to a file as fast as possible

Hi, I'm trying to figure out the best solution to the following problem, and I'm not yet that much experienced like you. :-) Basically I have to read a fairly large file, composed of "messages" , in order to display all of them through an user interface (made with QT). The messages that... (3 Replies)
Discussion started by: emitrax
3 Replies

3. AIX

amount of memory allocated to large page

We just set up a system to use large pages. I want to know if there is a command to see how much of the memory is being used for large pages. For example if we have a system with 8GB of RAm assigned and it has been set to use 4GB for large pages is there a command to show that 4GB of the *GB is... (1 Reply)
Discussion started by: daveisme
1 Replies

4. Shell Programming and Scripting

Perl Array / pattern match large CPU usage

Hi, I have one file in this format 20 value1 33 value2 56 value3 I have another file in this format: 34,30-SEP-09,57,100237775,33614510126,2,34 34,30-SEP-09,57,100237775,33620766654,2,34 34,30-SEP-09,108,100237775,33628458122,2,34 34,30-SEP-09,130,100237775,33635266741,2,254... (6 Replies)
Discussion started by: Donkey25
6 Replies

5. Shell Programming and Scripting

perl, put one array into many array when field is equal to sth

Hi Everyone, #!/usr/bin/perl use strict; use warnings; my @test=("a;b;qqq;c;d","a;b;ggg;c;d","a;b;qqq;c;d"); would like to split the @test array into two array: @test1=(("a;b;qqq;c;d","a;b;qqq;c;d"); and @test2=("a;b;ggg;c;d"); means search for 3rd filed. Thanks find the... (0 Replies)
Discussion started by: jimmy_y
0 Replies

6. Shell Programming and Scripting

How to tar large amount of files?

Hello I have the following files VOICE_hhhh SUBSCR_llll DEL_kkkk Consider that there are 1000 VOICE files+1000 SUBSCR files+1000DEL files When i try to tar these files using tar -cvf backup.tar VOICE* SUBSCR* DEL* i get the error: ksh: /usr/bin/tar: arg list too long How can i... (9 Replies)
Discussion started by: chriss_58
9 Replies

7. Shell Programming and Scripting

storing large data in unix array variable

Hi, I have table in sql ..from this table im storing the first coloumn values in shell array variable ... after this passing this variable as an arugument in SQL procedure. But the proc. is running fine only for 1024 values in array ... How to store more than 1024 values in the array... (5 Replies)
Discussion started by: ankitknit
5 Replies

8. Shell Programming and Scripting

How do I put data piped to my script into an array for repeated processing

Hi folks, Sorry for something I'm sure was answered already, I just could not find it in a search of the forums. I am trying to build a script that eats a config file as: cat file.cnf | ConProcess.shWhat I want to do inside the script is: !#/usr/bin/bash # captured piped cat into an... (6 Replies)
Discussion started by: Marc G
6 Replies

9. Shell Programming and Scripting

Read file and get the data then put it in array

Hi, I have a file called "readfile" it contains below parameters #cat readfile word=/abc=225,/abc/cba=150 three=12 four=45 five=/xyz/yza likewise multiple line. From the above file, I have to read "word" output should be like, /abc /abc/cba these values need to be put in... (3 Replies)
Discussion started by: munna_dude
3 Replies

10. Shell Programming and Scripting

How to make awk command faster for large amount of data?

I have nginx web server logs with all requests that were made and I'm filtering them by date and time. Each line has the following structure: 127.0.0.1 - xyz.com GET 123.ts HTTP/1.1 (200) 0.000 s 3182 CoreMedia/1.0.0.15F79 (iPhone; U; CPU OS 11_4 like Mac OS X; pt_br) These text files are... (21 Replies)
Discussion started by: brenoasrm
21 Replies
Net::DNS::Question(3)					User Contributed Perl Documentation				     Net::DNS::Question(3)

NAME
Net::DNS::Question - DNS question record SYNOPSIS
use Net::DNS::Question; $question = new Net::DNS::Question('example.com', 'A', 'IN'); DESCRIPTION
A Net::DNS::Question object represents a record in the question section of a DNS packet. METHODS
new $question = new Net::DNS::Question('example.com', 'A', 'IN'); $question = new Net::DNS::Question('example.com'); $question = new Net::DNS::Question('192.0.32.10', 'PTR', 'IN'); $question = new Net::DNS::Question('192.0.32.10'); Creates a question object from the domain, type, and class passed as arguments. One or both type and class arguments may be omitted and will assume the default values shown above. RFC4291 and RFC4632 IP address/prefix notation is supported for queries in both in-addr.arpa and ip6.arpa namespaces. decode $question = decode Net::DNS::Question($data, $offset); ($question, $offset) = decode Net::DNS::Question($data, $offset); Decodes the question record at the specified location within a DNS wire-format packet. The first argument is a reference to the buffer containing the packet data. The second argument is the offset of the start of the question record. Returns a Net::DNS::Question object and the offset of the next location in the packet. An exception is raised if the object cannot be created (e.g., corrupt or insufficient data). encode $data = $question->encode( $offset, $hash ); Returns the Net::DNS::Question in binary format suitable for inclusion in a DNS packet buffer. The optional arguments are the offset within the packet data where the Net::DNS::Question is to be stored and a reference to a hash table used to index compressed names within the packet. name $name = $question->name; Internationalised domain name corresponding to the qname attribute. Decoding non-ASCII domain names is computationally expensive and undesirable for names which are likely to be used to construct further queries. When required to communicate with humans, the 'proper' domain name should be extracted from a query or reply packet. $query = new Net::DNS::Packet( $example, 'ANY' ); $reply = $resolver->send($query) or die; ($question) = $reply->question; $name = $question->name; qname, zname $qname = $question->qname; $zname = $question->zname; Canonical ASCII domain name as required for the query subject transmitted to a nameserver. In dynamic update packets, this attribute is known as zname() and refers to the zone name. qtype, ztype $qtype = $question->qtype; $ztype = $question->ztype; Returns the question type attribute. In dynamic update packets, this attribute is known as ztype() and refers to the zone type. qclass, zclass $qclass = $question->qclass; $zclass = $question->zclass; Returns the question class attribute. In dynamic update packets, this attribute is known as zclass() and refers to the zone class. print $object->print; Prints the record to the standard output. Calls the string() method to get the string representation. string print "string = ", $question->string, " "; Returns a string representation of the question record. COPYRIGHT
Copyright (c)1997-2002 Michael Fuhr. Portions Copyright (c)2002-2004 Chris Reinhardt. Portions Copyright (c)2003,2006-2011 Dick Franks. All rights reserved. This program is free software; you may redistribute it and/or modify it under the same terms as Perl itself. SEE ALSO
perl, Net::DNS, Net::DNS::DomainName, Net::DNS::Packet, RFC 1035 Section 4.1.2 perl v5.18.2 2014-01-16 Net::DNS::Question(3)
All times are GMT -4. The time now is 03:38 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy