Sponsored Content
Top Forums Shell Programming and Scripting Optimize/speed-up perl extraction Post 302130261 by porter on Saturday 4th of August 2007 08:10:04 AM
Old 08-04-2007
Quote:
Originally Posted by pinpe
Imagine how huge this is.
We don't need to, you have just told us. Smilie

Quote:
Originally Posted by pinpe
How I can optimized the data extraction using perl.
How are you determining what you extract?

How do you extract it?

The fastest way would be to use a C program that can read each line into a single buffer, do the determination without any memory allocation/deallocation, then print the required sections again without memory allocation/deallocation.
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

metapattern extraction in PERL

Hi, I want to extract some part of a pattern that matches my requirement in a string with PERL. A case in point is a string like: $eqtst="abh nmae res = 10 s abh nmae req = 10 s"; from which I want the words preceding the "=" symbol. Previously I was assured that there would be only 2 such... (4 Replies)
Discussion started by: Abhishek Ghose
4 Replies

2. Shell Programming and Scripting

String Extraction in Perl

I have a string stored in a variable. For instance, $str = " Opcode called is : CM_OP_xxx " where xxx changes dynamically and can be either LOGIN or SEARCH..... depends on runtime. For example : $str = " Opcode called is : CM_OP_SEARCH " $str = " Opcode called is : CM_OP_LOGIN " I... (3 Replies)
Discussion started by: vkca
3 Replies

3. Shell Programming and Scripting

Perl function extraction

The log file reads as follows. D function_add() ADD input data 1021214 0 VAR1 STR 10 0 VAR2 STR 20 0 VAR3 STR 1 SUM=VAR1+VAR2 D function_add() ADD output data 1021267 0 DISPLAY SUM D function_sub() SUB input data 1021214 0 VAR1 STR 10 0 VAR2 STR 20 0 VAR3 STR 1 sub=VAR1-VAR2 D... (2 Replies)
Discussion started by: vkca
2 Replies

4. Shell Programming and Scripting

Function extraction in PERL

the log contains mathematical operation as follows fm_void_mathematics : PCM_OP_MATHS input function PIN_FLD_NUM1 INT 1 PIN_FLD_NUM2 INT 2 PIN_FLD_RESULTS int PIN_FLD_OUT INT * D Wed Sep 16 05:40:22 2009 solaris_testing fm_void_add : PIN_FLD_SUM int 3 D Wed Sep 16 05:40:22 2009... (1 Reply)
Discussion started by: vkca
1 Replies

5. Shell Programming and Scripting

Slow Perl script: how to speed up?

I had written a perl script to compare two files: new and master and get the output of the first file i.e. the first file: words that are not in the master file STRUCTURE OF THE TWO FILES The first file is a series of names ramesh sushil jonga sudesh lugdi whereas the second file (could be... (4 Replies)
Discussion started by: gimley
4 Replies

6. Shell Programming and Scripting

Data extraction in perl variable

HI, i have variable in perl like below $tmp="/home/sai/automation/work/TFP_GN.txt" it can conatain any path its filled from config file. now i want extarct the path upto this /home/sai/automation/work/ and put it in another variable say... (4 Replies)
Discussion started by: raghavendra.nsn
4 Replies

7. Shell Programming and Scripting

Data Extraction problem in perl

Hello, I want to extract the words from a file which starts with SRD-R or SRD-DR. I have written a script which is able to trace the word but it is printing the whole line. sub extract_SRD_tag{ my ($tag, $app, $path, @data, $word ); $path = shift; $app = shift; open (FILE, $path) or... (2 Replies)
Discussion started by: suvendu4urs
2 Replies

8. Shell Programming and Scripting

perl extraction of code between comments /* .. */

Hi, I am using the following code to retrieve the contents between C-style comments "/* .. */". perl -lne 'while(/(\/\*.*?\*\/)/g) {print "$1";}' This works fine when the commented section of code is present in a single line. But I also need to extract the data which is present inside... (3 Replies)
Discussion started by: royalibrahim
3 Replies

9. Shell Programming and Scripting

Perl-extraction using windows cmd

Hi, I need to extract Password expires from the output of windows command print `net user %USERNAME% /domain`; in perl. So i want to redirect the output of this win-cmd to a file and try extracting Password expires along with its value. i'm trying with this code but getting errors. #!usr/bin/perl... (1 Reply)
Discussion started by: sam_bd
1 Replies

10. Shell Programming and Scripting

Speed up extraction od tar.bz2 files using bash

The below bash will untar each tar.bz2 folder in the directory, then remove the tar.bz2. Each of the tar.bz2 folders ranges from 40-75GB and currently takes ~2 hours to extract. Is there a way to speed up the extraction process? I am using a xeon processor with 12 cores. Thank you :). ... (7 Replies)
Discussion started by: cmccabe
7 Replies
SDR2FILE(1)							  ICI executables						       SDR2FILE(1)

NAME
sdr2file - SDR data extraction test program SYNOPSIS
sdr2file configFlags DESCRIPTION
sdr2file stress-tests SDR data extraction by retrieving and deleting all text file lines inserted into a test SDR data store named "testsdrconfigFlags" by the complementary test program file2sdr(1). The operation of sdr2file echoes the cyclical operation of file2sdr: each linked list created by file2sdr is used to create in the current working directory a copy of file2sdr's original source text file. The name of each file written by sdr2file is file_copy_cycleNbr, where cycleNbr identifies the linked list from which the file's text lines were obtained. sdr2file may catch up with the data ingestion activity of file2sdr, in which case it blocks (taking the file2sdr test semaphore) until the linked list it is currently draining is no longer empty. EXIT STATUS
0 sdr2file has terminated. FILES
No configuration files are needed. ENVIRONMENT
No environment variables apply. DIAGNOSTICS
Can't use sdr. ION system error. Check for diagnostics in the ION log file ion.log. Can't create semaphore. ION system error. Check for diagnostics in the ION log file ion.log. SDR transaction failed. ION system error. Check for diagnostics in the ION log file ion.log. Can't open output file Operating system error. Check errtext, correct problem, and rerun. can't write to output file Operating system error. Check errtext, correct problem, and rerun. BUGS
Report bugs to <ion-bugs@korgano.eecs.ohiou.edu> SEE ALSO
file2sdr(1), sdr(3) perl v5.14.2 2012-05-25 SDR2FILE(1)
All times are GMT -4. The time now is 09:09 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy