Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Better way to run this perl command

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Better way to run this perl command
# 8  
Old 05-30-2012
Quote:
Originally Posted by Corona688
You are working with FOUR GIGABYTES of data. Of course it's slow. What's the file size, divided by your maximum disk speed? How much RAM do you have for cache? What's the bus speed of your RAM?

I don't think you're going to improve much on the awk version. You might try tricks with index() instead of regexes.

can you please elaborate on the index()? examples?
# 9  
Old 05-30-2012
Code:
awk 'index($0, "not a regex") && index($0, "also not a regex") { N++ } END { print N }'

...but I'm certain it's not going to help, because my questions weren't rhetorical. CPU probably isn't your bottleneck, and you're only going to get minor speed improvements from changing the program.

How fast is your disk? How fast is your memory? How large is your memory?

These are important questions when throwing around huge files.
# 10  
Old 05-30-2012
Quote:
Originally Posted by Corona688
Code:
awk 'index($0, "not a regex") && index($0, "also not a regex") { N++ } END { print N }'

...but I'm certain it's not going to help, because my questions weren't rhetorical. CPU probably isn't your bottleneck, and you're only going to get minor speed improvements from changing the program.

How fast is your disk? How fast is your memory? How large is your memory?

These are important questions when throwing around huge files.
yeah you're right. i have to do this on hundreds of servers that have different hardware so i cant really specify.

thank you
# 11  
Old 05-30-2012
With this much data to go through, it may be better to handle the data as-it-happens instead of after the fact. Or cache the results for later if you can't do that, since it's just not going to be fast, period.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Shell script run in a case statement call to run a php file, also Perl

Linux System having all Perl, Python, PHP (and Ruby) installed From a Shell script, can call a Perl, Python, PHP (or Ruby ?) file eg eg a Shell script run in a case statement call to run a php file, also Perl or/and Python file??? Like #!/usr/bin/bash .... .... case $INPUT_STRING... (1 Reply)
Discussion started by: hoyanet
1 Replies

2. Shell Programming and Scripting

Perl: How to run an POST/GET command via proxy?

Please what is the most simple way to run some perl command via proxy? i have perl script an in it: POST **something** GET **something** what should i do before it so its proxified via proxy without authentiffication, like 1.2.3.4:1080? any way to run whole .pl script via proxy? (3 Replies)
Discussion started by: postcd
3 Replies

3. Shell Programming and Scripting

Script for telnet and run one command kill it and run another command using while loop

( sleep 3 echo ${LOGIN} sleep 2 echo ${PSWD} sleep 2 while read line do echo "$line" PID=$? sleep 2 kill -9 $PID done < temp sleep 5 echo "exit" ) | telnet ${HOST} while is executing only command and exits. (5 Replies)
Discussion started by: sooda
5 Replies

4. Shell Programming and Scripting

Run perl command in script[solved]

Hi all, When I put the Perl command in a script, I got error. system("perl -pi -e 's@words@words@g' myFile"); The error is: Unrecognized character \x8A; marked by <-- HERE after دت مد�<-- HERE near column 15 at -e line 1. Thanks in advance. ---------- Post updated at 06:30 AM... (0 Replies)
Discussion started by: Lham
0 Replies

5. Shell Programming and Scripting

Run perl script, with command-line options

Hello everyone, I have a perl script which takes various command line options from user like : test.pl -i <input_file> -o <output_file> -d <value> -c <value> Now I have multiple input files in a directory: <input_file_1> <input_file_2> <input_file_3> <input_file_4> ..... .... ...... (6 Replies)
Discussion started by: ad23
6 Replies

6. Windows & DOS: Issues & Discussions

Cannot run command line scripts in perl or gawk

I originally posted this to a different forum (I am a new Perl user) and realized the error so I will ask here. I am on a WindowsXP machine trying to run perl and gawk scripts from the command line. I have perl and gawk installed and environment set to C:\perl\bin and cannot get a script to... (2 Replies)
Discussion started by: 10000springs
2 Replies

7. Shell Programming and Scripting

How to run the particular command continously for 30 mins in perl?

Hi, I have command that should run continuously for 30 mins but not every day not once in a week , not one in a month. whenever i call that particular program that command should run for 30 mins and stop. #Contents of test.pl `ls -l *.txt`; #some other lines of code to print ... (1 Reply)
Discussion started by: vanitham
1 Replies

8. Shell Programming and Scripting

Run system command in perl cgi

Hi guys, got a problem with a perl cgi script over here. I need it to run a system command to get the status of a process. Unfortunately the process is owned by a specific user and only this user can get its status. So i tried running the command from the perl cgi with "su", but then i get the... (12 Replies)
Discussion started by: polki
12 Replies

9. Shell Programming and Scripting

Run the command inside perl script

I have a command which will run fine in a unix command prompt. Can you tell how to interprete this command inside perl script...... The command is : perl -pe 's/(\|333\}.*)\}$/$1|1.6}/' FIA.txt This will search for the number 333 and appends 1.6 at the end of that line....... (1 Reply)
Discussion started by: vinay123
1 Replies

10. Shell Programming and Scripting

Perl run system command

Can perl execute a system command similar to the C function System()? Thanks. Gregg (1 Reply)
Discussion started by: gdboling
1 Replies
Login or Register to Ask a Question