I tried your perl script code and it can't be executed also i tried your last code optimized ; when i tried it directly it tells ambiguous output redirect when i write it in a separate file and run it as executable script this is the output: ./code2
./code2: -f: not found
geo@spiserver /home/geo/cdr/AnsweredCalls 58 > Can't open $a\{ print $0 c1 c2; c1 = c2 = ""; }
Please Advise; For you information the required response for half a million line is 1 minute to 1 and half minute and note that the file containing mapping codes will contain also large number of entries
If your awk won't accept a script on standard input, you can save the sed output to a file, and keep that (until the list of number code changes again).
You could also check if you have nawk or mawk or gawk; I would guess those might be faster than the old awk.
Do you have timing numbers from the perl script, are they acceptable?
Just printing 500,000 lines, awk (actually mawk) is much faster than perl on my system, but I believe Perl should have better regex optimizations, so for a script which does a lot of matching, I vaguely speculate that Perl might still be faster.
Did some more tests; at least for a small country-codes.txt, the awk is faster. The comparison is not entirely fair because the awk has the country codes hardcoded, whereas the perl script reads it in from disk every time.
Please Lets concentrate on the nawk ; i do have nawk ; i can't catch the perl script at all or may be i can't run it , what i did that i write the same code you gave me in a seperate file and i changed its mode to +x and i run it and it can't be run
Regarding your last script ; Note that the country codes.txt will be created once and left
Regarding the execution of last script also i write it in a seperate file and it the error is cant open /usr/bin/awk
Please can you specify your words in points so as to catch you ?
Finally i succeded to execute the perl script you gave me before but the output as you said is not may expectations at all here you the output
218914146348 matches , maps to
f matches , maps to
218925098033 matches , maps to
f matches , maps to
218928892045 matches , maps to
f matches , maps to
218924078768 matches , maps to
f matches , maps to
It maps only the $3 and not both ($3 and $4) also as you see the country codes in mappings.txt is not included and also i dont want f matched and maps to something i need only $3 and $4 to be mapped with the codes in the mappings.txt which looks like:
Egypt-Vodafone 2010
UK 44
Egypt-Mobinil 2012
Libya-Libyana 21892
France 33
And so on
I think it is good now that i have the fastest script according to your guess so please help me to customize it to have output like the following:
ISC Italy 1249 20125199518 218924831978 b 22
The output after the code will be:
ISC Italy 1249 20125199518 218924831978 b 22 Egypt-Mobinil Egypt Libyana
I meant to mention that if your awk has a different path, you need to change that bit.
I can't test these scripts on real-world data, but you have the data -- can you perform a test run like I did and see what's fast enough?
The Perl script might need some modifications, the output doesn't look correct any longer; did you change the field numbers at some point? I notice the awk script had $4 and $5 before, and now you have the phone numbers in $6 and $7 apparently.
The awk script can easily be converted to Perl using a2p so you can really compare them head to head if you just can get the latest awk script to work. Somewhat to my surprise, your awk script (which uses if inside a single main clause, instead of one little clause for each number code) seemed to be the fastest in my limited test (with some slight modifications I made). Here's my slightly modified version, and the timings:
Code:
vnix$ # This is the script I posted earlier, optimized version of unilover's
vnix$ yes phonelines.txt | head -125000 | time xargs ./country-codes.awk >/dev/null
2.39user 0.74system 0:03.55elapsed 88%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+5958minor)pagefaults 0swaps
vnix$ echo '#!/usr/bin/awk -f ' >phonylines.awk
vnix$ echo '{ c1 = c2 = "";' >>phonylines.awk
vnix$ sed 's%\(.*\) \(.*\)%if ($4~/^\2/) c1 = " \1"; if ($5~/^\2/) c2 = " \1";%' country-codes.txt >>phonylines.awk
vnix$ echo 'print $0 c1 c2; }' >>phonylines.awk
vnix$ chmod +x ./phonylines.awk
vnix$ ./phonylines.awk phonelines.txt
Belgium1 ISC 3 924556808 393475157928 b 1B Italy
Italy ISC 151 925082838 447717254923 b 26 England
ISC BT-GlasGow 152 88216900500 218925288836 f 1B Thuraya Satellite Libyana
ISC Spain-Telefonica 14 964188208970 218925735325 b 29 Lebanon Libyana
vnix$ yes phonelines.txt | head -125000 | time xargs ./phonylines.awk >/dev/null
2.25user 0.72system 0:03.38elapsed 88%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+5956minor)pagefaults 0swaps
vnix$ yes phonelines.txt | head -125000 | time xargs perl ./phonylines.pl >/dev/null
9.78user 1.28system 0:12.44elapsed 88%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+10935minor)pagefaults 0swaps
vnix$ # ouch
But it needs to be said that this is a very limited number of phone codes, and the game might change when you have a lot of them.
Last edited by era; 04-24-2008 at 05:07 PM..
Reason: Oops, needed to change sed separator character; reset c1 and c2 at beginning (again )-:
Hi All,
I'm new to the forum and to bash scripting. I did some stuff with VB.net, Batch, and VBScripting in the past, but because I shifted over to Linux, I am learning to script in Bash at this moment. So bear with me if I seem to script like a newbie, that's just because I am ;-)
OK, I... (9 Replies)
Can someone help me edit the below script to make it run faster?
Shell: bash
OS: Linux Red Hat
The point of the script is to grab entire chunks of information that concerns the service "MEMORY_CHECK".
For each chunk, the beginning starts with "service {", and ends with "}".
I should... (15 Replies)
Hi all,
In bash scripting, I use to read files:
cat $file | while read line; do
...
doneHowever, it's a very slow way to read file line by line.
E.g. In a file that has 3 columns, and less than 400 rows, like this:
I run next script:
cat $line | while read line; do ## Reads each... (10 Replies)
One of our servers runs Solaris 8 and does not have "ls -lh" as a valid command. I wrote the following script to make the ls output easier to read and emulate "ls -lh" functionality. The script works, but it is slow when executed on a directory that contains a large number of files. Can anyone make... (10 Replies)
Dear All
I have a file which contains lines looks like this:
ISC Egypt-Alex2 126 104541338 218926893238 f 1B
ISC BT-Colindale 26 249126190534 218913486850 b 29
ISC Egypt-Cairo2 199 129026052 218927661509 b 26
As you see in each line $4 and $5 are phone numbers , i want a script which... (19 Replies)
Hi Experts,
I am trying to write a ksh script that it should notify in case there is no response from the other script. I mean to say that I got a.sh and b.sh
the execution of b.sh depends on a.sh, so if there is no response from a.sh,
b.sh should notify me about the same.
Thanks in Advance (4 Replies)
