UNIX for Advanced & Expert Users

Dear all

I have group of input lines which look like this


These input lines is placed in a file named phonelines.txt and there is a script which match $4 and $5 with country codes placed in another file named country-codes.txt and its contents is :

Italy 39
Libyana 21892
Thuraya Satellite 88216
Lebanon 964
England 44

and so on this file will be containning country codes of alot of countries

I have a script which work on these files and the output look like this

Belgium1 ISC 3 924556808 393475157928 b 1B Libyana Italy
Italy ISC 151 925082838 447717254923 b 26 Libyana England
ISC BT-GlasGow 152 88216900500 218925288836 f 1B Thuraya Libyana
ISC Spain-Telefonica 14 964188208970 218925735325 b 29 Lebanon Libyana

as shown there are two added fields according to the country codes

The problem that the script i am using is slow because i have large input lines (may reach 300000) and i have lot of countries so i want a faster code

The code i have is:
sed 's=\(.*\) \(.*\)=awk :{if($4~/^\2/)print $0" \1";else print}: phonelines.txt=' country-codes.txt|tr ':' "'"|sh|awk 'NF==8{print}' >newlines.txt
sed 's=\(.*\) \(.*\)=awk :{if($5~/^\2/)print $0" \1";else print}: newlines.txt=' country-codes.txt|tr ':' "'"|sh|awk 'NF==9{print}'

So do you have faster one to be more reliable ?

Note : I am using Solaris 8 SUN280R

Please advise
Zanetti

Sorry but i forget to put the input lines which is mentioned in my first words; it look like this:

Belgium1 ISC 3 924556808 393475157928 b 1B
Italy ISC 151 925082838 447717254923 b 26
ISC BT-GlasGow 152 88216900500 218925288836 f 1B
ISC Spain-Telefonica 14 964188208970 218925735325 b 29

Thanks
Zanetti

The perl script I wrote three weeks ago might be faster than what you ended up with; have you tested?

https://www.unix.com/unix-advanced-ex...#post302180718

Do you have some sort of timing data, what's the current performance and what would you like for it to be?

Ugh, looking closer at that code, I can see that it's going to be slow. It starts a separate awk for each telephone code! Please test the Perl script on some real-world data; if it feels fast enough, we can work on making it format the output the way you want it.

(Certainly the awk code would be possible to optimize, at least to make it a bit less hungry for processes; but if your awk is old and tired, Perl is probably easier to work with. At least it won't throw syntax errors if we don't have exactly the same version.)

Actually, this any better?

Code:

vnix$ sed -e 's%\(.*\) \(.*\)%$4 ~ /^\2/ { c1=" \1" } $5 ~ /^\2/ { c2=" \1" }%' \
>   -e '$a\{ print $0 c1 c2; c1 = c2 = ""; }' country-codes.txt |
> awk -f - phonelines.txt
Belgium1 ISC 3 924556808 393475157928 b 1B Italy
Italy ISC 151 925082838 447717254923 b 26 England
ISC BT-GlasGow 152 88216900500 218925288836 f 1B Thuraya Satellite Libyana
ISC Spain-Telefonica 14 964188208970 218925735325 b 29 Lebanon Libyana

The output is ambiguous when you have one code but not the other, is that a problem?

Dear Era

The problem is that the perl script dont output my need at all ; i prefer to use the last code which is :

vnix$ sed -e 's%\(.*\) \(.*\)%$4 ~ /^\2/ { c1=" \1" } $5 ~ /^\2/ { c2=" \1" }%' \
> -e '$a\{ print $0 c1 c2; c1 = c2 = ""; }' country-codes.txt |
> awk -f - phonelines.txt

Do you mean that these lines will be wrote in a seperate then change to excutable and run it or run it manually from the command line

Is this seems faster; also does it in need any modifications?

Zanetti

The Perl code is a proof of concept. If it's faster (my uninformed guess is it will be) then changing it so it prints what you want is trivial, and you probably don't even need to ask here how to change it (but feel free if you can't figure it out).

Do what you will with the code; if having it in a file is more convenient then by all means put it in a file and mark it executable. Whatever you did with the previous script, you can do with this one. This is just an optimized version of unilover's script.

If it prints what you want, then it doesn't require modification. You tell us.

Can you run this one and the Perl script on your real-world data and report how long they take to run? (Never mind that the Perl script doesn't print the stuff you want; changing that won't make it faster or slower.) If the Perl script is not a lot faster then you can probably live with this awk+sed combo.