Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

using array inside awk

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting using array inside awk
# 1  
Old 08-07-2008
using array inside awk
Hi All,

I have the following code sequence for reading some bulk file and moving the content to two different arrays.

while read data
do
THREEG_PATTERN=`echo $data | egrep "3G"`
if [ "$data" == "$THREEG_PATTERN" ]
then
NEW_THREEG_PATTERN=`echo $THREEG_PATTERN | cut -d " " -f2`
THREEG_ARRAY[$IDX]=$NEW_THREEG_PATTERN
IDX=$(($IDX+1))
else
SP_ARRAY[$IDX_SP]=$data
IDX_SP=$(($IDX_SP+1))
fi

done < $_pat

This code is working perfectly but this is taking long time because of big file size.... How can i improve the performance of this process by using awk or other mechanism? Or how can i convert this entire code to awk? Looking forward ur reply

Thanks in advance
Subin
# 2  
Old 08-07-2008
If you post an example of your input and the desired output (or whatever you want to do with it), it would be easier.
# 3  
Old 08-07-2008
Yes, the old
Code:
while read rec; do 
    # process stuff
done < file

construct is notoriously slow and thus should only be used
on small to moderately big files.
I would go for Perl.
Without having looked too carefully at your processing,
and without even having tried for correct syntax
some Perl statement like this might do?
Code:
$ perl -ane 'push @ary,$F[1] if /3G/;END{$"="\n";print"@ary"}' file_to_read_from

# 4  
Old 08-07-2008
Hi ,
Thanks for the reply

My input file will be like this

3G ^Alcatel-OT-S920\/
3G ^Amoi.*A500
3G SPV.*C200
3G ^Levis_Original_3G
3G ASUS-P735
3G ^SIE-E71F\/
3G ^SIE-E71\/
3G ^SIE-ELF1
^Sagem-my401C
^SAGEM-my411C-orange
^SAGEM-my511X-orange
^SEC-SGHC520
^SEC-SGHD520\/
^SEC-SGHE570\/
^SEC-SGHF210
^SEC-SGHF300\/
^SAMSUNG-SGH-F490
^SEC-SGHP310
^SEC-SGHP520
^SAMSUNG-SGH-D500\/
^SAMSUNG-SGH-D600\/
^SAMSUNG-SGH-D900
3G SPV.*M650
3G ^HTC-P4550
3G ^HTC-P4550-orange\/
3G ^HTC-P5500-orange\/
3G SPV.*C100
3G SPV.*C600
3G SPV.*C700
3G SPV.*E65


And i need to stroe these things in to 2 differnt arrays ...
# 5  
Old 08-07-2008
OK,
and once you have built the arrays? What's the next task?
# 6  
Old 08-07-2008
After that some other functions will read the values from these two arrays and compare the values which they have ....

Thanks and Regards,
Subin
# 7  
Old 08-07-2008
It's quite easy to build those arrays with Awk and Perl, but you should continue to use those languages for the next task:

Code:
mint% awk 'END {
  while (++n <= i)
    printf "tg_a, element %d: %s\n", n, tg_a[n]
  while (++m <= j)
    printf "sp_a, element %d: %s\n", m, sp_a[m]
    }
{
  if (/^3G/)
    tg_a[++i] = $2
  else
    sp_a[++j] = $0
  }' file
tg_a, element 1: ^Alcatel-OT-S920\/
tg_a, element 2: ^Amoi.*A500
tg_a, element 3: SPV.*C200
tg_a, element 4: ^Levis_Original_3G
tg_a, element 5: ASUS-P735
tg_a, element 6: ^SIE-E71F\/
tg_a, element 7: ^SIE-E71\/
tg_a, element 8: ^SIE-ELF1
tg_a, element 9: SPV.*M650
tg_a, element 10: ^HTC-P4550
tg_a, element 11: ^HTC-P4550-orange\/
tg_a, element 12: ^HTC-P5500-orange\/
tg_a, element 13: SPV.*C100
tg_a, element 14: SPV.*C600
tg_a, element 15: SPV.*C700
tg_a, element 16: SPV.*E65
sp_a, element 1: ^Sagem-my401C
sp_a, element 2: ^SAGEM-my411C-orange
sp_a, element 3: ^SAGEM-my511X-orange
sp_a, element 4: ^SEC-SGHC520
sp_a, element 5: ^SEC-SGHD520\/
sp_a, element 6: ^SEC-SGHE570\/
sp_a, element 7: ^SEC-SGHF210
sp_a, element 8: ^SEC-SGHF300\/
sp_a, element 9: ^SAMSUNG-SGH-F490
sp_a, element 10: ^SEC-SGHP310
sp_a, element 11: ^SEC-SGHP520
sp_a, element 12: ^SAMSUNG-SGH-D500\/
sp_a, element 13: ^SAMSUNG-SGH-D600\/
sp_a, element 14: ^SAMSUNG-SGH-D900
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

sed inside the awk script to replace a string in the array

The requirement is i need to find an array value matching with pattern {5:{ , replace that with 5: and reassign that to same array index and print it. I write something like below and the issue is sed command is not working. If i replace " with "`" the script gives syntax error.how can i... (8 Replies)
Discussion started by: bhagya123
8 Replies

2. UNIX for Dummies Questions & Answers

Array inside sed

Hi guys Let me at first describe the whole thing that I'm trying to do. Lets say I have 100 files like the following one. Ow 1230 16.000000 -0.834000 16.083957 1.751652398 -17.20094528 -4.450623277 Hw 1231 ... (6 Replies)
Discussion started by: saleheen
6 Replies

3. Shell Programming and Scripting

How to use variable inside array?

I tried to use variable inside an array variable, but its not working as expected.:wall: ENV1=123 ENV1=789 ENV1=120 ENV2=567 if then name=ENV1 echo "${name}" echo "${name}" echo "${name}" else name=ENV1 echo "${name}" fi Output: ./val.sh 1 123 (2 Replies)
Discussion started by: Jayavinoth
2 Replies

4. Shell Programming and Scripting

HELP with AWK one-liner. Need to employ an If condition inside AWK to check for array variable ?

Hello experts, I'm stuck with this script for three days now. Here's what i need. I need to split a large delimited (,) file into 2 files based on the value present in the last field. Samp: Something.csv bca,adc,asdf,123,12C bca,adc,asdf,123,13C def,adc,asdf,123,12A I need this split... (6 Replies)
Discussion started by: shell_boy23
6 Replies

5. Shell Programming and Scripting

unique inside array

I have a file root@server # cat /root/list12 11.22.33.44 22.33.44.55 33.44.55.66 33.44.55.66 33.44.55.66 I try to pass to array and display unique. root@server# cat /root/test12.sh #!/bin/bash #delcare array badips and accumulate values to array elemenrs badips=( $( cat... (4 Replies)
Discussion started by: anil510
4 Replies

6. Shell Programming and Scripting

Calling array inside awk

Hello I have the file df.tmp FS is actually the / FS but escape character\ and end of line $ is used in order to fetch exctly / and not other filesystems. awk '/\/$/ {print $(NF-1)+0}' df.tmp will work properly and return a value eg. 60 but when I am trying to issue the command with the array... (3 Replies)
Discussion started by: drbiloukos
3 Replies

7. Shell Programming and Scripting

printing array elements inside AWK

i just want to dump my array and see if it contains the values i am expecting. It should print as follows, ignore=345fht ignore=rthfg56 . . . ignore=49568g Here is the code. Is this even possible to do? please help termReport.pl < $4 | dos2ux | head -2000 | awk ' BEGIN... (0 Replies)
Discussion started by: usustarr
0 Replies

8. Shell Programming and Scripting

split and making an array inside another array

I want to run an awk split on a value that has been pushed through an array and I was wondering what the syntax should be?? e.g. running time strings through an array and trying to examine just minutes: 12:25:30 10:15:13 08:55:23 awk ' NR==FNR{ ... (2 Replies)
Discussion started by: dcfargo
2 Replies

9. UNIX for Advanced & Expert Users

Array inside an array

hi All, I have a array as follows, array1=("xx" "abc" "def" "xyz") and each array1 is also storing some array values, like array1=abc and abc=("a" "b" "c") etcetera etcetra......... Note : each subarray under array1 have index 3 i.e. it can max contain 3 values if i echo ${abc} ... (5 Replies)
Discussion started by: manas_ranjan
5 Replies

10. Shell Programming and Scripting

looping a array inside inside ssh is not working, pls help

set -A arr a1 a2 a3 a4 # START ssh -xq $Server1 -l $Username /usr/bin/ksh <<-EOS integer j=0 for loop in ${arr} do printf "array - ${arr}\n" (( j = j + 1 )) j=`expr j+1` done EOS # END ========= this is not giving me correct output. I... (5 Replies)
Discussion started by: reldb
5 Replies
Login or Register to Ask a Question