Using array in script?


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Using array in script?
# 1  
Old 02-01-2013
Using array in script?

Dear all,

I have following set of code [1] which I want to modify
In the given code, the given PATH535 only accept the single path.
I would like to modify it to make it array and should be able to pass several
of the path address together.

for example like following:
Code:
PATH535[1]=/pathA/log/A/
PATH535[2]=/pathA/log/B/
PATH535[3]=/pathA/log/C/

And the code works for all the files kept at these locations of PATH535[] array?

Please help me,
thanks in advance -emily

[1]======================
Code:
 
PATH535=/pathA/log/A

doCheck() {
    for FileNameIndx in "${PATH535[@]}"
      do
      if [[ ! -e "dest_path/$FileNameIndx" ]]; then
          ls -ltr "$FileNameIndx" | grep root | awk -F_ '{print $3,$0}' OFS=\t | sort -n | cut -f2- >> $File0"_0"
          #ls -ltr "$FileNameIndx" | grep root | awk '{print string path $9}' string="$CONSTANT" path="$FileNameIndx"  >> "$File0"                    
          sort -nrk5 < $File0"_0" | awk -F_ '!x[$3]++' >> $File0"_1"
          grep -in "vg" $File0"_1" | awk '{print path string $9}' string="/" path="$FileNameIndx" >> $FileName
          echo "$FileNameIndx is copied"
      else
          echo "Check the FileName in ${PATHNAME[@]}"
      fi
      echo "--------"
}

# 2  
Old 02-01-2013
instead of modifying the function and add a loop in it,
maybe you could just do a loop which call that function.

But if it's just a matter of performance issue that makes you willing modify the code, maybe you should consider redesigning it completly.

I haven't tried to understand what it does exactly, but at a first look, i would say it is far to be optimized
# 3  
Old 02-01-2013
I partially agree with ctsgnb, and partly disagree. With all of the things that are not shown to us, I don't know if this script is optimized or extremely inefficient.

Here is your script again (after setting up PATH535 to be the array of 3 pathnames as you indicated you wanted to do earlier in your problem description) with line numbers added for discussion purposes:
Code:
 1 PATH535[1]=/pathA/log/A/
 2 PATH535[2]=/pathA/log/B/
 3 PATH535[3]=/pathA/log/C/
 4
 5 doCheck() {
 6     for FileNameIndx in "${PATH535[@]}"
 7       do
 8       if [[ ! -e "dest_path/$FileNameIndx" ]]; then
 9           ls -ltr "$FileNameIndx" | grep root | awk -F_ '{print $3,$0}' OFS=\t | sort -n | cut -f2- >> $File0"_0"
10           #ls -ltr "$FileNameIndx" | grep root | awk '{print string path $9}' string="$CONSTANT" path="$FileNameIndx"  >> "$File0"
11           sort -nrk5 < $File0"_0" | awk -F_ '!x[$3]++' >> $File0"_1"
12           grep -in "vg" $File0"_1" | awk '{print path string $9}' string="/" path="$FileNameIndx" >> $FileName
13           echo "$FileNameIndx is copied"
14       else
15           echo "Check the FileName in ${PATHNAME[@]}"
16       fi
17       echo "--------"
18 }

Since I have no idea what is in your log files, nor how the variables CONSTANT, File0, and FileName have been initialized before you call your doCheck() function, I can't make any educated guess as to what you're trying to do, but a few things look strange:
  1. I would expect the variable name FileNameIndex to be an index into an array named FileName. But starting on line 6, you use FileNameIndx as the name of a file (of type directory) from the array PATH535.
  2. On line 8 you check to see if a directory exists in a sub-directory of your current working directory. If it doesn't exist, you perform several operations, but none of them create the directory that you were looking for.
  3. On line 9, OFS=\t sets the output field separator for this awk command to t; not to a <tab> character. (If you want a <tab> character, change it to OFS='\t'.)
  4. Line 15 seems a bit strange. Why say:
    Code:
    Check the FileName in list of directory names

    when what it really means is:
    Code:
    Processing skipped: dest_path/$FileNameIndx already exists

  5. The do on line 7 doesn't have a matching done. Presumably the missing done should be added before or after line 17.
This User Gave Thanks to Don Cragun For This Post:
# 4  
Old 02-02-2013
Hi Don Cragun,
I appreciate your reply and the time you spend on that piece of code.
I completely agree with you and ctsgnb that code is far from optimization.

Being novice with shell scripting, I was happy with the idea that I could make working script which looks nasty but does my task pretty well. And the sole reason to post the question here was try to optimize it. That also make me learn more of shell scripting.

Now, Let me explain you the task of this piece of the script.
I have several of the files kept at some locations like
Code:
$ls /path/path_goesTo/A/
vgtree_132_1_vsd.root  vgtree_182_1_asq.root  vgtree_231_1_FWO.root  vgtree_281_1_RV7.root  vgtree_39_1_HCC.root   vgtree_89_1_oaM.root
vgtree_133_1_lDx.root  vgtree_183_1_1lF.root  vgtree_23_1_q4e.root   vgtree_28_1_OkE.root   vgtree_40_1_ROj.root   vgtree_90_1_nfF.root
vgtree_134_1_rtQ.root  vgtree_184_1_JDe.root  vgtree_232_1_rNt.root  vgtree_282_1_sK6.root  vgtree_41_1_1JA.root   vgtree_91_1_9r8.root
vgtree_135_1_U24.root  vgtree_185_1_S7u.root  vgtree_233_1_KrH.root  vgtree_283_1_3Wj.root  vgtree_4_1_YrR.root    vgtree_9_1_NME.root
vgtree_136_1_TrA.root  vgtree_186_1_HwX.root  vgtree_234_1_3pj.root  vgtree_284_1_N3Z.root  vgtree_42_1_DcY.root   vgtree_92_1_JQY.root
vgtree_137_1_yVj.root  vgtree_187_1_cda.root  vgtree_235_1_XrJ.root  vgtree_285_1_z8g.root  vgtree_43_1_msk.root   vgtree_93_1_oOe.root
vgtree_138_1_zOd.root  vgtree_188_1_N2U.root  vgtree_236_1_wpb.root  vgtree_286_1_CLY.root  vgtree_44_1_yDV.root   vgtree_94_1_1YG.root
vgtree_139_1_Gmf.root  vgtree_189_1_ohb.root  vgtree_237_1_PF2.root  vgtree_287_1_3oO.root  vgtree_45_1_ZWL.root   vgtree_95_1_SQx.root

And my purpose is to copy them in one common file named $FileName. Important is to make sure that if there is any repeating of files than only the file with the bigger size should be kept.

Repeating would implies, in the given example the deciding critera is to match the no '187' and if there are three different files, go for copying the file name vgtree_187_1_cda.root
Code:
-rw-r--r-- 1 pooja04 us_cms 38939100 Dec 19 21:38 vgtree_187_1_cda.root
-rw-r--r-- 1 pooja04 us_cms 8900 Dec 19 21:38 vgtree_187_2_jdy.root
-rw-r--r-- 1 pooja04 us_cms 939100 Dec 19 21:38 vgtree_187_3_chj.root

Following this criteria I want a final file $DataFile with the content as:
Code:
/path/path_goesTo/A/vgtree_95_1_SQx.root
/path/path_goesTo/A/vgtree_96_1_TGx.root
/path/path_goesTo/A/vgtree_97_2_YKx.root
/path/path_goesTo/A/vgtree_98_1_RKx.root


And the following as I explained for single $PATH would infect be repeated for several of PATH and with millions of root files.

I hope, I could clear as what I wanted to do.

Thanks a lot
pooja
# 5  
Old 02-03-2013
Code:
ls -l /path/path_goesTo/A/ | sort -r -k 5n -t_ -k 2n | awk '{split($9,x,".");split(x[1],a,"_");i=a[1]"_"a[2]}!F[i]++{print $9}' | xargs -i cat {} >> yourbigfile

or
Code:
...| sort -r -k 5n -t_ -k 2n | awk '{print $NF}' | awk -F"[._]" '{i=$1"_"$2}!F[i]++{print $0}' | xargs -i cat {} >> bigoutput

Depending on the implementation, the "sort" command may need some tweak so that it displays files by number
and if several index (ex: _1 _2 _3) for a same file number (187) it displays the biggest in size at first.

As far as i tested, the output given by the sort command works fine on my Ubuntu.
Once the file are sorted that way, it build the initial base name of a file number (that is contained in i) then print only the first met and add it into your big file.
appending this into the big file could also be added within awk using the "system" command (if so, remember to close each file after concatenation into the big one).

Last edited by ctsgnb; 02-03-2013 at 05:28 PM..
This User Gave Thanks to ctsgnb For This Post:
# 6  
Old 02-09-2013
thanks ctsgnb for the detailed reply.
I have small doubt, I still need to loop over many nested directories using
array.
Can you help me with that?

thanks again,
emily,
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Need Shell Script for Array

Hi all, I have an input file like below (a comma seperated file) 345,12,10 400,11,8 328,1,3 I need to get the output as below ... record 345 sum is 12 record 400 sum is 10 record 328 sum is 1 record 345 count is 10 record 400 count is 8 record 328 count... (15 Replies)
Discussion started by: hemanthsaikumar
15 Replies

2. UNIX for Advanced & Expert Users

Help in perl script Array.

hi Team, i need a help in perl , i need to get values(10 rows +) from perl GUI and insert those values into oracle table. am trying to achive this in Perl array, can you please help me on this. thanks senthil (1 Reply)
Discussion started by: senkerth
1 Replies

3. Shell Programming and Scripting

Array in shell script

Hi, check=("/usr/local/bin/chk_nag | awk -F":" '{print $1}'" "/usr/local/bin/chk_kas | awk -F":" '{print $1}'" "/usr/local/bin/chk_was | awk -F":" '{print $1}'" ) for i in "${check}"; do echo $i; done when I run this. It says Syntax error: "(" unexpected Please advise. (5 Replies)
Discussion started by: ashokvpp
5 Replies

4. Shell Programming and Scripting

sendEmail: Script and Array

I have a script that send email using sendEmail (CentOS shell) script is: sendEmail -vvv -u "TestCompany, Inc.: USB Added" -f user@domain.com -t To@domain.com -s ASPMX.L.GOOGLE.com -m "USB Activity" -o tls=aut o username=user3@domain.com password=password So far this works fine. Now i... (5 Replies)
Discussion started by: kashif.live
5 Replies

5. Shell Programming and Scripting

A typical array script

Hi All, I need to store the output of "find ." to an array one by one. Output of find . in my case will look like :- . ./one ./one/a ./one/b ./one/c ./two So my first array element should be "/one" and second one "/one/a" (need to remove "." from the output as well). Then I need to... (11 Replies)
Discussion started by: Renjesh
11 Replies

6. Emergency UNIX and Linux Support

Problem with Array in Script

Below is my script. This script is getting an error code such as this one. fileListener.bat: entityArray=craig.uss@pnc.com: not found craig.uss@pnc.com fileListener.bat: entityArray=duns_noncusts.txt: not found duns_noncusts.txt fileListener.bat: entityArray=duns_misc.cpy: not found... (4 Replies)
Discussion started by: mkjp
4 Replies

7. Shell Programming and Scripting

How to export an array to perl script?

hi all...... i want to use an array ,declared in bash, in embedded perl script. is there any way to export whole array so that i can use it '$ENV{}' or something.. thanx in advance!! regards, prayush (1 Reply)
Discussion started by: tprayush
1 Replies

8. Shell Programming and Scripting

KSH script not looping through array

Hi All, I'm trying to get a script to loop through an array. The array is basically a list of .zip files. I'd like the script to loop through and unzip the zip files contained in the zip file list. When I run the script, it unzip the first zip file correctly, and then stops Any thoughts? Here's... (2 Replies)
Discussion started by: kelldan
2 Replies

9. Shell Programming and Scripting

Please Correct this Array Script

Here is the script. I am not able to increment and Print echo "2=" ${arr} (Bold line) Can someone please help me correct it? I am thinking there is some escape sequence error. Just started learning scripts ... Thanks #!/bin/ksh set -A arr a1 a2 a3 a4 # START Server1=gmi ... (6 Replies)
Discussion started by: psreddy1234
6 Replies

10. Shell Programming and Scripting

Unix script array

Unix shell script; How do I create an array and dynamically populate it with values? How do I retrieve from the array? (4 Replies)
Discussion started by: leemjesse
4 Replies
Login or Register to Ask a Question