Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Efficient way to loop through list

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Efficient way to loop through list
# 1  
Old 07-19-2016
Efficient way to loop through list
i have a variable that contains a list of files:
Code:
varA="/var/tmp/hello_0_myapp
/var/tmp/mello_1_myapp
/var/tmp/jello_2_myapp
/var/tmp/fello_3_myapp"

And i'm calling this variable in several functions:

Code:
col0 () {
        FILE=$(echo "${varA}" | egrep "_0_myapp" | sed "s~_0_myapp~~g")
                        eachfile=${FILE}
}

col1 () {
        FILE=$(echo "${varA}" | egrep "_1_myapp" | sed "s~_1_myapp~~g")
                        eachfile=${FILE}
}

col2 () {
        FILE=$(echo "${varA}" | egrep "_2_myapp" | sed "s~_2_myapp~~g")
                        eachfile=${FILE}
}

col3 () {
        FILE=$(echo "${varA}" | egrep "_2_myapp" | sed "s~_2_myapp~~g")
                        eachfile=${FILE}
}

As you can see here, inside each function, im going through each file in the "varA" variable which contains a very huge list of files. i'm echoing the varA variable several times.

how can this be done more efficiently?

btw, i only listed 4 functions here. but in the actual script, there are several functions. so i need this to be very efficient
# 2  
Old 07-19-2016
Anything will be more efficient than the echo | grep | sed | awk | kitchen | sink you have now, but the basic shell for loop is intended for exactly this situation:
Code:
for X in $VAR
do
        echo "$X"
done

Do not quote $VAR, it depends on it being split on spaces or newlines.

It'd be helpful to see how you're actually using col0(), col1(), etc too.
These 2 Users Gave Thanks to Corona688 For This Post:
rbatte1 SkySmart
# 3  
Old 07-19-2016
Take a step back to where that variable is assigned. Where do the values come from? A file? The output of a "command substitution" (that could be redirected to a file as well)?
Why don't you read the values from that file in your functions?
These 2 Users Gave Thanks to RudiC For This Post:
rbatte1 SkySmart
# 4  
Old 07-20-2016
I agree, the taken division into functions seems not optimal.
How often is each function called? A function makes most sense when it is called multiple times.
--
A technical simplification: the two commands
Code:
grep "_1_myapp" | sed "s~_1_myapp~~g"

can be done by only sed
Code:
sed -n  "s~_1_myapp~~gp"

The p modifier prints if a substitution took place.
The -n option suppresses the default print.
The g modifier only makes sense when you expect multiple _1_myapp per line.
This User Gave Thanks to MadeInGermany For This Post:
SkySmart
# 5  
Old 07-20-2016
OK, you seem to need the FILE variable to become a chopped-off filename depending on an integer 0 - 3. Given your variable contains
Code:
echo "$varA"
/var/tmp/hello_0_myapp
/var/tmp/mello_1_myapp
/var/tmp/jello_2_myapp
/var/tmp/fello_3_myapp

(whereever this might come from), you might get results very efficiently using an array (assuming bash):
Code:
ArrA=( ${varA//_?_myapp/} )
echo ${ArrA[@]}
/var/tmp/hello /var/tmp/mello /var/tmp/jello /var/tmp/fello

and then assign FILE=${ArrA[2]}, for example, yielding
Code:
echo $FILE
/var/tmp/jello

. Would this come close to what you need? The number of "functions" would be (not quite, but almost) unlimited.
Last edited by RudiC; 07-20-2016 at 05:01 PM..
This User Gave Thanks to RudiC For This Post:
SkySmart
# 6  
Old 07-20-2016
In bash or ksh93 you could use arrays, like so (keeping you col functions, but consolidating them into one:

Code:
varA="/var/tmp/hello_0_myapp
/var/tmp/mello_1_myapp
/var/tmp/jello_2_myapp
/var/tmp/fello_3_myapp"

oldIFS=$IFS
IFS=$'\n' 
arrA=( $varA )
IFS=$oldIFS

col() {
  eachfile=${arrA[$1]%%_*}
}

col 0; echo $eachfile
col 1; echo $eachfile
col 2; echo $eachfile
col 3; echo $eachfile


Output:
Code:
/var/tmp/hello
/var/tmp/mello
/var/tmp/jello
/var/tmp/fello

This User Gave Thanks to Scrutinizer For This Post:
SkySmart
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

More efficient for loop on files

is there a more efficient way to write the following code: for eachlfile in $(ls -ltcrd ${BACKUPDIR}/${BACKUPNAME}*/content_${BACKUPNAME}* 2>/dev/null | awk '{print $NF}') do echo "==========================" ... (4 Replies)
Discussion started by: SkySmart
4 Replies

2. Shell Programming and Scripting

scp list of files using FOR LOOP in ksh

hi i want to scp files from remote server B to my local server A... and i have a file containing list of all files to be scped from remote server B there is a passwordless connectivity set between remote server and my local server. need a ksh script.. with a for loop that goes through... (2 Replies)
Discussion started by: billpeter3010
2 Replies

3. UNIX for Dummies Questions & Answers

Iterate/Loop Through XML Node List

I need to load an XML file and loop through a list of nodes in it to execute a shell script for each one using the attributes for each node as parameters for the script. Any ideas? Any help will be much appreciated. (1 Reply)
Discussion started by: bradlecat
1 Replies

4. Shell Programming and Scripting

How to loop(Iterate) through List with foreach(csh)

Hey all,, I know cshell is harmful:) but I am using this just "to know" - for educational purposes!... not for a long-term use. lets say i have a list.. set arr=(x y z e f) I wanna iterate the list with foreach ,, not with while.!! foreach i $arr echo $i end does not work (2 Replies)
Discussion started by: eawedat
2 Replies

5. UNIX for Dummies Questions & Answers

For Loop for a list of tab delimited variables

Hello, I need to run a command for a set of input variables that are present in a tab delimited file, a sample of which is shown below: 1 3749 1 4129 1 5980 2 6201 2 9925 2 6894 3 1338 3 6477 3 6242 3 3632 Every row represents the two input values... (2 Replies)
Discussion started by: Gussifinknottle
2 Replies

6. Shell Programming and Scripting

Displaying list of backup files using for loop

Hi, I want to display list of last 10 backup files (with numbering) from the temporary file c:/tmp/tmp_list_bkp.txt Example : 1) backup_file1 2) backup_file2 3) backup_file3 ........ ........ I tried as below but not working. for file in c:/tmp/tmp_list_bkp.txt do echo... (3 Replies)
Discussion started by: milink
3 Replies

7. AIX

Efficient coding / Alternates for "For" loop

Hi, I have the current script with the following code and it works ok. for i in `find . -name "???ABCEDFGH*"` do SESS_NO=`echo $i | awk -F "." '{print $3}'` rm -f $LOGDIR/${OT_QUEUEID}*${SESS_NO}.log 2>/dev/null rm -f $i 2>/dev/null done I want to remove the usage of the... (2 Replies)
Discussion started by: jerardfjay
2 Replies

8. UNIX for Dummies Questions & Answers

List & While Loop

I want to store a set of string values in a 1-D array and then iterate the list by using a loop. Please help this dummy:) (1 Reply)
Discussion started by: ngagemaniac
1 Replies

9. Shell Programming and Scripting

Use file as list in for loop...

Can I use a file as the list in a for loop? I tried it like this: workstation=( $(cat /home/me/file) ) for ws in $workstation do command done The contents of file are like: abc def ghi It only read the initial line. Does each item have to be quoted? Do they have to all be... (11 Replies)
Discussion started by: earnstaf
11 Replies

10. Shell Programming and Scripting

while loop from list

is it possible to run a while loop by comparing a variable with a list ? taglist="<html> </html> <head> </head> <body> </body> <p> </p> <ul> </ul> <li> </li> <em> </em> <title> </title>" read newtag while ] do echo "Entry is not in list, please try again' read newtag done... (6 Replies)
Discussion started by: forever_49ers
6 Replies
Login or Register to Ask a Question