Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

More efficient for loop on files

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting More efficient for loop on files
# 1  
Old 07-23-2016
More efficient for loop on files
is there a more efficient way to write the following code:


Code:
                        for eachlfile in $(ls -ltcrd ${BACKUPDIR}/${BACKUPNAME}*/content_${BACKUPNAME}* 2>/dev/null | awk '{print $NF}')
                        do
                                echo "=========================="
                                echo ${eachlfile} | awk -F"__" '{print $NF}'
                                echo "=========================="
                                cat ${eachlfile}
                        done

i want to be able to run this on AIX, SunOS and LInux systems. So the solution should be portable.
# 2  
Old 07-23-2016
Yes. Drop the -l, -t, -c, and -r options for ls, and you can also drop the awk pipe...
This User Gave Thanks to RudiC For This Post:
SkySmart
# 3  
Old 07-23-2016
Quote:
Originally Posted by SkySmart
is there a more efficient way to write the following code:
Code:
                        for eachlfile in $(ls -ltcrd ${BACKUPDIR}/${BACKUPNAME}*/content_${BACKUPNAME}* 2>/dev/null | awk '{print $NF}')
                        do
                                echo "=========================="
                                echo ${eachlfile} | awk -F"__" '{print $NF}'
                                echo "=========================="
                                cat ${eachlfile}
                        done

i want to be able to run this on AIX, SunOS and LInux systems. So the solution should be portable.
Hello SkySmart,

Could you please let us know what you are trying to do here. Because I don't think you are trying to traverse directories and trying to get files, if you could explain your requirement with details, we could try to help.

Thanks,
R. Singh
This User Gave Thanks to RavinderSingh13 For This Post:
SkySmart
# 4  
Old 07-23-2016
You can keep the ls -trcd options, but -l has to go. And parameter expansion in the shell is MUCH faster than invoking another awk per file processed. Maybe something more like:
Code:
			ls -tcrd | while IFS= read -r eachlfile 
			do
				echo "=========================="
				printf '%s\n' "${eachlfile##*__}"
				echo "=========================="
				cat "$eachlfile"
			done

would do what you want. This should work with ksh or bash on any of those systems. On a Solaris 10 or earlier SunOS system, you'd have to use /usr/xpg4/bin/sh instead of /bin/sh (if you insist on using a shell named sh). The printf is safer than echo. The output from echo can vary depending on what characters are in the pathnames being printed; the printf will give you the pathname itself. Of course, with no indication of what the pathnames you're processing really look like, bite above is completely untested.
This User Gave Thanks to Don Cragun For This Post:
SkySmart
# 5  
Old 07-23-2016
I made a mistake in my last post... The first line:
Code:
			ls -tcrd | while IFS= read -r eachlfile

should have been:
Code:
			ls -tcrd ${BACKUPDIR}/${BACKUPNAME}*/content_${BACKUPNAME}* |
			while IFS= read -r eachlfile

I should have also noted that you can't cat a directory and with or without ls -d you can end up with directory names from your ls if there are any directories matching the pattern ${BACKUPDIR}/${BACKUPNAME}*/content_${BACKUPNAME}*. If that is a problem, you might want to consider changing the line:
Code:
				cat "$eachlfile"

to:
Code:
				if [ -f "$eachlfile" ]
				then	cat "$eachlfile"
				else	printf '*** Not a regular file. ***\n'
				fi

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Efficient way to loop through list

i have a variable that contains a list of files: varA="/var/tmp/hello_0_myapp /var/tmp/mello_1_myapp /var/tmp/jello_2_myapp /var/tmp/fello_3_myapp" And i'm calling this variable in several functions: col0 () { FILE=$(echo "${varA}" | egrep "_0_myapp" | sed "s~_0_myapp~~g")... (5 Replies)
Discussion started by: SkySmart
5 Replies

2. Shell Programming and Scripting

Moving files from parent path to multiple child path using bash in efficient way

Hi All, Can you please provide some pointers to move files from Base path to multiple paths in efficient way.Folder Structure is already created. /Path/AdminUser/User1/1111/Reports/aaa.txt to /Path/User1/1111/Reports/aaa.txt /Path/AdminUser/User1/2222/Reports/bbb.txt to... (6 Replies)
Discussion started by: karthikgv417
6 Replies

3. Shell Programming and Scripting

Most efficient method to extract values from text files

I have a list of files defined in a single file , one on each line.(No.of files may wary each time) eg. content of ETL_LOOKUP.dat /data/project/randomname /data/project/ramname /data/project/raname /data/project/radomname /data/project/raame /data/project/andomname size of these... (5 Replies)
Discussion started by: h0x0r21
5 Replies

4. Shell Programming and Scripting

How to sort and compare files in more efficient manner?

Hello All, Iam using below method to sort and compare files. First iam doing sorting and changing the same file and then doing comparing and taking the final result to another file. sort -o temp.txt file1 mv temp.txt file1 sort -o temp.txt file2 mv temp.txt file2 sort -o temp.txt... (6 Replies)
Discussion started by: Vikram_Tanwar12
6 Replies

5. Shell Programming and Scripting

Monitor log entries in log files with no Date format? - Efficient logcheck?

is there a way to efficiently monitor logfiles that do not have a date or time format? i have several logs on several different servers that need to be monitored. but i realized writing a script for this would be very complex and time consuming giving the variety of things i need to check for i.e.... (2 Replies)
Discussion started by: SkySmart
2 Replies

6. Emergency UNIX and Linux Support

Help to make awk script more efficient for large files

Hello, Error awk: Internal software error in the tostring function on TS1101?05044400?.0085498227?0?.0011041461?.0034752266?.00397045?0?0?0?0?0?0?11/02/10?09/23/10???10?no??0??no?sct_det3_10_20110516_143936.txt What it is It is a unix shell script that contains an awk program as well as... (4 Replies)
Discussion started by: script_op2a
4 Replies

7. AIX

Efficient coding / Alternates for "For" loop

Hi, I have the current script with the following code and it works ok. for i in `find . -name "???ABCEDFGH*"` do SESS_NO=`echo $i | awk -F "." '{print $3}'` rm -f $LOGDIR/${OT_QUEUEID}*${SESS_NO}.log 2>/dev/null rm -f $i 2>/dev/null done I want to remove the usage of the... (2 Replies)
Discussion started by: jerardfjay
2 Replies

8. Shell Programming and Scripting

Is there an efficient way in finding and deleting files?

Hi I have a script to find and delete the files which are say, noDaysOld, I am interested to find the number of such files I am fniding for deleting and then deleting it. So, the script I wrote, first finds the number of such files and then deletes, clearly this is two different steps. ... (3 Replies)
Discussion started by: guruparan18
3 Replies

9. Shell Programming and Scripting

Can you suggest a more efficient way for this?

Hi I have the following at the end of a service shutdown script used in part of an active-passive failover setup: ### # Shutdown all primary Network Interfaces # associated with failover ### # get interface names based on IP's # and shut them down to simulate loss of # heartbeatd ... (1 Reply)
Discussion started by: mikie
1 Replies

10. Shell Programming and Scripting

Is there a more efficient way?

I'm using korn shell to connect to oracle, retrieve certain values, put them in a list, and iterate through them. While this method works, I can't help but think there is an easier method. If you know of one, please suggest a shorter, more efficient method. ############### FUNCTIONS ... (6 Replies)
Discussion started by: SelectSplat
6 Replies
Login or Register to Ask a Question