For loop in process each file


 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers For loop in process each file
# 1  
Old 05-13-2015
For loop in process each file

Hi

I have following code
Code:
cd /tmp/test/
for vfile in `ls -1`           
   do                             
     for vlink in `ls -l /tmp/testfile/*|bin/grep "local/init\.d/$vfile$"|bin/awk -F"->" '{print($1)}'|bin/awk -F"/" '{print($NF)}'`

I know `ls -1` list only file, but I don't understand second for loop,

I got below o/p when run ls -1
Code:
abc
def
ghi

So I tried to manually run
Code:
ls -l /tmp/testfile/*|bin/grep "local/init\.d/abc"|bin/awk -F"->" '{print($1)}'|bin/awk -F"/" '{print($NF)}'

but I am not getting any o/p

The program checking the link file

Last edited by rbatte1; 05-13-2015 at 08:42 AM.. Reason: Added ICODE tags and tightened CODE tags
# 2  
Old 05-13-2015
You are grep'ing for local/init.d/abc, but ls output is only abc, so it cant find it.
grep only for abc, and you should see output.
Also, your first code has no closing done.

Furthermore, you are calling bin/grep and bin/awk so you are expecting them to be in the subdrir bin of your current path.

hth
This User Gave Thanks to sea For This Post:
# 3  
Old 05-13-2015
In addition to what sea said (and to what i wholeheartedly agree):

1) do not use "open-ended" for-loops, because they can have side-effects. Something like (you are effectively doing the same)

Code:
for VAR in <list of filenames> ; do
     do_something_with "$VAR"
done

will work with several or several hundreds of files but might break with a "too many arguments"-error or a "input line too long"-error with several thousands of files. Do a

Code:
ls -1 | grep <some-filter-criteria> |\
while read VAR ; do
     do_something_with "$VAR"
done

instead. This will work with even the biggest amount of files.

2) Never, NEVER use backticks any more. They are outdated and only there for backwards compatibility with other outdated software. Use the POSIX-compliant

Code:
$(command | command2 | .... )

instead.

Btw.: i am not sure what you are trying to do but to me this seems to be a search for symbolic links. This can more easily be done by:

Code:
find /start/dir -type l -print

see the man page for "find" for details.



I hope this helps.

bakunin
# 4  
Old 05-14-2015
Quote:
Originally Posted by bakunin
... ... ...

1) do not use "open-ended" for-loops, because they can have side-effects. Something like (you are effectively doing the same)

Code:
for VAR in <list of filenames> ; do
     do_something_with "$VAR"
done

will work with several or several hundreds of files but might break with a "too many arguments"-error or a "input line too long"-error with several thousands of files. Do a

Code:
ls -1 | grep <some-filter-criteria> |\
while read VAR ; do
     do_something_with "$VAR"
done

instead. This will work with even the biggest amount of files.

... ... ...

bakunin
When you invoke a utility with a long argument list (or a large list of environment variables) you can get an E2BIG error from the exec*() or posix_spawn() call. A for loop is run entirely in the shell (no exec or spawn involved) so you can't get an argument list too long error from the for.

So, ls * | while IFS= read -r i may get an error if there are a lot of files in the current directory (due to an exec*() failure while invoking ls), but for i in * should not. But, of course (especially on 32-bit systems), the shell could run out of space trying to gather the arg list and fail with an ENOMEM error. So I wouldn't try a command like:
Code:
for i in $(find / -print)
do	whatever
done

when the command:
Code:
find / -print | while IFS= read -r i
do	whatever
done

will be much more likely to work and (besides requiring much less memory for the shell) will also have the advantage of running the find and the loop processing in parallel.

Note also that standards say that shell scripts (and standard input for an interactive shell) are text files with unlimited line lengths. So you shouldn't ever get a line too long error when processing the arguments to a for loop.
 
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Script to loop process

As I would like to test the open files usage , I would like to have a process that use the open files up to a certain amount eg. 1000 . If I want to have a script ( may be run in a loop ) that could repeatly use open files resource , so that the usage of open files increases , may I know how to... (10 Replies)
Discussion started by: ust
10 Replies

2. Shell Programming and Scripting

Process a file for line count using for loop in awk

Hi, I have a file with contents So what I have to do is In short, break the file after every 6 lines and then truncate new line to tab for these 6 lines. I am not able to execute the for loop in awk properly. The idea is something like this: less file| awk '{for .... {if ((nr>=1)... (7 Replies)
Discussion started by: rossi
7 Replies

3. UNIX for Dummies Questions & Answers

Timed loop to scan for a process

Hi all, Looking for some help, I have written a very simple script to pass to PowerHA to act as an indicator to activate failover required. Where i get completely lost is I have to create a wait and carry out the grep for the process once every 10 seconds this works but need to embed this... (1 Reply)
Discussion started by: Gary Hay
1 Replies

4. Shell Programming and Scripting

need to process for loop faster

I have the following code running against a file. The file can have upwards of 10000 lines. problem is, the for loop takes a while to go through all those lines. is there a faster way to go about it? for line in `grep -P "${MONTH} ${DAY}," file | ${AWK} -F" " '{print $4}' | awk -F":"... (2 Replies)
Discussion started by: SkySmart
2 Replies

5. Shell Programming and Scripting

Loop to process 2 files with same name in different path

Hello forum members, I hope you can help me with this I don't know hot to reach. I have a list of files in "/home/MyPath1/" and in "/home/MyPath2/". The files have the same name in both folders. (but different content, the content doesn't matter here I think) /home/MyPath1/ filename1.txt... (4 Replies)
Discussion started by: Ophiuchus
4 Replies

6. Shell Programming and Scripting

Process checking loop

Hi, I want to create a script who will check if the java process is running & if it finds the process is still there it continues to execute & when the process completes it exit from the script. I have written a code to check & notify the process existence but i am not getting how to write... (4 Replies)
Discussion started by: d8011
4 Replies

7. Shell Programming and Scripting

How to loop this process?

for two txt files, f1 and f2, I like to do the following grep "abcde" f1 > abcde$f1 grep "xyz" f1 > xyz$f1 can I use a loop to get this done? Thanks for i in f1 f2 do grep "abcde" $i > abcde$i grep "xyz" $i > xyz$i ... done (11 Replies)
Discussion started by: ksgreen
11 Replies

8. Shell Programming and Scripting

loop when process running

Hi Gurus, Could you please help me to create a shell script that will be started by a cron job once every night at 24.00 h (that should bee easy:)) The shell script should control every 30 seconds the name of a process, and when the process doesn't run anymore it should execute a few further... (12 Replies)
Discussion started by: blackwire
12 Replies

9. Shell Programming and Scripting

Check for statup process in loop?

I've written a script to check for Oracle's listener, eventman and pmon processes however there are several databases that startup which can take several minutes. I'd like to add code to my current script that greps for the process “startup” and whether its condition is true or false. If the... (1 Reply)
Discussion started by: dataciph3r
1 Replies

10. AIX

loop through the directory for files and sort by date and process the first file

hello i have a requirement where i have a direcotry in which i get files in the format STOCKS.20080114.dat STOCKS.20080115.dat STOCKS.20080117.dat STOCKS.20080118.dat i need to loop through the directory and sort by create date descending order and i need to process the first file. ... (1 Reply)
Discussion started by: dsdev_123
1 Replies
Login or Register to Ask a Question