Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: For loop in process each file
Top Forums UNIX for Dummies Questions & Answers For loop in process each file Post 302943837 by bakunin on Wednesday 13th of May 2015 08:53:06 AM
Old 05-13-2015
In addition to what sea said (and to what i wholeheartedly agree):

1) do not use "open-ended" for-loops, because they can have side-effects. Something like (you are effectively doing the same)

Code:
for VAR in <list of filenames> ; do
     do_something_with "$VAR"
done

will work with several or several hundreds of files but might break with a "too many arguments"-error or a "input line too long"-error with several thousands of files. Do a

Code:
ls -1 | grep <some-filter-criteria> |\
while read VAR ; do
     do_something_with "$VAR"
done

instead. This will work with even the biggest amount of files.

2) Never, NEVER use backticks any more. They are outdated and only there for backwards compatibility with other outdated software. Use the POSIX-compliant

Code:
$(command | command2 | .... )

instead.

Btw.: i am not sure what you are trying to do but to me this seems to be a search for symbolic links. This can more easily be done by:

Code:
find /start/dir -type l -print

see the man page for "find" for details.



I hope this helps.

bakunin
 

10 More Discussions You Might Find Interesting

1. AIX

loop through the directory for files and sort by date and process the first file

hello i have a requirement where i have a direcotry in which i get files in the format STOCKS.20080114.dat STOCKS.20080115.dat STOCKS.20080117.dat STOCKS.20080118.dat i need to loop through the directory and sort by create date descending order and i need to process the first file. ... (1 Reply)
Discussion started by: dsdev_123
1 Replies

2. Shell Programming and Scripting

Check for statup process in loop?

I've written a script to check for Oracle's listener, eventman and pmon processes however there are several databases that startup which can take several minutes. I'd like to add code to my current script that greps for the process “startup” and whether its condition is true or false. If the... (1 Reply)
Discussion started by: dataciph3r
1 Replies

3. Shell Programming and Scripting

loop when process running

Hi Gurus, Could you please help me to create a shell script that will be started by a cron job once every night at 24.00 h (that should bee easy:)) The shell script should control every 30 seconds the name of a process, and when the process doesn't run anymore it should execute a few further... (12 Replies)
Discussion started by: blackwire
12 Replies

4. Shell Programming and Scripting

How to loop this process?

for two txt files, f1 and f2, I like to do the following grep "abcde" f1 > abcde$f1 grep "xyz" f1 > xyz$f1 can I use a loop to get this done? Thanks for i in f1 f2 do grep "abcde" $i > abcde$i grep "xyz" $i > xyz$i ... done (11 Replies)
Discussion started by: ksgreen
11 Replies

5. Shell Programming and Scripting

Process checking loop

Hi, I want to create a script who will check if the java process is running & if it finds the process is still there it continues to execute & when the process completes it exit from the script. I have written a code to check & notify the process existence but i am not getting how to write... (4 Replies)
Discussion started by: d8011
4 Replies

6. Shell Programming and Scripting

Loop to process 2 files with same name in different path

Hello forum members, I hope you can help me with this I don't know hot to reach. I have a list of files in "/home/MyPath1/" and in "/home/MyPath2/". The files have the same name in both folders. (but different content, the content doesn't matter here I think) /home/MyPath1/ filename1.txt... (4 Replies)
Discussion started by: Ophiuchus
4 Replies

7. Shell Programming and Scripting

need to process for loop faster

I have the following code running against a file. The file can have upwards of 10000 lines. problem is, the for loop takes a while to go through all those lines. is there a faster way to go about it? for line in `grep -P "${MONTH} ${DAY}," file | ${AWK} -F" " '{print $4}' | awk -F":"... (2 Replies)
Discussion started by: SkySmart
2 Replies

8. UNIX for Dummies Questions & Answers

Timed loop to scan for a process

Hi all, Looking for some help, I have written a very simple script to pass to PowerHA to act as an indicator to activate failover required. Where i get completely lost is I have to create a wait and carry out the grep for the process once every 10 seconds this works but need to embed this... (1 Reply)
Discussion started by: Gary Hay
1 Replies

9. Shell Programming and Scripting

Process a file for line count using for loop in awk

Hi, I have a file with contents So what I have to do is In short, break the file after every 6 lines and then truncate new line to tab for these 6 lines. I am not able to execute the for loop in awk properly. The idea is something like this: less file| awk '{for .... {if ((nr>=1)... (7 Replies)
Discussion started by: rossi
7 Replies

10. Shell Programming and Scripting

Script to loop process

As I would like to test the open files usage , I would like to have a process that use the open files up to a certain amount eg. 1000 . If I want to have a script ( may be run in a loop ) that could repeatly use open files resource , so that the usage of open files increases , may I know how to... (10 Replies)
Discussion started by: ust
10 Replies

LEARN ABOUT DEBIAN

bup-margin

bup-margin(1)						      General Commands Manual						     bup-margin(1)

NAME

       bup-margin - figure out your deduplication safety margin

SYNOPSIS

       bup margin [options...]

DESCRIPTION

       bup margin  iterates  through  all  objects  in	your  bup repository, calculating the largest number of prefix bits shared between any two
       entries.  This number, n, identifies the longest subset of SHA-1 you could use and still encounter a collision between your object ids.

       For example, one system that was tested had a collection of 11 million objects (70 GB), and bup margin returned 45.  That  means  a  46-bit
       hash  would be sufficient to avoid all collisions among that set of objects; each object in that repository could be uniquely identified by
       its first 46 bits.

       The number of bits needed seems to increase by about 1 or 2 for every doubling of the number of objects.  Since SHA-1 hashes have 160 bits,
       that  leaves 115 bits of margin.  Of course, because SHA-1 hashes are essentially random, it's theoretically possible to use many more bits
       with far fewer objects.

       If you're paranoid about the possibility of SHA-1 collisions, you can monitor your repository by running bup margin occasionally to see	if
       you're getting dangerously close to 160 bits.

OPTIONS

       --predict
	      Guess  the offset into each index file where a particular object will appear, and report the maximum deviation of the correct answer
	      from the guess.  This is potentially useful for tuning an interpolation search algorithm.

       --ignore-midx
	      don't use .midx files, use only .idx files.  This is only really useful when used with --predict.

EXAMPLE

	      $ bup margin
	      Reading indexes: 100.00% (1612581/1612581), done.
	      40
	      40 matching prefix bits
	      1.94 bits per doubling
	      120 bits (61.86 doublings) remaining
	      4.19338e+18 times larger is possible

	      Everyone on earth could have 625878182 data sets
	      like yours, all in one repository, and we would
	      expect 1 object collision.

	      $ bup margin --predict
	      PackIdxList: using 1 index.
	      Reading indexes: 100.00% (1612581/1612581), done.
	      915 of 1612581 (0.057%)

SEE ALSO

       bup-midx(1), bup-save(1)

BUP

       Part of the bup(1) suite.

AUTHORS

       Avery Pennarun <apenwarr@gmail.com>.

Bup unknown-															     bup-margin(1)
All times are GMT -4. The time now is 03:54 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy