No offense taken. I was just making sure that my response didn't come across as petty. So, now that we're done being nice, back to business
Quote:
Originally Posted by drewk
You did not comment on the ASCIZ string form that I stated was an alternative. I do think that ASCIZ strings are better in many cases, especially when using find in the form of:
Code:
find . -type f -maxdepth X -print0 | while read -d $'\0' file
do
do stuff
done
I don't use bash very often so I've never used that read option before. Usually, in similiar situations I make do with xargs. However, looking at it, I spot two bugs (ironically, one of them is a field splitting bug) in that code. Check it out:
Code:
#Create a directory whose name begins with leading spaces
$ mkdir ' 8spaceslater'
#Place two files in that directory, with the first one ending in a backslash
$ touch \ \ \ \ \ \ \ \ 8spaceslater/1\\
$ touch \ \ \ \ \ \ \ \ 8spaceslater/2
#Now let's try that code
$ find ' 8spaceslater' -type f -print0 | while read -d $'\0' file; do echo "$file"; done
8spaceslater/1
What happened to the leading spaces? The leading spaces are lost during the field splitting step. Let's disable field splitting by setting IFS to an empty string:
Code:
$ find ' 8spaceslater' -type f -print0 | while IFS='' read -d $'\0' file; do echo "$file"; done
8spaceslater/1
Now that we got our leading white space through intact, what about the trailing backslash of "1\" and the second file, "2"? The backslash is consumed as part of an escape sequence which results in a null byte in $file, which is why the second file does not appear in the output (a nullbyte marks the end of a string in C). This backslash escape sequence processing can be disabled by passing read the -r option, which enables raw mode.
For maximum "robustness":
Code:
$ find ' 8spaceslater' -type f -print0 | while IFS='' read -rd $'\0' file; do echo "$file"; done
8spaceslater/1\
8spaceslater/2
In this particular example, the only advantage to using null byte delimiters is proper handling of filenames containing linefeeds. If that's not an issue, nothing is gained.
... snip a bunch above ....
In this particular example, the only advantage to using null byte delimiters is proper handling of filenames containing linefeeds. If that's not an issue, nothing is gained.
Regards,
Alister
Wow! That was immensely educational and valuable to me.
There are inconsistencies in the BASH handling of leading spaces when using a "while read -d" as you pointed out. I had not been using the "while IFS = '' read -rd" form and so leading spaces in files names would not have been handled. I had always assumed that it was the same as using xargs.
It is not just leading spaces that are at issue. Consider:
Code:
#create three challenging directories:
$ mkdir {' 3spacesthere','dir?name','spaces in middle'}
#create challenging file names in the directories:
$ touch {' 3spacesthere','dir?name','spaces in middle'}/{1,1\\,2}
#find the '1's -- my earlier example:
$ find . -type f -maxdepth 2 -name "1*" -print0 | while read -d $'\0' file; do printf "%s\n" "$file"; done
./ 3spacesthere/1
./ 3spacesthere/1
./dir?name/1
#WHOOPS! No backslash printed and no 'dir?name' or 'space in middle' files! 83% failure...
#try in 'raw' mode:
$ find . -type f -maxdepth 2 -name "1*" -print0 | while read -rd $'\0' file; do printf "%s\n" "$file"; done
./ 3spacesthere/1
./ 3spacesthere/1\
./dir?name/1
./dir?name/1\
./spaces in middle/1
./spaces in middle/1\
#mostly there, except:
$ find ' 3spacesthere' 'dir?name' 'spaces in middle' -type f -maxdepth 2 -name "1*" -print0 | while read -rd $'\0' file; do printf "%s\n" "$file"; done
3spacesthere/1
3spacesthere/1\
dir?name/1
dir?name/1\
spaces in middle/1
spaces in middle/1\
#No Leading spaces on ' 3spacesthere' rendering the string unusable for a file name...
# Try with IFS set to ''
$ find ' 3spacesthere' 'dir?name' 'spaces in middle' -type f -maxdepth 2 -name "1*" -print0 | while IFS='' read -rd $'\0' file; do printf "%s\n" "$file"; done
3spacesthere/1
3spacesthere/1\
dir?name/1
dir?name/1\
spaces in middle/1
spaces in middle/1\
#OK....
#xargs works fine:
$ find ' 3spacesthere' 'dir?name' 'spaces in middle' -type f -maxdepth 2 -name "1*" -print0 | xargs -0 printf "%s\n"
3spacesthere/1
3spacesthere/1\
dir?name/1
dir?name/1\
spaces in middle/1
spaces in middle/1\
The form of "for" loop with globbing is more robust than I have given it credit for. Consider:
Code:
$ for f in {' 3spacesthere','dir?name','spaces in middle'}/1*; do printf "%s\n" "$f"; done
3spacesthere/1
3spacesthere/1\
dir?name/1
dir?name/1\
spaces in middle/1
spaces in middle/1\
#Just what it is supposed to be the first time...
The PROBLEM one is:
$ for f in `find ' 3spacesthere' 'dir?name' 'spaces in middle' -type f -maxdepth 2 -name "1*" -print0`; do printf "%s\n" "$f"; done
3spacesthere/1
3spacesthere/1\
dir?name/1
dir?name/1
spaces
in
middle/1
spaces
in
middle/1\
#disater...
I have been using 'while read -d' form of loop for a long time thinking it was the best around for this situation. I also have a personal preference for "while" instead of "for" if the looping quantity seems more ethereal. I have also had a mild suspicion of the 'for i in *' as being less robust than you have demonstrated it to be since I knew that ` produced disaster.
#!/bin/sh
sqlplus -s "/ as sysdba" << EOF
SET HEADING OFF
SET FEEDBACK OFF
Select
pt.user_concurrent_program_name , OUTFILE_NAME
FROm
apps.fnd_concurrent_programs_tl pt,
apps.fnd_concurrent_requests f
where
pt.concurrent_program_id = f.concurrent_program_id
and pt.application_id =... (1 Reply)
Hey guys,
I have wrote the following script to apply a module named "trinity" on my files. (it takes two input files and spit a trinity.fasta as output)
#!/bin/bash -l
#SBATCH -p node
#SBATCH -A <projectID>
#SBATCH -n 16
#SBATCH -t 7-00:00:00
#SBATCH --mem=128GB
#SBATCH --mail-type=ALL... (1 Reply)
Hi,
I want to rename several files like this:
example:
A0805120817.BHN
A0805120818.BHN
.....
to:
20120817.0805.N
20120818.0805.N
......
How can i do this via terminal or in shell bash script ?
thanks, (6 Replies)
Hi,
In my directory I have many files, for e.g.
file_123
file_124
file_125
file_126
file_127
Instead of renaming these files one by one, I would like to rename them at a same time using same command... they should appear like
123
124
125
126
127
What command(awk or ls or... (3 Replies)
Hi
I need to reanme the multiple file using unix script
I have multiple file
like:
sample_YYYYMMDD.xls
test new_YYYYMMDD.xls
simple_YYYYMMDD.xls
I need to rename this file
sample.xls
testnew.xls
SIMPLE.xls
thanks (8 Replies)
Hello,
I am having 1800 files in a directory with a specified format, like
amms_850o_prod.000003uNy
amms_850o_prod.000003u8x
amms_850o_prod.000003taP
amms_850o_prod.000003tKy
amms_850o_prod.000003si4
amms_850o_prod.000003sTP
amms_850o_prod.000003sBg
amms_850o_prod.000003rvx... (12 Replies)
Hi, i need a bit of help writting a tcsh script which renames all ascii text files in the current directory by adding a number to their names before the extension
so for example, a directory containing the files
Hello.txt
Hello.t
Hello
should have the following changes,
Hello.txt... (2 Replies)
Hey Guys....
Just need some help as I am not proficient in Unix shell script...
Doubt:
---------------
Suppose there will be some of the following files inside a directory called OUT ...
Path: - /appdb1/product/batch/rms/OUT
files inside OUT directory:-
POSU_75002_20090127_20090129035442... (4 Replies)
Hi,
can anyone have a ksh script to rename multiple files (ie to remove .Z extension of the files)
can someone correct this?
for i in *.Z
do
var1 = substr($i, 1,at(".Z",$i)-1)
mv $i $var1
done
Thanks..
Antony (13 Replies)
03-04-2010
