Shell Programming and Scripting

@DGPickett:
What do you mean by this? How?

The construct with xargs -0 works fine on my machine (centos 5.3; GNU bash, version 3.2.25).

That's exactly why I suggested it. Your original reply failed when I ran it on a dir with files containing spaces.

"find *" is relative paths, "find $PWD" is absolute paths.

Absolute paths means lines never match, dir1 to dir2.

My bro said "file names with spaces, only Microsoft would so something stupid like that!" Actually, they are just about as legal in UNIX, just shunned by the culture. Use a '_', '.', '-' or nothing. I like the -, handy and easy to read past.

I agree that including spaces in filenames is a very dirty practice, yet it is not uncommon. My version with xargs -0 worked fine -- I ran it with relative path like this:

Code:

comm -3 < ( cd dir1; find . -type f -print0 | xargs -0 -n999 cksum | 
                     sed 's/\(.*\) \(.*\)/\2 \1/' | sort ) 
        < ( cd dir2;  find . -type f -print0 | xargs -0 -n999 cksum | 
                     sed 's/\(.*\) \(.*\)/\2 \1/' | sort ) | 
sed '
  s/^\t/dir2 /
  t 
  s/^/dir1 /
'

You used find * , but the result should be the same

I disagree. I personally see nothing "stupid" or "dirty" about spaces in filenames. If a system has problems with spaces in filenames, I'm more inclined to look unfavorably on the system than on whoever named the file.

The problem, in my opinion, is that since UNIX filenames are allowed to contain '\t' and '\n', the output from find(1) is not guaranteed to be a decipherable text stream where newlines delimit a record/line and tabs delimit fields. (It doesn't help that the portable subset of xargs features is a bit lacking as well.)

Perhaps the text stream has outlived its usefulness. Time for everyone to migrate to Powershell?

Regards,
Alister

Don't be hasty. There's one and exactly one character that's not allowed to be in a UNIX filename: NULL. if find can be made to print NULL, and xargs can be made to use NULL as a delimiter, that is a 100% safe to delimit filenames.

As it happens GNU find has the -print0 option to print nulls instead of newlines, and GNU xargs has the --null option to use nulls as separators.

If your argument is just that these arguments aren't portable, perhaps it's time to mandate them in POSIX.

There's always -exec, too, which will always be given a correct filename.

The find/xarg -print0/-0 tandem is useful but it's of no help when the output is coming from ls or stat or lsof or any one of countless tools which output filenames.

I do agree with you though that it would be useful for posix xargs to support some means of specifying a delimiter.

My argument is simply that traditional unix tools were designed to pipe text streams but filenames, a central system abstraction, cannot be part of a text stream without the potential for ambiguity/corruption (Is this newline actually the beginning of a new line or part of a foolish filenaming scheme?). Due to this we often have to choose between a simple, 80% solution (which is often sufficient) or a comparatively cumbersome approach.

The design decision was made long ago; it's not going anywhere; and in practice it isn't usually a problem. Still, I think would have been a good decision to have disallowed \t and \n in filenames. <insert visions of {file,path}names shuttling through pipes without a care in the world>

Regards,
Alister

Run, high priests on rampage !

Yes, xargs -0 is nice for spaces in file names, if you have them, and similarly, find . is safer than find *, no meta-in-entry-name vulnerability, if you do not mind the './' prefix, or tell sed to toss it.