UNIX for Dummies Questions & Answers

fyi- When TAR was used from unix shell, the text file is still binary.

Well, all files are binary, even text files.

I guess I can see if you're using tar / pax (either one can read the USTAR format) if you've got a lot of files, but other than that, you should use ASCII ftp transfer if you're moving text files. Depending on the host system, a text file isn't always just an ASCII file. Also, you have to worry about the Endian-ness of the file (byte-ordering). I've had both bite me in the ass before.

I'd be interested to see if your host machine is big-endian, and the client is little-endian.

The problem is the ftp transfer. Different OS's use different ways to terninate a line in a text file. Ascii mode in ftp converts between OS's.

In particular, unix uses a newline character. And DOS/Windows uses a carriage return/line feed. The newline and line feed are the same character. So the difference is the presence or absence of the carriage return.

You don't want to transfer a binary file in ascii mode since some transformations are occur that would corrupt the file. And you don't want to transfer a text file in binary mode because the lines may be terminated wrong.

Here we have a tar archive with both types of files. The best solution is to transfer in binary mode. This will screw up the text files. So after the text files are extracted they must be fixed.

Doing that is a faq...

Can we use sed for this?

windows file to unix :end line chararcter

Convert ASCII to BINARY

replacing \n by \r\n

Very interesting point made about the host system. There is another detail here that I will now expose. I assumed it was not relevant in the problem, but it may be now:

The context in which we are executing the PAX / TAR -x is the IBM mainframe using the USS segment aka HFS file system. It is sort of a 'virtual unix' running on the mainframe.

This might square w/ the other poster on this thread who said he did not have a problem doing mixed format tar/untar's. Maybe he was not going across different operating systems.

m

My mixed tars are indeed going from UNIX (or Linux) to UNIX. Even when I use my Windoze box to upload, I do so under Cygwin which is a Linux emulation layer. So both systems are always using just a line feed to terminate each line.

So that would probably explain why I have never encountered a problem - because the end-of-line/file terminators are identical on both hosts.