I've been trying to figure this out since last night, and I'm just stumped. The last time I did any shell scripting was 8 years ago on a Unix box, and it was never my strong suit. I'm on a Mac running Leopard now. Here's my dilemma - hopefully someone can point me in the right direction.

I'm trying to process product datafeeds - plain text, thousands of lines, pipe delimited. My shell script prepares a directory, then downloads via curl a zip file. It unzips the file. It greps out lines containing a keyword and redirects into a new file. I use a perl command to find/replace my ID number in the file. Everything I've described works perfectly - but the next step is the problem.

I need to add the field names to the top of the file so I can put the file into WebMerge and process it. The field names are on a single line in a separate text file called FirstLine.txt. I use this:

cat FirstLine.txt kids.txt > readyToProcess.txt

to combine the files. What I expect is that if FirstLine.txt contains foo and kids.txt contains bar, then readyToProcess.txt should contain foo(new line)bar. What I'm getting instead is that the file contains foobar.

Is there a situation where the cat command wouldn't put the contents of the first file on a new line?

Thanks for any help you can offer - I'm rather stuck until I figure this out!

~Daniel

Make sure that there is a proper line terminator in FirstLine.txt. cat doesn't put that in for you, it just does a 1:1 output.

Ah-ha. Okay, I think I'm onto something... I spent some time researching the line terminator thing (have been since last night actually) but everything I came up with kept telling me that my line terminator was Unix-style. Just now though, I punched file FirstLine.txt into Terminal and it came back with FirstLine.txt: ASCII text, with no line terminators.

Great. So... how do I insert a line terminator? I found many, many resources that talk about the tr command and changing \r to \n but nothing in the way of adding a line terminator where none exists.

Open the file in your preferred text editor, hit <End>, press <Enter>, save, close.
Or, on the command line, enter printf "<lt>" >> FirstLine.txt, where <lt> is one of the following:

• DOS/Windows: \r\n
• UNIX (including Linux, BSD, not OSX): \n
• MacOS: \r

I was hoping you wouldn't say that See, I figured there must be some trick or something, because when I simply add a new line like that (by opening in TextWrangler and hitting Enter at the end of the line), my output gets... weird.

This is a short example of what it was doing when I first posted.

FirstLine.txt:kids.txt:cat FirstLine.txt kids.txt > readyToProcess.txtAfter adding the new line to FirstLine.txt, I try cat'ing again, but this time, the resulting readyToProcess.txt file looks like this:So, it added the first line great - but why on earth would it throw an extra line break between all the rest of the lines in the file?

Also - I ran printf "\n" >> FirstLine.txt and the result was the same as when I open the file in TextWrangler and add the new line manually. Oddly, when I cleared that new line and ran printf "\r" >> FirstLine.txt the output is almost correct. I get this:There's an extra space at the beginning of every line starting with line 3 which I've noted with [] because vBulletin doesn't allow a space at the beginning of a line in a quote tag

That's strange. You could try to edit it with VIm, adding an empty line and deleting it again, as it usually adds the appropriate EOL when saving a file. Also, if you enter ':set list', you'll see any special characters (eg. $ for EOL, ^I for tab, ...)