Shell Programming and Scripting

Hi All,

I have to merge the data in all the files in a folder such that the data of the earliest file come first then the second file's data and so on. Please help.

Thanks.

Under ksh or bash (sort by date)

Code:

for f in $(ls -tr); do cat "$f" >> /tmp/my_accumulated_data_file; done

if sort by file name just drop -tr from ls

You can use ls with suitable options to generate a list of the directories contents in the proper order. That list can be piped into a while-loop for reading. Use test/[ to verify that the item is indeed a file before cat'ing. Redirect the output of the while-loop to a file of your choice.

Regards,
Alister

Usually it's a very bad thing to use ls instead of find for task like this, but... being really careful, try to see:

Code:

cd mydirectory
ls -1rt | while read item; do [[ -f $item ]] && cat $item >> /somewhereelse/outputfile ; done

That's an unsafe and unreliable way to work with a list of filenames.

Code:

$ mkdir test && cd test
$ touch -- '?' '[a]' a '-n'
$ ls -1
?
a
[a]
-n
$ ls | while IFS= read -r f; do printf '%s\n' "$f"
?
a
[a]
-n
$ for f in $(ls); do echo "$f"; done
?
a
a
a

That for-loop is badly broken. Instead of the four files, only two are printed, one of them three times. Pathname expansion (globbing) after the command substitution replaces ? with ? a and [a] with a, leading to the three instances of a. The -n is treated is silently consumed by echo as an option.

Beware pathname expansion after command substitution. Beware feeding echo arbitrary arguments. printf is safer and portable (and, like echo, usually a shell builtin).

Regards,
Alister

---------- Post updated at 06:14 PM ---------- Previous update was at 06:10 PM ----------

If done correctly, there's absolutely nothing wrong or unsafe about piping ls into a while-read loop. The only text it cannot handle is a newline in a pathname because ls and read use it as their delimiter.

Your example, however, isn't well-constructed. You must use IFS= to prevent trimming of leading/trailing whitespace and -r to treat backslashes literally. To read arbitrary lines of text verbatim: IFS= read -r line

Regards,
Alister

Thanks all for reply..I need to append the timestamp of the file at the end of each record as well...I am new to UNIX,so please provide the code.Thanks for the help..

I knew. When I wrote "be very careful", I meant: I know that a lot of things can go wrong... but at the moment I couldn't say precisely what they are.

I'm just a shell newbie, and bash is only my latest hobby. So please take it into account when I write some nonsense. I'm sorry in advance.

Ok, it's easy to remember. "IFS= ", and read will catch the whole line 100% of times. Thanks.