HI all,
could please help me in this code. I have 560 files containing the same columns but different rows. i want to concatenate all these files in one big file. i want to keep the header of the first file then add the dat from other files horizontally.
the name of my files contains 2 variables : chunk number and chromosome number. this an example

code:
IMP_chunk11-chr6_file

i have tried this code but it works only for one variable and 22 files. anu suggestion to adapt it to my case

code :

for p in 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22
do
more +2 EUR.chr${p}.Fst > EUR.chr${p}.Fst.tmp
done

cp EUR.chr1.Fst EUR.chr1.Fst.tmp

cat EUR.chr*.Fst.tmp > EUR.all.Fst

p=2
k=0
cat EUR.1.fst >EUR.all.fst
    while [ p -lt 561 ]
        do
             while read line
               do
                if [ k -eq 1 ]
                  then
                           echo "$line" >>EUR.all.fst
                  fi
            k=1
        done <EUR.${p}.Fst
       p=`expr $p + 1`
       k=0
done

You didn't tell us if there is any naming convention on the file names, so it is difficult to suggest a loop where you don't have to name the possible file names but instead generate them. In any case the loop you use should NOT be a for-loop, because it can break if a file glob expands to something you didn't foresee.

Instead, use a while-loop, like this:

ls <some-file-glob> | while read filename ; do
     # do something with "$filename"
done

sed -n '3,$ p' inputfile >> collectfile

#! /bin/ksh

typeset outfile="/path/to/output"
typeset infile=""

rm -rf "$outfile" 2>/dev/null >/dev/null

ls *input | while read infile ; do
     if [ -e "$outfile" ] ; then
          sed -n '3,$ p' "$infile" >> "$outfile"
     else
          cp "$infile" "$outfile"
     fi
done

thanks so much Bakunin for your help,
i created a filelist with all the names for the files that i want to concatenate. as you can see attached in "files_name" the names of these files is doest fellow a rules.
how can you script be adjusted to this please
thanks again

Attached Files

files_name.txt (11.6 KB, 5 views)

I think that you can change the above to:

p=2
k=0 
cat file1 >>EUR.all.Fst     
while read file         
do              
while read line                
do                 
if [ k -eq 1 ]                   
   then                            echo "$line" >>EUR.all.fst
 fi             
k=1         
done <$file         
k=0 
done <file.list

If you already have the list in a file, just "cat" it into the loop. The loop just takes one file name in every run, so as long as you provide this one filename all is fine. "ls" would have provided such a list of file names and "cat /path/to/yourlist" can too. Therefore replace:

# ls *input | while read infile ; do
cat /path/to/your/file | while read infile ; do

Just shell and cat:

{
  read filename
  cat "$filename"
  while read filename
  do
     {
       read                 # use "read; read" to skip two header lines
       cat
     } < "$filename"
  done
} < file_name.txt > concatenated_file