merging text files into one


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting merging text files into one
# 1  
Old 02-09-2011
Java merging text files into one

Hi good day
Is there a shell script I can use to join multiple text files in a folder and put the name(excluding the extension) of the text file before it's contents and put EOF at the end of each portion.
so for example, say I have

file1.dat, file2.dat, file3.dat

i'll get
Code:
file1
{text from file1}
EOF
{blank line}
file2
{text from file2}
EOF
{blank line}
file3
{text from file3}
EOF

thanks much.

Last edited by Franklin52; 02-09-2011 at 07:25 AM.. Reason: Please use code tags, thank you
# 2  
Old 02-09-2011
Try:
Code:
awk 'FNR==1{if(NR>1)print x;print FILENAME}1' file[123].dat

# 3  
Old 02-09-2011
Hi,

Test this solution. The last character is '*', witch selects all files in the directory where you run the script.

Code:
$ perl -e 'while (@ARGV) { ($param = $ARGV[0]) =~ s/(.*)\..*/\1/ ; print "$param", "\n", (qx/cat $ARGV[0]/), "EOF\n\n" ; shift @ARGV }' *

Regards,
Birei

Last edited by birei; 02-09-2011 at 09:17 AM..
This User Gave Thanks to birei For This Post:
# 4  
Old 02-09-2011
Sorry but I'm new to this
This doesn’t work. I have a format.sh file in the folder that I run from the shell using “sh format.sh”.
I was hoping to see something like this
Step 1 (or line one of script): ls *.dat | while read file; do awk “magic code “to and insert filename, without extension, into the text plus EOF at end” here” $file > file_formatted ; done
Step 2 (or line two of script): code to join all the file_formatted files that were created in step 1 with a blank line between each.

thanks

---------- Post updated at 07:19 AM ---------- Previous update was at 07:14 AM ----------

now testing code from birei..1 sec

---------- Post updated at 07:20 AM ---------- Previous update was at 07:19 AM ----------

now testing code from birei..1 sec
# 5  
Old 02-09-2011
You can use awk to do it all:
Code:
awk 'FNR==1{if(NR>1)print x;f=FILENAME;sub(/\.[^.]+$/,x,f);print f}1' file[123].dat > outfile

or use shell:
Code:
first=true
for f in file[123].dat
do
  if ! $first; then
    echo
  fi
  first=false
  echo "${f%.*}"
  while IFS= read -r line
  do
    printf "%s\n" "$line"
  done < "$f"
done > outfile

But I would not repeatedly call awk within a loop....

Last edited by Scrutinizer; 02-09-2011 at 08:49 AM..
# 6  
Old 02-09-2011
the perl script from birei works brilliantly....thanks

---------- Post updated at 10:20 AM ---------- Previous update was at 10:15 AM ----------

scrutinizer...thanks also but I am having difficulty doing it with your method...
Although your stuff looks easier to understand....birei stuff works on the first go, so i used it.
# 7  
Old 02-09-2011
Hi you're welcome. Good that you found a solution. For the interest of this thread are you on Solaris? In that case you will find that most awk solutions on this forum will only work with /usr/xpg4/bin/awk instead of awk ( the same goes for sed, grep, and even sh )
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Comparing and merging two text files

Hey everybody, I am new here and already a question to ask, I just recently started some bioinformatic work for my PhD so I am slowly learning Anyway, here is my problem, I have two text files, one contains the complete data file with 43000 genes and their read counts for all my samples... (1 Reply)
Discussion started by: ant55
1 Replies

2. UNIX for Dummies Questions & Answers

Merging two text files by two columns

Hi, I have two text files that I would like to merge/join. I would like to join them if the first columns of both text files match and the second column of the first text file matches the third column of the second text file. Example input: First file: 1334 10 0 0 1 5.2 1334 12 0 0 1 4.5... (4 Replies)
Discussion started by: evelibertine
4 Replies

3. UNIX for Dummies Questions & Answers

Merging two text files by a column

I have two text files. One has two columns and looks like below: rs# otherallele_freq rs10399749 0 rs4030303 0 rs4030300 0 rs940550 1.000 rs13328714 0 rs11490937 0 rs6683466 0 rs12025928 1.000 rs6650104 0 rs11240781 0... (5 Replies)
Discussion started by: evelibertine
5 Replies

4. UNIX for Dummies Questions & Answers

Merging two text files by a column

So I have two text files. The first one looks like this: refsnp_id chr_name chrom_start 1 rs1000000 12 126890980 2 rs10000010 4 21618674 3 rs10000012 4 1357325 4 rs10000013 4 37225069 5 rs1000002 3 183635768 And the second one looks like this: AUC rs1000000 0.03 0.1240 AUC ... (4 Replies)
Discussion started by: evelibertine
4 Replies

5. UNIX for Dummies Questions & Answers

Merging two text files with variable coloumns

Hi All, I have two text files containing space delimited columns. The first file contains 9 columns and the second one contain 3 columns. I want to copy the 3 coloumns from the 2nd file and paste them in 1st file after 9 coloumns. Ex. File1.txt contains 9 coloumns C1 C2 C3 C4 C5 C6 C7 C8 C9... (6 Replies)
Discussion started by: Unilearn
6 Replies

6. Shell Programming and Scripting

AWK Script For Merging Text Files

Hello, I am trying to merge data from two text files. One file (File1) contains a listing of data which includes the trial number in Column 5, while the other text file (File2) contains what category the trial belongs to. Here is a snippet of what File1 looks like. 1 Arrow_ST 9.738 0.905... (2 Replies)
Discussion started by: Jahn
2 Replies

7. UNIX for Dummies Questions & Answers

Merging all text files in the order which they recieved

Hi guru's :) I am new to UNIX. requirement is merging all text files from one folder in the order which they have received. EX: under /Test folder files received in certain intervals. abc.txt @ 1:00 AM 123.txt @ 2:00 AM xyz.txt @ 4:00 AM . . . 456.txt @ 10:00 PM ... (4 Replies)
Discussion started by: arund_01
4 Replies

8. Shell Programming and Scripting

Merging all text files in the order which they recieved

Hi There, I am new to UNIX. requirement is merging all text files from one folder in the order which they have received. EX: under /Test folder files received in certain intervals. abc.txt @ 1:00 AM 123.txt @ 2:00 AM xyz.txt @ 4:00 AM . . . 456.txt @ 10:00 PM so need to develop a... (3 Replies)
Discussion started by: arund_01
3 Replies

9. Shell Programming and Scripting

merging text files

hi, i need to merge 2 lakh text files ..... can somebody please help me with a script/program for it... (8 Replies)
Discussion started by: code19
8 Replies

10. Shell Programming and Scripting

merging few columns of two text files to a new file

hi i need to select a few columns of two txt files and write it to a new file. there is one common field for both of these files. plz help me in this thanks in advance (4 Replies)
Discussion started by: kolvi
4 Replies
Login or Register to Ask a Question