Combining multiple files into one with the same name/different extension
I've been trying to find information in regard to creating a script that will generate HTML files. I currently have a series of files that contain code I need to surround with a <textarea> tag for easy viewing. I have about a thousand files that contain code, one file that contains the HTML code up to the textarea tag, and then a second file that contains everything from /textarea down. I need a script that will perform this process:
1) Concatenate the contents of file1.txt into an HTML file
2) Concatenate the contents of a code file into the HTML file
3) Concatenate the contents of file2.txt into the HTML
4) Set the HTML file name to the same name as the code file in step 2 with the .html extension
5) Move onto the next file
I looked around online and with a little help from another forum, I wound up with this:
Code:
#!/bin/bash
FILES="*"
for f in "$FILES"
do
echo "Processing $f file..."
cat file1.txt $f file2.txt > $f.html
cat $f
done
Naturally, this doesn't work. Can anyone help me figure out why or perhaps offer a better way? I'm rather new to Linux, so I'd appreciate if you could be as specific and descriptive as possible. Thank you in advance.
Thank you for the reply. When I run the script, it creates one file. I'm looking for one HTML file per code file. Apparently I need the script to run equal to the number of files I have in the directory. So using your example:
| HTML HEADER | + | Firstfile.c | + | HTML tails to first two | into Firstfile.html
| HTML HEADER | + | Secondfile.c | + | HTML tails to first two | into Secondfile.html
...
...
| HTML HEADER | + | Lasttfile.c | + | HTML tails to first two | into Lastfile.html
---------- Post updated at 02:16 ---------- Previous update was at 02:05 ----------
[/COLOR]
Quote:
Originally Posted by 12o
When I run the script, it creates one file.
Code:
for f in "$FILES"
If you quote $FILES than the "*" does not get "translated" into filenames.
Remove the quotes and it will work.
Besides I think the "cat $f" line is unnecessary.
When the script ends, you will need to manually delete file1.txt.html, file2.txt.html and nameofyourscript.txt.html file.
Last edited by pseudocoder; 04-29-2010 at 11:40 PM..
$ for i in `ls | egrep -v -e '(file1.txt|file2.txt)'`
> do
> echo "Processing $i file..."
> cat file1.txt $i file2.txt > $i.html
> done
$
I don't mean to nitpick your contribution, but I just wanted to point out an easy way to improve its reliability and efficiency.
That for loop list cannot handle filenames that include IFS characters (by default, this includes spaces, tabs, and newlines). Also, the egrep will not strictly match a file named "file1.txt" or "file2.txt"; "afile1.txt" would match as well.
I would suggest the following alternative, which remedies both issues and doesn't require external executables:
Code:
for i in *; do
case $i in
file[12].txt) continue;;
esac
... further processing ...
done
First create the fixed header and tail files (text in my case, HTML in your case...)
Code:
$ echo "I am the header" > header.txt
$ echo "I am the tail" > tail.txt
$ cat header.txt
I am the header
$ cat tail.txt
I am the tail
$ for f in {1..10}; do echo "file name=$f.txt">$f.txt; done
$ cat {1..10}.txt
file name=1.txt
file name=2.txt
file name=3.txt
file name=4.txt
file name=5.txt
file name=6.txt
file name=7.txt
file name=8.txt
file name=9.txt
file name=10.txt
Next combine the header, body (1-10), and tail into new files (1-10) with a new extension:
Code:
$ for f in {1..10}.txt; do cat header.txt "$f" tail.txt > "${f%.txt}.html"; done
$ cat {1..10}.html
I am the header
file name=1.txt
I am the tail
I am the header
file name=2.txt
I am the tail
...
I am the header
file name=10.txt
I am the tail
$
The line "${f%.txt}.html" changes the file name extension for the output. Bash substring replacement is documented HERE
---------- Post updated at 07:17 PM ---------- Previous update was at 05:41 PM ----------
Quote:
Originally Posted by pseudocoder
If you quote $FILES than the "*" does not get "translated" into filenames.
Remove the quotes and it will work.
$ for i in "*two*.txt"; do ls -l $i; done
-rw-r--r-- 1 andrew andrew 0 Apr 29 18:32 3 two 1.txt
-rw-r--r-- 1 andrew andrew 0 Apr 29 18:29 one two three.txt
# notice that the * glob expands inside the "*two*.txt"
# you can also use: for i in *two*.txt; do ls -l "$i"; done
# or
$ string="one 2 three 4 five 6 seven ocho"
$ for i in $string; do echo $i; done
one
2
three
4
five
6
seven
ocho
# "$string" is broken on spaces (IFS) and the loop is executed 8 times
$ for i in "$string"; do echo $i; done
one 2 three 4 five 6 seven ocho
# $string is quoted so no break on IFS on the value of $string
for i in '$string'; do echo $i; done
$string
# single quotes, no variable expansion....
Finally, if you do not quote certain forms, you will get unexpected results entirely, such as with a regex:
Code:
$ for i in [[:digit:]]*.txt; do ls -l $i; done
ls: 1: No such file or directory
ls: 2: No such file or directory
ls: 3.txt: No such file or directory
ls: 1.txt: No such file or directory
ls: 3: No such file or directory
ls: two: No such file or directory
ls: 10: No such file or directory
ls: 11.txt: No such file or directory
ls: 9: No such file or directory
# the unquoted regex with POSIX [:digit:] character class found the files but then the shell breaks the return on IFS...
$ for i in "[[:digit:]]*"; do ls -l $i; done
-rw-r--r-- 1 andrew andrew 0 Apr 29 18:59 1 2 3.txt
-rw-r--r-- 1 andrew andrew 0 Apr 29 18:59 3 two 1.txt
-rw-r--r-- 1 andrew andrew 0 Apr 29 18:59 9 10 11.txt
#alternatively:
$ for i in [[:digit:]]*; do ls -l "$i"; done
-rw-r--r-- 1 andrew andrew 0 Apr 29 18:59 1 2 3.txt
-rw-r--r-- 1 andrew andrew 0 Apr 29 18:59 3 two 1.txt
-rw-r--r-- 1 andrew andrew 0 Apr 29 18:59 9 10 11.txt
# BUT! quoting both does not work:
$ for i in "[[:digit:]]*"; do ls -l "$i"; done
ls: [[:digit:]]*: No such file or directory
So long story short:
1) Globs [*?] glob inside double or single quotes at the shell to expand file names,
2) Variable expansion happens inside double quotes but not single quotes.
Alister: PLEASE correct that if incorrect!!! :-}}
Quote:
Originally Posted by pseudocoder
When the script ends, you will need to manually delete file1.txt.html, file2.txt.html and nameofyourscript.txt.html file.
Hello Unix gurus,
I have a large number of files (say X) each containing two columns of data and the same number of rows.
I would like to combine these files to create a unique merged file containing X columns corresponding to the second column of each file (with a bonus of having the first... (3 Replies)
Hello Everyone,
I have 4 different files (one column in each) that I'm trying to combine into 1 file with four columns. Having issues trying to get the columns to format properly. I have tried the following:
paste file1 file2 file3 file4 | column -s $'\t' -t > results.txt
paste file1 file2... (1 Reply)
Hi
I have a file with 100 million rows. I want to split them into 1000 subfiles and name them from 1.xls to 1000.xls.. Can I do it in awk?
Thanks, (8 Replies)
I have 2 files.
each having 3 coloums
1st field date as 20130322
2nd field time as 05:55
3rd field numberic value
File 2 has entries missing for some date time.
FILE1
20130322 05:35 2219
20130322 05:40 1809
20130322 05:45 1617
20130322 05:50 ... (2 Replies)
Hello All,
I have several column files like this
$cat a_b_s1.xls
1wert
2tg
3asd
4asdf
5asdf
$cat c_d_s2.xls
1wert
2tg
3asd
4asdf
5asdf
desired put put
$cat combined.txt
s1 s2 (2 Replies)
I'm trying to combine colums from multiple file to a single file but having some issues, appreciate your help.
The filenames are the same except for the extension,
path1.m0
---------
a b c
d e f
g h i
path1.m1
---------
m n o
p q r
s t u
File names are path1.m
The... (3 Replies)
Situation:
Our system currently executes a job (COBOL Program) that generates an interface file to be sent to one of our vendors. Because this system processes information for over 100,000 employees/retirees (and growing), we'd like to multi-thread the job into processing-groups in order to... (4 Replies)
All,
I want to combine multiple files in one file.
Something like what we do on the commad line as follows ->
cat file1 file2 file3 > Main_File.
Can something like this be done in a perl script very efficiently?
Thanks,
Rahul. (1 Reply)
I have a good script to rename multiple files, but what's the best way I can remove some text from multiple filenames? Say I have a directory with 35 files with a .XLS at the end, how can I rename them to remove the .XLS but keep everything the same, without having to mv manually. Thanks. (6 Replies)
04-29-2010
