Find, Append, Move & Rename Multiple Files


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Find, Append, Move & Rename Multiple Files
# 1  
Old 08-25-2007
Find, Append, Move & Rename Multiple Files

Using a bash script, I need to find all files in a folder "except" the newest file. Then I need to insert the contents of one text file into all the files found. This text needs to be placed at the beginning of each file and needs a blank line between it and the current contents of the file. Then I need to move all those modified files to another folder and change the extension portion of their name from .log to .101

My bash skills are absolutely nil. I have used find and can find all the files I want and I have managed to use paste to paste into "1" file but cannot figure out how to do this entire routine for all the found files.

They switched over to linux here and the windows app that creates these files is now running on wine. For some reason the dos batch file I used to do this routine fails to work on wine.
# 2  
Old 08-25-2007
if you are ok, post your batch code. There's equivalent unix commands you can use and people here may guide you on which ones to use.. then you can start to learn shell...
# 3  
Old 08-26-2007
cudnt understand ur queries...
just paste ur code to get an idea what u are trying to do...
# 4  
Old 08-26-2007
Here's the dos batch code. I note that in my original post that I failed to mention that the original files I am working with also get moved off to an archive, unmodified. There's never a filename conflict because each original file has a date name ( s082607.log, etc ). I've explained each step in the process. I have no doubt this dos batch could be more efficient but I am no dos batch pro. The script did work correctly for over 3 years in both NT4 and Win2K. It just doesn't run under wine NT4 or Win2k emulation even though the drive and directory layouts remain the same. Perhaps the wine cmd interpreter doesn't like the code or something.

Code:
:::: Find all logs except most recent ::::
for /f "skip=1 delims=" %%a in ('dir /a-d /b /o-d "d:\buzzard\logs\s-series\*.log" ')

:::: Append a text header message to each found file and copy to a temp dir :::::
do copy d:\buzzard\scripts\addon.txt+d:\buzzard\logs\s-series\"%%a" d:\buzzard\temp\sendlogs\s-series\"%%a"

::::: Rename modified files :::::
ren d:\buzzard\temp\sendlogs\s-series\*.log *.101

::::: Move the modified files to a mail queue for future delivery ::::
move d:\buzzard\temp\sendlogs\s-series\*.101 e:\promail\queue\

::::: Move originals - except most recent - off to an archive :::::
for /f "skip=1 delims=" %%b in ('dir /a-d /b /o-d "d:\buzzard\logs\s-series\*.log" ') do move d:\buzzard\logs\s-series\"%%b" "f:\buzzard\logs\s-series\

# 5  
Old 08-29-2007
Well, I have made a bit of headway in converting this MS-DOS batch over to bash. This seems to work from a terminal. Have yet to try it in a script. I have no doubt there's a much easier, much shorter, and more correct way to do all this but it's what I've figured out so far. There's also one problem that arises concerning the pasting to append a glue header to the logs.

Code:
::: Find all files except the current file and move to a temp dir :::
cd /home/trapper/.wine/drive_d/buzzard/logs/s-series
find . -daystart -mtime +0 -exec mv {} /home/trapper/.wine/drive_d/buzzard/temp \;

Code:
::: Paste glue header and each log file to new files in the queue :::
for i in /home/trapper/.wine/drive_d/buzzard/temp/*.log; 
do paste -s /home/trapper/.wine/drive_d/buzzard/addon.txt $i >> /home/trapper/.wine/drive_d/promail/queue/${i/.log}.101;

Code:
::: Move original log files off to archive :::
mv /home/trapper/.wine/drive_d/buzzard/temp/* /home/trapper/.wine/drive_f/archive/logs/s-series/

THE PROBLEM:
When I paste addon.txt to the log files it screws up the formatting. The files I am working with are DOS txt format. I end up with files in the queue that the MS-based mail server simply discards because they are not in the required format.

This is what the 101's need to look like:
Quote:
$$ admin@mydomain.com
T stats@mydomain.com

To: stats@mydomain.com
From: admin@mydomain.com
Subject: Daily Status Logs
Date: 08-29-2007
X-Mailer: 101template

DAILY STATUS LOG
<<< snipped >>>
This is what the 101's look like in gedit:
Quote:
HTML Code:
$$ [email]admin@mydomain.com[/email]
        T [email]stats@mydomain.com[/email]
	
        To: [email]stats@mydomain.com[/email]
        From: [email]admin@mydomain.com[/email]
        Subject: Daily Status Logs
        Date: 08-29-2007
        X-Mailer: 101template
	
        DAILY STATUS LOG
        <<< snipped >>>
This is what the 101's look like in windows notepad. This is what the mail server sees and rejects:
Quote:
$$ admin@mydomain.com|T stats@mydomain.com| To: stats@mydomain.com | From: admin@mydomain.com | Subject: Daily Status Logs| (etc. etc. etc.)

Last edited by Trapper; 08-29-2007 at 12:05 PM..
# 6  
Old 08-30-2007
I happened upon this code while googling. It has resolved my text format problem. It maintains ms-dos format when appending the files.
Code:
for i in *.log; do (echo '0r /home/trapper/.wine/drive_d/buzzard/addon.txt'; echo 'wq') | ed $i; done

That made me make some adjustments to my script. This is what I finally ended up with. It does what I originally wanted to do. As I said previously, I am sure there's a much more efficient and proper way to do all this and, eventually, when I become more knowledgeable in bash scripting, I'll probably change and streamline it. Time's a premium and that just has to wait.

Here's my final code:

Code:
# Move all logs, except today's log to a temp dir.

cd /home/trapper/.wine/drive_d/buzzard/logs/s-series
find *.log -daystart -mtime +0 -exec mv {} /home/trapper/.wine/drive_d/buzzard/temp \;

# Send a copy of the logs off to archive.

cd /home/trapper/.wine/drive_d/buzzard/temp
cp *.log /home/trapper/.wine/drive_f/archive/logs/s-series/

# Insert header text and retain current text formatting.

for i in *.log; do (echo '0r /home/trapper/.wine/drive_d/buzzard/addon.txt'; echo 'wq') | ed $i; done

# Bulk rename log files to extension needed by mail server.

ls *|sed 's/\(.*\)\.log/mv \1.log  \1.101/' |sh

# Move .101's to mail queue for delivery.

mv *.101 /home/trapper/.wine/drive_d/promail/queue

That's it.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Rename (move) multiple files on remote server using sftp

I want to rename (move) multiple files on remote server. I tried the following command to move all TXT files from my_dir directory to /new_dir. But it does not work. Any help? #!/bin/ksh sftp -dev3 << ABC cd my_dir $(for i in TXT; do echo "ls *.$i" ; rename $x /new_dir/$x;... (1 Reply)
Discussion started by: Soham
1 Replies

2. Shell Programming and Scripting

SBATCH trinity for multiple files and rename/move the output files

Hey guys, I have wrote the following script to apply a module named "trinity" on my files. (it takes two input files and spit a trinity.fasta as output) #!/bin/bash -l #SBATCH -p node #SBATCH -A <projectID> #SBATCH -n 16 #SBATCH -t 7-00:00:00 #SBATCH --mem=128GB #SBATCH --mail-type=ALL... (1 Reply)
Discussion started by: @man
1 Replies

3. UNIX for Dummies Questions & Answers

Rename files in a directory and move them

I have a directory e2e_ms_xfer/cent01 this contains the multiple files some of which will be named below with unique date time stamps e2e_ms_edd_nom_CCYYMMDD_HHMM.csv What I want to do is in a loop 1) Get the oldest file 2) Rename 3) Move it up one level from e2e_ms_xfer/cent01 to... (1 Reply)
Discussion started by: andymay
1 Replies

4. Shell Programming and Scripting

Find all images, append unique prefix to name and move to different directory

Hi, I have a directory with Multiple subdirectories and 1000s of pictures (jpg) in each directory. The problem is that each directory has a 001.jpg in them. I want to append a unique name (the directory_name)would be fine. and then move them to one main backup directory once they have been... (1 Reply)
Discussion started by: kmaq7621
1 Replies

5. Shell Programming and Scripting

Script to find & replace a multiple lines string across multiple php files and subdirectories

Hey guys. I know pratically 0 about Linux, so could anyone please give me instructions on how to accomplish this ? The distro is RedHat 4.1.2 and i need to find and replace a multiple lines string in several php files across subdirectories. So lets say im at root/dir1/dir2/ , when i execute... (12 Replies)
Discussion started by: spfc_dmt
12 Replies

6. Shell Programming and Scripting

Find & Replace string in multiple files & folders using perl

find . -type f -name "*.sql" -print|xargs perl -i -pe 's/pattern/replaced/g' this is simple logic to find and replace in multiple files & folders Hope this helps. Thanks Zaheer (0 Replies)
Discussion started by: Zaheer.mic
0 Replies

7. Shell Programming and Scripting

Need to move and rename a list of files

Hi, I need to do something easy but I can't seem to figure out how to do this. Let's say I have 6 files in the directory below: /ebsbeta_f/flash/EBSUATQB/onlinelog o1_mf_6_55klt7nr_.log o1_mf_3_55klskj4_.log o1_mf_4_55klsrl1_.log o1_mf_5_55klt09p_.log o1_mf_2_55klv1ts_.log... (10 Replies)
Discussion started by: exm
10 Replies

8. Shell Programming and Scripting

Rename/Move files and schedule the script

Hello, I'm new in the forum and in UNIX scripting, what I need is to write a simple batch script that renames or move the files back & forth from one directory to another, and then schedule the script to run on the server when the scheduled down time is, which is on Thursdays at 8pm and during... (5 Replies)
Discussion started by: dannyghost
5 Replies

9. UNIX for Dummies Questions & Answers

Using Rename to move files

I am connecting to a remote server (Unix) and doing a ftp dowmload of files. The script (VB script) works fine except for not being able to move the downloaded files on the remote server to another folder. I need to move all files with an .asc extesnion from folder "tovecellio_edi" to folder... (1 Reply)
Discussion started by: snufse
1 Replies

10. Shell Programming and Scripting

Move files and rename ??

1. If I have a file-yyyymmdd.dat in a directory DATA1, then how do I move this file to directory DATA2 and the file name change to file-yyyymmdd.dat.currenttime I can manual do this $mv fileA-yyyymmdd.dat ./DATA2/fileA-yyyymmdd.dat.`date +%Y%m%d%H%M%S` but how do I move all of the files in... (1 Reply)
Discussion started by: sabercats
1 Replies
Login or Register to Ask a Question