I am writing a basic shell script to go out and rename files in a filesystem then scp them to a server. The problem is that there are 11 directories and files. I know I can write this 2 ways, one way the long way and one way the short way. I know how to do the long way... I'm curious about the short way. Here is the basic script I've got so far:
I didn't go on and copy the parts I am wanting to do... since it seems like I can do this easier via some for loop and an array. I'm just not sure how to do that... still learning scripting Thanks.
Btw... the files won't be called file1,2,3,4,5 for the File Name section and also directories. I'm using those names as examples for now.
I don't see any reason for keeping directories and files separated, as you are mapping one file to each directory in a one-to-one correspondence. You could create two arrays and then access the items as you like, but I think a simpler and straightforward way for doing this could be just creating a list of files including the full path. You could loop through your list of files like this:
# ------- START PROCESSING ---------
for i in $theFilesList
do
FNAMETMP=/usr/bin/find . -name *.txt -newer $FILECOMP 2>>$LOGDIR
# Rename
mv $FNAMETMP $i 2>>$LOGDIR
# Move to archive
mv $FNAMETMP archive 2>>$LOGDIR
# SCP to another server
scp $i $SCPINFO 2>>$LOGDIR
# Update the File for comparison
touch $FILECOMP
done
exit
If you are interested in using arrays, then you can define two arrays like:
set -A theDirectories Test1/Ready Test2/Ready ....
set -A theFiles file1.csv file2.csv ...
To access an array element:
${Directories[_position_of_the_item]}
To loop through an array:
for i in ${theFiles[@]}
do
echo $theFiles
done
I don't see any reason for keeping directories and files separated, as you are mapping one file to each directory in a one-to-one correspondence. You could create two arrays and then access the items as you like, but I think a simpler and straightforward way for doing this could be just creating a list of files including the full path. You could loop through your list of files like this:
# ------- START PROCESSING ---------
for i in $theFilesList
do
FNAMETMP=/usr/bin/find . -name *.txt -newer $FILECOMP 2>>$LOGDIR
# Rename
mv $FNAMETMP $i 2>>$LOGDIR
# Move to archive
mv $FNAMETMP archive 2>>$LOGDIR
# SCP to another server
scp $i $SCPINFO 2>>$LOGDIR
# Update the File for comparison
touch $FILECOMP
done
exit
This looks pretty straight foward. How does it know to go into each directory though.. I see you have MAINDIR defined but then don't call it above. Would I just put $MAINDIR in the find?
Secondly, it's better to redirect all LOGGING commands all at once.
There's a variety of ways to do this.... however, my preferred way
is through a subshell and redirection. It's simple and straightforward:
This way, you don't clutter up code with a bunch of ">> $LOGFILE" stuff.
Plus, you save stdout and stderr in one fell swoop and cannot lose anything.
I don't understand why you'd want to clobber a bunch of *.txt files with *.csv
files in this way....
it seems to me that this:
... is what you're looking for.
It finds a bunch of *.csv files.
Moves them to an archive directory as *.txt files.
Then scp's them over to somewhere else.
And stores any error messages in a log file.
1.) Go through each directory (DIR1, DIR2, DIR3, DIR4 ... )
2.) Find newest .txt file in that directory
3.) Copy the .txt file to archive directory under that directory with .timestamp
3.) Rename .txt file from list to (file1.csv file2.csv file3.csv)
4.) move to archive directory with new .csv file
5.) scp file to server
1.) Go through each directory (DIR1, DIR2, DIR3, DIR4 ... )
2.) Find newest .txt file in that directory
3.) Copy the .txt file to archive directory under that directory with .timestamp
3.) Rename .txt file from list to (file1.csv file2.csv file3.csv)
4.) move to archive directory with new .csv file
5.) scp file to server
Hey guys,
I have wrote the following script to apply a module named "trinity" on my files. (it takes two input files and spit a trinity.fasta as output)
#!/bin/bash -l
#SBATCH -p node
#SBATCH -A <projectID>
#SBATCH -n 16
#SBATCH -t 7-00:00:00
#SBATCH --mem=128GB
#SBATCH --mail-type=ALL... (1 Reply)
I am getting some files with this mask:
aaaa_SP_bbb.txt
aaaa_FX_bbbb.txt
aaaabbbb.txt
I want to rename the files containing:
_SP_ -> Rename - >> fileSP.TXT
_FX_ -> Rename - >> fileFX.txt
and other file that does not contain these words rename them:
filenamexx.txt
as I... (4 Replies)
Hi all,
I have a many folders with zipped files in them. The zipped files are txt files from different folders. The txt files have the same names. If i try to
find . -type f -name "*.zip" -exec cp -R {} /myhome/ZIP \; it fails since the ZIP files from different folders have the same names and... (2 Replies)
Hi,
I would like to rename several files, for example:
MSIRP_CatastropheLog_Data_Extract_-_TCAPMI_SMB_20111116_040028.txt
MSIRP_CatastropheLog_Data_Extract_-_TCAPMI_SMB_20111117_040023.txt
MSIRP_CatastropheLog_Data_Extract_-_TCAPMI_SMB_20111118_040039.txt... (3 Replies)
Hi Experts,
I am new to Unix i wrote below code to rename if the file size less than 8MB.
I might have more than one file to rename sometimes:wall:.
Please help me in writing the code for multiple files.
Eg:- 071720112200.file001 and 071820112200.file001
Single File... (1 Reply)
I have various .sh and .pl files in one directory. I want to rename all the .sh files to .pl
i.e testscript.sh --> testscript.pl
I am trying to use mv *.sh *.pl
It doesnt work though!! (3 Replies)
Hi Guys,
I have to rename about 180 files in different folders in linux. For example,
abc_110117.txt
eff_110117.txt
zzz_110117.txt
After renaming the files, these files should like like
abc.txt
eff.txt
zzz.txt
I created a small script to rename the files like
ls... (2 Replies)
Hi,
I'm not very experienced in shell scripting and that's probably why I came across the following problem:
I do have several hundred pairs of text files (PF00x.spl and PF00x.shd) where the first file (PF00x.spl) needs to be renamed according a string that is included in the second file... (12 Replies)
hey all,
I have files in the format of
ABCD20061101
and
ABCDEF20061101
in one directory, I would like to change all ABCD20061101 to ABCDEF20061101 and the problem is if I do a simple pattern match of ABCD, then those ABCDEF20061101 would also... (2 Replies)
:confused: How can i rename a file 'x.log' to 'x_20020512 072909.log'
:eek: i'm using perl, with system command from a unix web server, and need to timestamp my logs if the above format (filename _ year month day hr min sec .log) (9 Replies)