Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Rename files - need help with array?

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Rename files - need help with array?
# 1  
Old 12-18-2008
Rename files - need help with array?
I am writing a basic shell script to go out and rename files in a filesystem then scp them to a server. The problem is that there are 11 directories and files. I know I can write this 2 ways, one way the long way and one way the short way. I know how to do the long way... I'm curious about the short way. Here is the basic script I've got so far:

Code:
#!/usr/xpg4/bin/sh
#
#
MAINDIR=/apps/main1
FILECOMP=/apps/scripts/rename.txt
SCPINFO=box1:/path/to/location

# Directories
DIRNAME1=Test1/Ready/ 
DIRNAME2=Test2/Ready/ 
DIRNAME3=Test2_Part2/ 
DIRNAME4=Test3_Part3/Ready/ 
DIRNAME5=Test4/Ready/ 
DIRNAME6=Test5/Ready/ 
DIRNAME7=Test6/Ready/ 
DIRNAME8=Test7/Ready/ 
DIRNAME9=Test8/Ready/ 
DIRNAME10=Test9/Ready/ 
DIRNAME11=Test10/Ready/

# File Names
FNAME_1=file1.csv 
FNAME_2=file2.csv 
FNAME_3=file3.csv 
FNAME_4=file4.csv 
FNAME_5=file5.csv 
FNAME_6=file6.csv 
FNAME_7=file7.csv 
FNAME_8=file8.csv 
FNAME_9=file9.csv 
FNAME_10=file10.csv 
FNAME_11=file11.csv

umask -S
# Dates the log file
echo "" >>$LOGDIR 2>&1
`/usr/bin/date  >>$LOGDIR 2>&1`

# ------- START PROCESSING  ---------
# Go into first directory  
/usr/bin/cd $MAINDIR/$DIRNAME1 

# Get newest file - Files are named by date: ex:   Data_081202-021150_000.txt
FNAMETMP=/usr/bin/find . -name *.txt -newer $FILECOMP 2>>$LOGDIR

# Rename
/usr/bin/mv $FNAMETMP $FNAME_1 2>>$LOGDIR

# Move to archive
/usr/bin/mv $FNAMETMP archive  2>>$LOGDIR

# SCP to another server
/usr/bin/scp $FNAME_1 $SCPINFO 2>>$LOGDIR


# Update the File for comparison
touch $FILECOMP
exit

I didn't go on and copy the parts I am wanting to do... since it seems like I can do this easier via some for loop and an array. I'm just not sure how to do that... still learning scripting Smilie Thanks.

Btw... the files won't be called file1,2,3,4,5 for the File Name section and also directories. I'm using those names as examples for now.
# 2  
Old 12-19-2008
I don't see any reason for keeping directories and files separated, as you are mapping one file to each directory in a one-to-one correspondence. You could create two arrays and then access the items as you like, but I think a simpler and straightforward way for doing this could be just creating a list of files including the full path. You could loop through your list of files like this:

#!/usr/xpg4/bin/sh
#
#
MAINDIR=/apps/main1
FILECOMP=/apps/scripts/rename.txt
SCPINFO=box1:/path/to/location

theFilesList='Test1/Ready/file1.csv Test2/Ready/file2.csv Test3_Part3/Ready/file3.csv ....'

umask -S
# Dates the log file
echo "" >>$LOGDIR 2>&1
`/usr/bin/date >>$LOGDIR 2>&1`

# ------- START PROCESSING ---------
for i in $theFilesList
do
FNAMETMP=/usr/bin/find . -name *.txt -newer $FILECOMP 2>>$LOGDIR
# Rename
mv $FNAMETMP $i 2>>$LOGDIR
# Move to archive
mv $FNAMETMP archive 2>>$LOGDIR
# SCP to another server
scp $i $SCPINFO 2>>$LOGDIR
# Update the File for comparison
touch $FILECOMP
done
exit


If you are interested in using arrays, then you can define two arrays like:
set -A theDirectories Test1/Ready Test2/Ready ....
set -A theFiles file1.csv file2.csv ...

To access an array element:
${Directories[_position_of_the_item]}

To loop through an array:
for i in ${theFiles[@]}
do
echo $theFiles
done

To get the number of elements:
${#theFiles[@]}

That is using the Korn Shell though...
# 3  
Old 02-03-2009
RE: script
Quote:
Originally Posted by lamano
I don't see any reason for keeping directories and files separated, as you are mapping one file to each directory in a one-to-one correspondence. You could create two arrays and then access the items as you like, but I think a simpler and straightforward way for doing this could be just creating a list of files including the full path. You could loop through your list of files like this:

#!/usr/xpg4/bin/sh
#
#
MAINDIR=/apps/main1
FILECOMP=/apps/scripts/rename.txt
SCPINFO=box1:/path/to/location

theFilesList='Test1/Ready/file1.csv Test2/Ready/file2.csv Test3_Part3/Ready/file3.csv ....'

umask -S
# Dates the log file
echo "" >>$LOGDIR 2>&1
`/usr/bin/date >>$LOGDIR 2>&1`

# ------- START PROCESSING ---------
for i in $theFilesList
do
FNAMETMP=/usr/bin/find . -name *.txt -newer $FILECOMP 2>>$LOGDIR
# Rename
mv $FNAMETMP $i 2>>$LOGDIR
# Move to archive
mv $FNAMETMP archive 2>>$LOGDIR
# SCP to another server
scp $i $SCPINFO 2>>$LOGDIR
# Update the File for comparison
touch $FILECOMP
done
exit
This looks pretty straight foward. How does it know to go into each directory though.. I see you have MAINDIR defined but then don't call it above. Would I just put $MAINDIR in the find?

FNAMETMP=/usr/bin/find $MAINDIR -name *.txt -newer $FILECOMP 2>>$LOGDIR
# 4  
Old 02-03-2009
A couple of things:

First of all, this:

Code:
FNAMETMP=/usr/bin/find $MAINDIR -name *.txt -newer $FILECOMP 2>>$LOGDIR

Will not work.
I think what you want is:

Code:
/usr/bin/find $MAINDIR -name *.txt -newer $FILECOMP | head -1 | read FNAMETMP
echo $FNAMETMP


Secondly, it's better to redirect all LOGGING commands all at once.
There's a variety of ways to do this.... however, my preferred way
is through a subshell and redirection. It's simple and straightforward:

Code:
(
date
echo some other commands
echo more commands
echo etc...
date
) > LOG 2>&1

This way, you don't clutter up code with a bunch of ">> $LOGFILE" stuff.
Plus, you save stdout and stderr in one fell swoop and cannot lose anything.

I don't understand why you'd want to clobber a bunch of *.txt files with *.csv
files in this way....

it seems to me that this:

Code:
(
find $MAINDIR -name \*.csv -print |
while read file_nm ; do

  ### change csv to txt
  tmp=${file_nm%.csv}.txt

  ### remove directory name ( like basename )
  new_file_nm=${tmp##*/}

  mv $file_nm archive/$new_file_nm

  scp archive/$new_file_nm $SCPINFO

done
) > $LOGFILE 2>&1

... is what you're looking for.

It finds a bunch of *.csv files.
Moves them to an archive directory as *.txt files.
Then scp's them over to somewhere else.
And stores any error messages in a log file.
# 5  
Old 02-03-2009
Ack
Actually, what I'm wanting to do is

1.) Go through each directory (DIR1, DIR2, DIR3, DIR4 ... )
2.) Find newest .txt file in that directory
3.) Copy the .txt file to archive directory under that directory with .timestamp
3.) Rename .txt file from list to (file1.csv file2.csv file3.csv)
4.) move to archive directory with new .csv file
5.) scp file to server
# 6  
Old 02-03-2009
Quote:
Originally Posted by darkoth
Actually, what I'm wanting to do is

1.) Go through each directory (DIR1, DIR2, DIR3, DIR4 ... )
2.) Find newest .txt file in that directory
3.) Copy the .txt file to archive directory under that directory with .timestamp
3.) Rename .txt file from list to (file1.csv file2.csv file3.csv)
4.) move to archive directory with new .csv file
5.) scp file to server
good thing i'm bored today...
Code:
date '+%Y-%m-%d-%H-%M-%S' |
  sed -e 's/-//g' | ### in case of sccs...
  read time_stamp

for dir in DIR1 DIR2 DIR3 DIR4 ; do

  cd $dir
  /bin/ls -t *.txt | head -1 | read latest_file

  new_file_nm=${latest_file%.*}.csv

  cp $latest_file $ARCH_DIR/$latest_file.$time_stamp

  mv $latest_file $ARCH_DIR/$new_file_nm

  scp $ARCH_DIR/$new_file_nm $SCPINFO

  cd ..

done

# 7  
Old 02-04-2009
Quote:
Originally Posted by quirkasaurus
good thing i'm bored today...
Code:
date '+%Y-%m-%d-%H-%M-%S' |
  sed -e 's/-//g' | ### in case of sccs...
  read time_stamp

for dir in DIR1 DIR2 DIR3 DIR4 ; do

  cd $dir
  /bin/ls -t *.txt | head -1 | read latest_file


That will fail in all shells except ksh because each element of a pipeline is executed in its own subshell. This will work in any POSIX shell:

Code:
latest_file=$( ls -t *.txt | head -1 )

Quote:
Code:
  new_file_nm=${latest_file%.*}.csv

  cp $latest_file $ARCH_DIR/$latest_file.$time_stamp

  mv $latest_file $ARCH_DIR/$new_file_nm

  scp $ARCH_DIR/$new_file_nm $SCPINFO

  cd ..

done
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

SBATCH trinity for multiple files and rename/move the output files

Hey guys, I have wrote the following script to apply a module named "trinity" on my files. (it takes two input files and spit a trinity.fasta as output) #!/bin/bash -l #SBATCH -p node #SBATCH -A <projectID> #SBATCH -n 16 #SBATCH -t 7-00:00:00 #SBATCH --mem=128GB #SBATCH --mail-type=ALL... (1 Reply)
Discussion started by: @man
1 Replies

2. Shell Programming and Scripting

Rename files

I am getting some files with this mask: aaaa_SP_bbb.txt aaaa_FX_bbbb.txt aaaabbbb.txt I want to rename the files containing: _SP_ -> Rename - >> fileSP.TXT _FX_ -> Rename - >> fileFX.txt and other file that does not contain these words rename them: filenamexx.txt as I... (4 Replies)
Discussion started by: Jomeaide
4 Replies

3. Shell Programming and Scripting

Script to unzip files and Rename the Output-files

Hi all, I have a many folders with zipped files in them. The zipped files are txt files from different folders. The txt files have the same names. If i try to find . -type f -name "*.zip" -exec cp -R {} /myhome/ZIP \; it fails since the ZIP files from different folders have the same names and... (2 Replies)
Discussion started by: pmkenya
2 Replies

4. UNIX for Dummies Questions & Answers

Rename several files

Hi, I would like to rename several files, for example: MSIRP_CatastropheLog_Data_Extract_-_TCAPMI_SMB_20111116_040028.txt MSIRP_CatastropheLog_Data_Extract_-_TCAPMI_SMB_20111117_040023.txt MSIRP_CatastropheLog_Data_Extract_-_TCAPMI_SMB_20111118_040039.txt... (3 Replies)
Discussion started by: wsoethe
3 Replies

5. Shell Programming and Scripting

Rename Files in array

Hi Experts, I am new to Unix i wrote below code to rename if the file size less than 8MB. I might have more than one file to rename sometimes:wall:. Please help me in writing the code for multiple files. Eg:- 071720112200.file001 and 071820112200.file001 Single File... (1 Reply)
Discussion started by: rajubollas
1 Replies

6. UNIX for Dummies Questions & Answers

Rename all .sh files to .pl

I have various .sh and .pl files in one directory. I want to rename all the .sh files to .pl i.e testscript.sh --> testscript.pl I am trying to use mv *.sh *.pl It doesnt work though!! (3 Replies)
Discussion started by: chrisjones
3 Replies

7. Shell Programming and Scripting

How to rename files

Hi Guys, I have to rename about 180 files in different folders in linux. For example, abc_110117.txt eff_110117.txt zzz_110117.txt After renaming the files, these files should like like abc.txt eff.txt zzz.txt I created a small script to rename the files like ls... (2 Replies)
Discussion started by: naveed
2 Replies

8. Shell Programming and Scripting

rename files Ax based on strings found in files Bx

Hi, I'm not very experienced in shell scripting and that's probably why I came across the following problem: I do have several hundred pairs of text files (PF00x.spl and PF00x.shd) where the first file (PF00x.spl) needs to be renamed according a string that is included in the second file... (12 Replies)
Discussion started by: inCH
12 Replies

9. Shell Programming and Scripting

rename files

hey all, I have files in the format of ABCD20061101 and ABCDEF20061101 in one directory, I would like to change all ABCD20061101 to ABCDEF20061101 and the problem is if I do a simple pattern match of ABCD, then those ABCDEF20061101 would also... (2 Replies)
Discussion started by: mpang_
2 Replies

10. UNIX for Dummies Questions & Answers

How to rename files?

:confused: How can i rename a file 'x.log' to 'x_20020512 072909.log' :eek: i'm using perl, with system command from a unix web server, and need to timestamp my logs if the above format (filename _ year month day hr min sec .log) (9 Replies)
Discussion started by: CompuTelSystem
9 Replies
Login or Register to Ask a Question