Script to compare substrings of multiple filenames and move to different directory


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Script to compare substrings of multiple filenames and move to different directory
# 1  
Old 11-19-2012
Script to compare substrings of multiple filenames and move to different directory

Hi there,

I am having trouble with a script I have written, which is designed to search through a directory for a header and payload file, retrieve a string from both filenames, compare this string and if it matches make a backup of the two files then move them to a different directory for processing. Example filenames are:

Code:
S202SCTWELADEDD20121119000662PACS008.xml
WELAIO_20121119000662PACS008.dat
WELADEDD_20121119150656_00000663CAMT056.xml
WELAIO_20121119000663CAMT056.dat
WELADEDD_20121119150650_00000664CAMT056.xml
WELAIO_20121119000664CAMT056.dat

I have one function which deals with the 'PACS' files - the xml is the payload and the .dat is the header. These always get moved as only one pair tends to appear at a time.

I have trouble with the CAMT files, as two pairs can appear at any one time. In order to minimise the risk of picking up the wrong files, I have been quite specific in my functions. The first function retrieves the CAMT payload file code:

Code:
GetCAMTCode ()
{
if [ $DEBUG -eq 1 ]
then
        set -xv
fi
cd $TRANSITDIR
LogMsg INFO "Checking CAM payload file codes"
ls *.xml | awk '{ print substr($1,18,9)}' | read camtcode

GrabAndMoveCAMT
}

For the examples above, this would return
150650_00
150656_00

I then run another function to retrieve the header code for the second half of the pair:

Code:
GrabAndMoveCAMT ()
{
if [ $DEBUG -eq 1 ]
then
        set -xv
fi
cd $TRANSITDIR
LogMsg INFO "Checking for CAMT files..."
for camstr in `ls *.dat | grep CAMT | awk '{ print substr($1,16,13)}'`
do
        if [[ -f WELAIO_${DATE}${camstr}.dat && -f ${PAYLOAD}${camtcode}${camstr}.xml ]]
        then
                LogMsg INFO "WELAIO_${DATE}${camstr}.dat and ${PAYLOAD}${camtcode}${camstr}.xml found in ${TRANSITDIR}"
                cp *${camstr}.* ${TRANSITDIR}/backup
                if [[ -f ${TRANSITDIR}/backup/WELAIO_${DATE}${camstr}.dat && -f ${TRANSITDIR}/backup/${PAYLOAD}${camtcode}${camstr}.xml ]]
                then
                        LogMsg INFO "WELAIO_${DATE}${camstr}.dat and ${PAYLOAD}${camtcode}${camstr}.xml copied to ${TRANSITDIR}/backup"
                        mv *${camstr}.* ${IMPORTDIR}
                        LogMsg INFO "WELAIO_${DATE}${camstr}.dat and ${PAYLOAD}${camtcode}${camstr}.xml moved to ${IMPORTDIR}"
                else
                        LogMsg FAIL "Backup copy fail WELAIO_${DATE}${camstr}.dat and ${PAYLOAD}${camtcode}${camstr}.xml"
                fi
        else
                LogMsg INFO "No files found"
        fi
done

This second function returns
000663CAMT056

The problem is that this compares the payload file WELADEDD_20121119150650_00000664CAMT056.xml with header file WELAIO_20121119000663CAMT056.dat and fails. I have tried using a substring from the end of the filename as a comparison, but because the rest of the filename differs I can't use filemasks in my comparison test (i.e. if [ -f *${camstr} ] translates to literally *000663CAMT056 instead of WELxxx00066 etc).

It has been suggested that I use an array. I have modified the first function like so:

Code:
set -A CAMTCODE `ls *.xml | awk '{ print substr($1,18,9)}'

`

This assigns the two strings correctly, but the problem appears when I do the comparison. If I look at my logs in debug mode, I see it is testing for the existence of these two files:
Code:
[[ -f WELADEDD_20121119150650_00 150656_00000663CAMT056.xml ]]

Which is incorrect.

If anyone can suggest to me how I can use the contents of an array to compare two strings in two different filenames, it would be much appreciated. Let me know if further information is required.

This is on korn shell, Solaris 10.

Al

Last edited by jim mcnamara; 11-19-2012 at 07:47 PM..
# 2  
Old 11-19-2012
Call GrabAndMoveCAMT function for each camtcode value:-
Code:
for xml_file in *.xml
do
   camtcode=$( echo $xml_file | cut -c 18-26 )
   GrabAndMoveCAMT
done

This User Gave Thanks to Yoda For This Post:
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Help with file compare and move script

I'm running debian (the raspbian version of it) and working on a script to compare files in 2 directories, source and target, move files with duplicate names to a 3rd directory, then move remaining files in source to target. I can't get the syntax right, keep getting syntax errors and can't get... (7 Replies)
Discussion started by: mattz40
7 Replies

2. Shell Programming and Scripting

Move multiple files to different directory using a single command

I have multiple files test1, test2, test3 etc. I want to move to a different directory with ABC_ prefixed to every file and and current dat time as postfix using a single command. (I will be using this is sftp with ! (command for local server). I have tried the following but it gives error ... (5 Replies)
Discussion started by: Soham
5 Replies

3. Shell Programming and Scripting

Shell script cannot create directory and move the file to that directory

I have a script, which is checking if file exists and move it to another directory if then mkdir -p ${LOCL_FILES_DIR}/cool_${Today}/monthly mv report_manual_alloc_rpt_A_I_ASSIGNMENT.${Today}*.csv ${LOCL_FILES_DIR}/cool_${Today}/monthly ... (9 Replies)
Discussion started by: digioleg54
9 Replies

4. Shell Programming and Scripting

Script to compare partial filenames in two folders and delete duplicates

Background: I use a TV tuner card to capture OTA video files (.mpeg) and then my Plex Media Server automatically optimizes the files (transcodes for better playback) and places them in a new directory. I have another Plex Library pointing to the new location for the optimized .mp4 files. This... (2 Replies)
Discussion started by: shaky
2 Replies

5. Shell Programming and Scripting

There are multiple filenames in the directory.How to return the the lastest files for each file name

there are mutiple file nams in the directory. How to return the the lastest files for each file name. ex. abc1234_050201 abc1234_050206 abc1234_050208 xyz34_050204 xyz34_050210 xyz34_050218 thanks (4 Replies)
Discussion started by: grand_sam
4 Replies

6. Shell Programming and Scripting

Move file in to directory- script

Hi In directory /mnt/upload I have about 100 000 files (*.png) that have been created during the last six months. Now I need to move them to right folders. eg: file created on 2014-10-10 move to directory /mnt/upload/20141010 file created on 2014-11-11 move to directory /mnt/upload/20141111... (6 Replies)
Discussion started by: primo102
6 Replies

7. Shell Programming and Scripting

Comparing multiple substrings for a match

I have a tab-delimited file containing a large genetic dataset with binary base calls, in this format: Chr7 26021407 1/1:0,0,0:5 1/1:0,0,0:5 1/1:0,0,0:5 Chr7 26022023 1/1:0,0,0:3 1/1:0,0,0:3 1/1:28,3,0:5 Chr7 26022087 1/1:0,0,0:6 1/1:25,3,0:9 1/1:25,3,0:9 Chr7 26022656 1/1:0,0,0:3... (1 Reply)
Discussion started by: ljk
1 Replies

8. Shell Programming and Scripting

Script to parse filenames in a directory and do some actions

Hi All, I don't have much experience in scripting, and couldn't find anything that will help me to write a script I need, hopefully you can help me with it. I have lots of files in one directory with the following file name pattern: 100001-something.ext1 100101-something2.ext2... (4 Replies)
Discussion started by: troman
4 Replies

9. Shell Programming and Scripting

move filenames with to another filename without the last character ~

Hi, I need to move filenames with the following format below into filenames without the ~ like sample below. I hope you can help me create a simple unix script that will do this. Thanks in advance! move filename from: AIRS20081225-235641.BSP~ AIRS20081225-235648.BSP~ AIRS20081225-235640.BSP~... (3 Replies)
Discussion started by: ayhanne
3 Replies

10. Shell Programming and Scripting

Script to backup multiple files and amend their filenames

Hi, I'm trying to write a command that backs up certain files into my current directory and adds a prefix to the backed up file name. I realise this can be done in a script by specifying each individual file but would like to know if it can be done on one line and made an alias. I have the... (5 Replies)
Discussion started by: m223464
5 Replies
Login or Register to Ask a Question