Sponsored Content
Top Forums Shell Programming and Scripting sort retrieved data files in different dirs Post 302511947 by leafei on Friday 8th of April 2011 03:15:52 AM
Old 04-08-2011
sort retrieved data files in different dirs

Hi,

I have a script to retrieve data files from server and store them in a directory on local disk. The script runs everyday as cron job. But now those files are too many so my boss wants me to put the files into different directories base on dates.

Those files look like this:

Code:
FCNCP104500104_SPA_2011-04-08_04-53-56-GMT_P08-00.nar

My original script:


Code:
#! /bin/bash

EMC_HOST='3.3.3.3'
EMC_USER='admin'
EMC_PASSWORD='password'
NAV_LOG_FILE='/var/log/naviseccli'
NAVISECCLI='/opt/Navisphere/bin/naviseccli'
NAV_BASE_CMD="${NAVISECCLI} -h ${EMC_HOST} -user ${EMC_USER} -password ${EMC_PASSWORD} -scope 0"
RETRIEVE_DIR='/mnt/emc_data'

#
# Simple logger.
#
log () {
    echo ${1} >> ${NAV_LOG_FILE}
}

#
# Retrieve file.
#
retrieve () {
    log "Retrieve file ${1}..."
    cmd="$NAV_BASE_CMD analyzer -archive -file ${1} -o"
    eval $cmd
    if [ $? -ne 0 ] ; then
        log "$(date): Retrieve $1 failed"
        log "command: $cmd"
        log "error code: $?"
        exit -1
    fi
}

#
# List all archives on EMC.
#
list () {
    file_list=$($NAV_BASE_CMD analyzer -archive -list | awk 'BEGIN {IGNORECASE=1;} /nar$/ { print $NF }')
    if [ $? -ne 0 ] ; then
        log "$(date): list archives failed"
        log "error code: $?"
        exit -2
    fi
}

cd $RETRIEVE_DIR || exit -1

list
for f in ${file_list} ; do
    [ -f ${f} ] || retrieve ${f}
done

Any suggestions to improve (rewrite) the script? Smilie
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

I need to ls all files in 4-6 deep dirs

I need to print to file , a listing of all files below a certain directory. Example: I need to print to file a listing of all files below the etc dir (including the subdirectories) with their full path. Any ideas on how to do this with one command. Or is this something I need to do on all... (4 Replies)
Discussion started by: gforty
4 Replies

2. Shell Programming and Scripting

script find files in two dirs HELP

I have a directory which is /home/mark/files/ , inside this particular I have a bunch of filles (see examples below) TST_SHU_00014460_20090302.txt TST_SHU_00016047_20090302.txt TST_SHU_00007838_20090303.txt TST_SHU_00056485_20090303.txt TST_SHU_00014460_20090303.txt... (2 Replies)
Discussion started by: fierusbentus
2 Replies

3. Shell Programming and Scripting

AWK help print dirs with files in it

Hi, I'm writing some start of day checks for my work. I want to check some dirs for files that have been created longer than 10 mins ago and not been transfered. I've already used a find command to write a list of files that meet this criteria to a log called sod.log i.e. ... (1 Reply)
Discussion started by: elcounto
1 Replies

4. Homework & Coursework Questions

Shell script calling Perl function, sort and find data, write to new files

Use and complete the template provided. The entire template must be completed. If you don't, your post may be deleted! 1. The problem statement, all variables and given/known data: I must write a shell script that calls two external Perl functions--one of which sorts the data in a file, and... (6 Replies)
Discussion started by: kowit010
6 Replies

5. UNIX for Dummies Questions & Answers

FTP SERVER logging of files retrieved

Is there any addon or finctionality in ftp (SFTP) Server that definitively indicates if a file was retireved and successfully? I am looking for this type of command with logging so -files retrieved -files deleted by remote users can be logged Must be compatible with AIX and UBUNTU. ... (0 Replies)
Discussion started by: cdc01
0 Replies

6. Shell Programming and Scripting

Creating a list of files retrieved from MGET

Is there a way to create a txt file with the names of the files I retreive using mget *.file. I want to use this file as input to a delete command. Using HP-UX (3 Replies)
Discussion started by: jagf
3 Replies

7. Debian

Problem with files/dirs deletion

Hi, The other day i installed a PHP based CMS (modx) on my shell account and noticed that i couldn't delete any of files/dirs it created after. Also, i noticed that all that stuff is owned by username-www instead of username. I tried chown, chmod and using a PHP script to do the same wti... (4 Replies)
Discussion started by: pentago
4 Replies

8. Shell Programming and Scripting

Moving files into dirs corresponding to dates

I am trying to find a way to move files into corresponding date files. i=0 while read line do array="$line" (( i++ )) done < <(ls) cd $(echo ${array}) echo ${array}} pwd #cd "$(array}" ] || mkdir 2015 cd "2015" ] || mkdir 02-February ] || mkdir 03-March ] || mkdir... (10 Replies)
Discussion started by: newbie2010
10 Replies

9. Shell Programming and Scripting

Merge and Sort tabular data from different text files

I have 42 text files; each containing up to 34 lines with following structure; file1 H-01 23 H-03 5 H-05 9 H-02 14 . . file2 H-01 17 H-02 43 H-04 7 H-05 8 H-03 7 . . file3 (6 Replies)
Discussion started by: Syeda Sumayya
6 Replies

10. Shell Programming and Scripting

Read info from api website and show retrieved data

good evening, i'm still new in scripting but i'm learning every day and i'm enjoying it. so i have api website (htt p://api.nobelprize.org/v1/prize.json), i want to make a script that allows me to give it two arguments like ./test.sh 2005 physics, 2000 is for the year and physics is category... (1 Reply)
Discussion started by: kalbsghir
1 Replies
REARJ(1)							   Arj Software 							  REARJ(1)

NAME
rearj - Converts other archive formats to .arj files SYNOPSIS
rearj [options] <!listfile(s)|filespec(s)|wildname(s)> DESCRIPTION
The rearj command can repackage other archive formats to arj. OPTIONS
-+ Ignore REARJ_SW variable. -a[suffix] Convert archives within archives ("*" for all formats). -bcommand Execute command before extracting files. -ccommand Execute command on extracted files before counting them. -d Delete original archives. -e No error switch. -f Convert diskette archives. -g Skip creation of output archive. -i[name] Check integrity of rearj program. -j Skip if output archive size is larger than the original. -k Skip archive timestamping. -l[name] Write append log file (default name is rearj.log). -m[date] Select archives before date in YYMMDDHHMMSS format. -n[date] Select archives on or after date in YYMMDDHHMMSS format. -o Allow overwrite of existing target archive. -p Ignore long filenames under Windows 95. -q Query for each archive to convert. -r Recurse through subdirectories. -s Skip verify of file count and total size. -tsuffix Create suffix type archives. -u[bak] Allow update of archive with backup (default is BAK). -v Execute configured command on extracted files. -wdir Assign work directory. -x[!]file Exclude by filename, wildname, or listfile. -y[text] Delete output archive and write text to log (testing mode). -z Simulate operation. ENVIRONMENT
REARJ_SW If set, its value will be used as command line options. EXIT STATUS
0 Success. 1 File not found. 2 File is not a configured archive type. 3 Target archive already exists. 4 Not enough disk space. 5 User skipped or user did not select update option. 6 UNPACK command error. 7 PACK command error. 8 Target cannot support directories. 9 Wrong file count. 10 Wrong total size. 11 Internal archive REARJ error. 12 Rename archive error. 13 Invoked -v command error (found a virus?). 14 Output archive is larger. EXAMPLE
rearj * -r -d Convert all archives to ARJ format, searching all subdirectories, deleting original archives. SEE ALSO
arj(1). AUTHOR
This manual page was written by Ola Lundqvist <opal@debian.org> in pod format, then converted by Guillem Jover <guillem@debian.org> to nroff format. 3.10 2003-03-10 REARJ(1)
All times are GMT -4. The time now is 08:32 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy