Sponsored Content
Top Forums Shell Programming and Scripting sort retrieved data files in different dirs Post 302511947 by leafei on Friday 8th of April 2011 03:15:52 AM
Old 04-08-2011
sort retrieved data files in different dirs

Hi,

I have a script to retrieve data files from server and store them in a directory on local disk. The script runs everyday as cron job. But now those files are too many so my boss wants me to put the files into different directories base on dates.

Those files look like this:

Code:
FCNCP104500104_SPA_2011-04-08_04-53-56-GMT_P08-00.nar

My original script:


Code:
#! /bin/bash

EMC_HOST='3.3.3.3'
EMC_USER='admin'
EMC_PASSWORD='password'
NAV_LOG_FILE='/var/log/naviseccli'
NAVISECCLI='/opt/Navisphere/bin/naviseccli'
NAV_BASE_CMD="${NAVISECCLI} -h ${EMC_HOST} -user ${EMC_USER} -password ${EMC_PASSWORD} -scope 0"
RETRIEVE_DIR='/mnt/emc_data'

#
# Simple logger.
#
log () {
    echo ${1} >> ${NAV_LOG_FILE}
}

#
# Retrieve file.
#
retrieve () {
    log "Retrieve file ${1}..."
    cmd="$NAV_BASE_CMD analyzer -archive -file ${1} -o"
    eval $cmd
    if [ $? -ne 0 ] ; then
        log "$(date): Retrieve $1 failed"
        log "command: $cmd"
        log "error code: $?"
        exit -1
    fi
}

#
# List all archives on EMC.
#
list () {
    file_list=$($NAV_BASE_CMD analyzer -archive -list | awk 'BEGIN {IGNORECASE=1;} /nar$/ { print $NF }')
    if [ $? -ne 0 ] ; then
        log "$(date): list archives failed"
        log "error code: $?"
        exit -2
    fi
}

cd $RETRIEVE_DIR || exit -1

list
for f in ${file_list} ; do
    [ -f ${f} ] || retrieve ${f}
done

Any suggestions to improve (rewrite) the script? Smilie
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

I need to ls all files in 4-6 deep dirs

I need to print to file , a listing of all files below a certain directory. Example: I need to print to file a listing of all files below the etc dir (including the subdirectories) with their full path. Any ideas on how to do this with one command. Or is this something I need to do on all... (4 Replies)
Discussion started by: gforty
4 Replies

2. Shell Programming and Scripting

script find files in two dirs HELP

I have a directory which is /home/mark/files/ , inside this particular I have a bunch of filles (see examples below) TST_SHU_00014460_20090302.txt TST_SHU_00016047_20090302.txt TST_SHU_00007838_20090303.txt TST_SHU_00056485_20090303.txt TST_SHU_00014460_20090303.txt... (2 Replies)
Discussion started by: fierusbentus
2 Replies

3. Shell Programming and Scripting

AWK help print dirs with files in it

Hi, I'm writing some start of day checks for my work. I want to check some dirs for files that have been created longer than 10 mins ago and not been transfered. I've already used a find command to write a list of files that meet this criteria to a log called sod.log i.e. ... (1 Reply)
Discussion started by: elcounto
1 Replies

4. Homework & Coursework Questions

Shell script calling Perl function, sort and find data, write to new files

Use and complete the template provided. The entire template must be completed. If you don't, your post may be deleted! 1. The problem statement, all variables and given/known data: I must write a shell script that calls two external Perl functions--one of which sorts the data in a file, and... (6 Replies)
Discussion started by: kowit010
6 Replies

5. UNIX for Dummies Questions & Answers

FTP SERVER logging of files retrieved

Is there any addon or finctionality in ftp (SFTP) Server that definitively indicates if a file was retireved and successfully? I am looking for this type of command with logging so -files retrieved -files deleted by remote users can be logged Must be compatible with AIX and UBUNTU. ... (0 Replies)
Discussion started by: cdc01
0 Replies

6. Shell Programming and Scripting

Creating a list of files retrieved from MGET

Is there a way to create a txt file with the names of the files I retreive using mget *.file. I want to use this file as input to a delete command. Using HP-UX (3 Replies)
Discussion started by: jagf
3 Replies

7. Debian

Problem with files/dirs deletion

Hi, The other day i installed a PHP based CMS (modx) on my shell account and noticed that i couldn't delete any of files/dirs it created after. Also, i noticed that all that stuff is owned by username-www instead of username. I tried chown, chmod and using a PHP script to do the same wti... (4 Replies)
Discussion started by: pentago
4 Replies

8. Shell Programming and Scripting

Moving files into dirs corresponding to dates

I am trying to find a way to move files into corresponding date files. i=0 while read line do array="$line" (( i++ )) done < <(ls) cd $(echo ${array}) echo ${array}} pwd #cd "$(array}" ] || mkdir 2015 cd "2015" ] || mkdir 02-February ] || mkdir 03-March ] || mkdir... (10 Replies)
Discussion started by: newbie2010
10 Replies

9. Shell Programming and Scripting

Merge and Sort tabular data from different text files

I have 42 text files; each containing up to 34 lines with following structure; file1 H-01 23 H-03 5 H-05 9 H-02 14 . . file2 H-01 17 H-02 43 H-04 7 H-05 8 H-03 7 . . file3 (6 Replies)
Discussion started by: Syeda Sumayya
6 Replies

10. Shell Programming and Scripting

Read info from api website and show retrieved data

good evening, i'm still new in scripting but i'm learning every day and i'm enjoying it. so i have api website (htt p://api.nobelprize.org/v1/prize.json), i want to make a script that allows me to give it two arguments like ./test.sh 2005 physics, 2000 is for the year and physics is category... (1 Reply)
Discussion started by: kalbsghir
1 Replies
SYSLOGOUT(8)						      System Manager's Manual						      SYSLOGOUT(8)

NAME
syslogout - modular centralized shell logout mechanism DESCRIPTION
syslogout is a generic approach to enable centralized shell logout actions for all users of a given system in a modular and centralized way mostly aimed at avoiding work for lazy sysadmins. It has only been tested to work with the bash shell. It basically consists of the small /etc/syslogout shell script which invokes other small shell scripts having a .bash suffix which are con- tained in the /etc/syslogout.d/ directory. The system administrator can drop in any script he wants without any naming convention other than that the scripts need to have a .bash suffix to enable automagic sourcing by the /etc/syslogout script. For shell sessions, the contents of /etc/syslogout.d/" will be sourced by every user at logout if the following lines are present in his $HOME/.bash_logout: if [ -f /etc/syslogout ]; then . /etc/syslogout fi If used for X sessions it is advisable to include the former statement into the Xreset script of the X display manager instead to prevent that closing of an terminal emulator window yields unexpected results in your running X session if your X11 terminal emulator is using a login shell. Be sure then to run it under the user-id of the X session's user. See the example files in /usr/share/doc/syslogout/ for illustration. Users not wanting /etc/syslogout to be sourced for their environment can easily disable it's automatic mechanism. It can be disabled by simply creating an empty file called $HOME/.nosyslogout in the user's home directory using e.g. the touch(1) command. Any single configuration file in /etc/syslogout.d/ can simply be overridden by any user by creating a private $HOME/.syslogout.d/ directory which may contain a user's own version of any configuration file to be sourced instead of the system default. It's names have just to match exactly the system's default /etc/syslogout.d/ configuration files. Empty versions of these files contained in the $HOME/.syslo- gout.d/ directory automatically disable sourcing of the system wide version. Naturally, users can add and include their own private scripts to be automagically executed by /etc/syslogout at logout time. OPTIONS
There are no options other than those dictated by shell conventions. Anything is defined within the configuration scripts themselves. SEE ALSO
The README files and configuration examples contained in /usr/share/doc/syslogout/ and the manual page for bash(1), xdm(1x), xdm.options(5), and wdm(1x). Recommended further reading is everything related with shell programming. If you need a similar mechanism for executing code at login time check out the related package sysprofile(8) which is a very close compan- ion to syslogout. BUGS
syslogout in its current form is mainly restricted to bash(1) syntax. In fact it is actually a rather embarrassing quick and dirty hack than anything else - but it works. It serves the practical need to enable a centralized bash configuration until something better becomes available. Your constructive criticism in making this into something better" is very welcome. Before i forget to mention it: we take patches... ;-) AUTHOR
syslogout was developed by Paul Seelig <pseelig@debian.org> specifically for the Debian GNU/Linux system. Feel free to port it to and use it anywhere else under the conditions of either the GNU public license or the BSD license or both. Better yet, please help to make it into something more worthwhile than it currently is. SYSLOGOUT(8)
All times are GMT -4. The time now is 07:04 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy