07-04-2006
Sort files by Date-timestamps available in filename & pick the sortedfiles one by one
Hi,
I am new to Unix shell scripting. Can you please help me with this immediate requirement to code.. The requirement is as given below.
In a directory say Y, I have files like this.
PP_100000_28062006_122731_746.dat
PP_100000_28062006_122731_745.dat
PP_100000_28062006_122734_745.dat
PP_100000_28062006_122732_745.dat
PP_100000_28062006_122801_745.dat
QQ_100001_28062006_122733_745.dat
QQ_100001_28062006_122731_745.dat
RR_100002_28062006_122731_745.dat
RR_100002_28062006_122732_745.dat
RR_100002_28062006_122729_745.dat
format: <type>_<somesequence>_DDMMYYYY_HHMMSS_<organisationID>
Now, I will have an input parameter coming in as 'Directory path' (path to Y) and type (can be PP, QQ or RR. If type is NULL, means I need to process all the 3 types)
Assuming we get type as PP
Now I need to pick up files of this type. We can see there are 5 files matching this string.
Now, I need to check the date and time stamp as available in the file names (and not the unix system date timestamp), compare it, and pick up the file with the lowest date timestamp in its name, first - and process it.
As you see I have 2 files with same timestamp here - PP_100000_28062006_122731_746.dat and PP_100000_28062006_122731_745.dat. In this case I have to process it in ascending order of <organisationID>. ie, PP_100000_28062006_122731_745.dat first, followed by PP_100000_28062006_122731_746.dat
Once processed, I have to pick the next lowest date time stamp file for processing.
I will be archiving successfully processed files in another directory, and renaming failed files to say FAILED (suffix) so that it wont be picked up again.
Any sample code/ pointers/ suggestions would be of great help.
Thanks a lot in advance,
Chindhu
10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
Hi,
I have some question and need some guidance how to sort and match multiple files.
1. all the data in the files are numbers
e.g. 1234567
1584752
2563156
2. each sorted file have their own ouput. e.g. test.csv -> test_sorted.csv
3. Then, I need to match all... (4 Replies)
Discussion started by: nazri76
4 Replies
2. Shell Programming and Scripting
Hi,
I am trying to write a script to ftp and get all the files between two date/time stamps from a archive directory. I have sent an attatchment of my archive directory. With the script I intend to get files for ex: between request.log.2008-08-22-03-53-49.gz &... (3 Replies)
Discussion started by: openspark
3 Replies
3. UNIX for Dummies Questions & Answers
I need to edit the file name with date and time while writing the script.
please help. (1 Reply)
Discussion started by: manish.s
1 Replies
4. UNIX for Dummies Questions & Answers
dear all,
i have .dat files named as:
34.dat
2.dat
16.dat
107.dat
i would like to sort them by their filenames as:
2.dat
16.dat
34.dat
107.dat
i have tried numerous combinations of sort and ls command (in vain) to obtain :
107.dat
16.dat
2.dat
34.dat (1 Reply)
Discussion started by: chen.xiao.po
1 Replies
5. Shell Programming and Scripting
Hi, I am a newbie to shell programming and I need some help in sorting a list of files in ascending order of date in the filenames.
The file format is always : IGL01_AC_D_<YYYYMMDD>_N01_01
For example, in a directory MyDirectory I have the following files:
IGL01_AC_D_20110712_N01_01.dat... (11 Replies)
Discussion started by: Yuggy
11 Replies
6. Shell Programming and Scripting
Hi all.
I am very new to linux scripting and i have a task i can only solve with a script.
I need to sort files base on the date string in their filenames and create a folder using the same date string then move the files to their respective folders.
Scenario:
Folder Path:... (1 Reply)
Discussion started by: ace47
1 Replies
7. Shell Programming and Scripting
Hi all.
I am very new to linux scripting and i have a task i can only solve with a script.
I need to sort files base on the date string in their filenames and create a folder using the same date string then move the files to their respective folders.
Scenario:
Folder Path:... (1 Reply)
Discussion started by: ace47
1 Replies
8. Shell Programming and Scripting
Hi,
I have a list of log files as follows:
name_date_0001_ID0.log
name_date_0001_ID2.log
name_date_0001_ID1.log
name_date_0002_ID2.log
name_date_0004_ID0.log
name_date_0005_ID0.log
name_date_0021_ID0.log
name_date_0025_ID0.log
.......................................... (4 Replies)
Discussion started by: alex2005
4 Replies
9. Shell Programming and Scripting
So given filenames of varying lengths, I was wondering how I would remove or modify appended timestamps of the current date DD-MM-YY.
So say:
test_DD-MM-YY.txt
coolbeans_DD-MM-YY.pdf
And what I expect the output to be:
test.txt
coolbeans.pdf
Thanks :) (2 Replies)
Discussion started by: sodaboyz
2 Replies
10. Shell Programming and Scripting
Hi All,
I need to find all files other than first two files dates & last file date for month and month/year wise list.
lets say there are following files in directory
Mar 19 2012 c.txt
Mar 19 2012 cc.txt
Mar 21 2012 d.txt
Mar 22 2012 f.txt
Mar 24 2012 h.txt
Mar 25 2012 w.txt
Feb 12... (2 Replies)
Discussion started by: Makarand Dodmis
2 Replies
LEARN ABOUT DEBIAN
gmt_shell_functions
GMT_SHELL_FUNCTIONS.SH(1gmt) Generic Mapping Tools GMT_SHELL_FUNCTIONS.SH(1gmt)
NAME
gmt_shell_functions.sh - Practical functions to be used in GMT bourne shell scripts
SYNOPSIS
gmt_init_tmpdir
gmt_remove_tmpdir
gmt_clean_up [prefix]
gmt_message message
gmt_abort message
gmt_nrecords file(s)
gmt_nfields string
gmt_get_field string
gmt_get_region file(s) [options]
gmt_get_gridregion file [options]
gmt_get_map_width -R -J
gmt_get_map_height -R -J
gmt_set_psfile file
gmt_set_framename prefix framenumber
gmt_set_framenext framenumber
DESCRIPTION
gmt_shell_functions.sh provides a set of functions to Bourne (again) shell scripts in support of GMT. The calling shell script should
include the following line, before the functions can be used:
. gmt_shell_functions.sh
Once included in a shell script, gmt_shell_functions.sh allows GMT users to do some scripting more easily than otherwise. The functions
made available are:
gmt_init_tmpdir
Creates a temporary directory in /tmp or (when defined) in the directory specified by the environment variable TMPDIR. The name of
the temporary directory is returned as environment variable GMT_TMPDIR. This function also causes GMT to run in `isolation mode',
i.e. all temporary files will be created in GMT_TMPDIR and the .gmtdefaults file will not be adjusted.
gmt_remove_tmpdir
Removes the temporary directory and unsets the GMT_TMPDIR environment variable.
gmt_cleanup
Remove all files and directories in which the current process number is part of the file name. If the optional prefix is given then
we also delete all files and directories that begins with the given prefix.
gmt_message
Send a message to standard error.
gmt_abort
Send a message to standard error and exit the shell.
gmt_nrecords
Returns the total number of lines in file(s)
gmt_nfields
Returns the number of fields or words in string
gmt_get_field
Returns the given field in a string. Must pass string between double quotes to preserve it as one item.
gmt_get_region
Returns the region in the form w/e/s/n based on the data in table file(s). Optionally add -Idx/dy to round off the answer.
gmt_get_gridregion
Returns the region in the form w/e/s/n based on the header of a grid file. Optionally add -Idx/dy to round off the answer.
gmt_map_width
Expects the user to give the desired -R -J settings and returns the map width in the current measurement unit.
gmt_map_height
Expects the user to give the desired -R -J settings and returns the map height in the current measurement unit.
gmt_set_psfile
Create the output PostScript file name based on the base name of a given file (usually the script name $0).
gmt_set_framename
Returns a lexically ordered filename stem (i.e., no extension) given the file prefix and the current frame number, using a width of
6 for the integer including leading zeros. Useful when creating animations and lexically sorted filenames are required.
gmt_set_framenext
Accepts the current frame integer counter and returns the next integer counter.
NOTES
1. These functions only work in the bourne shell (sh) and their derivatives (like ash, bash, ksh and zsh). These functions do not work in
the C shell (csh) or their derivatives (like tcsh), and cannot be used in DOS batch scripts either.
2. gmt_shell_functions.sh were first introduced in GMT version 4.2.2 and have since been regularly expanded with other practical scripting
short-cuts. If you want to suggest other functions, please do so by mailing to the GMT mailing list: gmt-help@lists.hawaii.edu.
SEE ALSO
GMT(1), sh(1), bash(1), minmax(1), grdinfo(1)
GMT 4.5.7 15 Jul 2011 GMT_SHELL_FUNCTIONS.SH(1gmt)