concatenate files sorted by date


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting concatenate files sorted by date
# 1  
Old 06-07-2008
concatenate files sorted by date

I am a beginner in script writing, i tried to do the following
I have a set of files sorted by date in the format YYMMDD.s and .x and .r
I need to concatenate a header file to these sets of files so I used the following code
Code:
echo "enter Swath number"
read s
echo "please enter first date and MMDD press ENTER"
read i
echo "please enter last date and MMDD  press ENTER"
read j
for((i; i <= j; i++))
do
cat NJSC_Alam_Ph1_Blk1_3D_HDR_S /ud/SPECSDATA/Final_SPS/Alam3D/Swath_${s}/08${i}.s  > /ud/SPECSDATA/Final_SPS/Alam3D/Swath_${s}/08${i}_Final.s

cat NJSC_Alam_Ph1_Blk1_3D_HDR_R /ud/SPECSDATA/Final_SPS/Alam3D/Swath_${s}/08${i}.r  > /ud/SPECSDATA/Final_SPS/Alam3D/Swath_${s}/08${i}_Final.r

cat NJSC_Alam_Ph1_Blk1_3D_HDR_X /ud/SPECSDATA/Final_SPS/Alam3D/Swath_${s}/08${i}.x  > /ud/SPECSDATA/Final_SPS/Alam3D/Swath_${s}/08${i}_Final.x

done

I expect it to concatenate the files based on the user input.
I have 2 problems, the first on is that it doesn't do this for example if I enter the value for i and j to be 0530 and 0531 for 30 and 31 of May it searches for a file named 08345 instead of 080530.
the other problem is how can i make the script to work for 2 different months? like I have the first file as 300508 and last file as 050608, the loop is not going o work there
# 2  
Old 06-09-2008
Problem number 1 is you're mixing date math with integer math. You cannot add 1 to your "0530" and expect to get the next day. For example, there is no integer "0530". There is an integer "530", however, and if you add 1 to it you will get "531". ...NOT the "0531" that you need. And, as you mentioned, adding 1 to that will not get you into June.

The problem of shell date arithmetic is fairly difficult. See "Date math in Linux shell script?": Tech Support from Ask Dave Taylor! .

When you do your comparison, you can test for your end condition in either of two ways:
  • If the current date == the end date (string comparison)
  • If the current iterations number of seconds since the epoch is less than or equal to the end date's number of seconds since the epoch.
Also, here's a couple of more gotchas you need to be aware of. If you do your date math using "number of seconds since the Epoch",
  • You may want your first date to actually be midnight of the first date- that is, the 0'th second of that day.
  • You may want your second date to actually be 23:59:59 on that day.
This is because if you do your "begin <= end" comparison, you may be comparing 3pm on the end day to 2pm on the end day and that iteration of the script will not run.

I mention that because it's convenient to actually do the date math by:
  1. Convert the first date to the number of seconds since the epoch
  2. Increment the date by adding 86,400 (the number of seconds in a day) to the date for each iteration.
You also must be aware of when the year changes. Don't forget you may be comparing dates in January to dates in December.
-mschwage
# 3  
Old 08-16-2008
Date variable definition

thank you for the reply, I get what you want to say fine, just one request:
what is the script command to identify a date variable? is there a variable called date?
I tried something like date i, and dim i as date but it didn't work.

I also tried "date -d ${i} +"%y%m%d"" but it said that i is not a command

can you please help me on that?

Last edited by docaia; 08-16-2008 at 04:54 AM.. Reason: adding some info
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Is there a way to join 2 text files sorted by

Can anyone please help me i have 2 text files setup like the one below. Textfile1: randomemail1:randompassword1 randomemail2:randompassword2 randomemail3:randompassword3 randomemail4:randompassword4 randomemail5:randompassword5 Textfile2: randompassword1:randomphrase1... (8 Replies)
Discussion started by: nufc
8 Replies

2. Shell Programming and Scripting

Concatenate many files which contents the same date as part of name file

Gents, I have lot of files in a folder where each file name includes the date of generation, then I would like to merge all the files for each date in a complete file. list of files in forder. dsd01_121104.txt dsd01_121105.txt dsd01_121106.txt dsd03_121104.txt dsd03_121105.txt... (7 Replies)
Discussion started by: jiam912
7 Replies

3. Shell Programming and Scripting

sort the files based on timestamp and execute sorted files in order

Hi I have a requirement like below I need to sort the files based on the timestamp in the file name and run them in sorted order and then archive all the files which are one day old to temp directory My files looks like this PGABOLTXML1D_201108121235.xml... (1 Reply)
Discussion started by: saidutta123
1 Replies

4. Shell Programming and Scripting

combine multiple files by column into one files already sorted!

I have multiple files; each file contains a certain data in a column view simply i want to combine all those files into one file in columns example file1: a b c d file 2: 1 2 3 4 file 3: G (4 Replies)
Discussion started by: ahmedamro
4 Replies

5. UNIX for Dummies Questions & Answers

write a program in c in unix that display the files(includ sub-direc and files within) in a sorted

the sorting is based on name of file, file size modification time stamps o f file it should dislay the output in the following format "." and ".." enteries should be ignored please give some idea how to do it (1 Reply)
Discussion started by: pappu kumar jha
1 Replies

6. Programming

merging two files with sorted integers in c

Hi, I want to merge two files with sorted integers. I want to merge two files to a single file such that there are no duplicates and the merged file should contain the integers in the sorted order. Just some idea or pointer to an implementation will help me a lot. Thanks & Regards,... (4 Replies)
Discussion started by: brett01
4 Replies

7. Shell Programming and Scripting

executing code on files in the sorted order -help!

Say i have 2 files in the giving format: file1 1 2 3 4 1 2 3 4 1 2 3 4 file2 1 2 3 4 1 2 3 4 1 2 3 4 I have a PERL code (loaned by one of u -i forgot who - thanks!) that extracts the 2nd column from each file and append horizontally to a new file: perl -ane 'push @{$L->}, $F; close... (1 Reply)
Discussion started by: epi8
1 Replies

8. UNIX for Dummies Questions & Answers

sort by date and concatenate first three

Hi: I am trying to create some script that sorts the files in a subdirectory by date and concatenates the thre most recently created files. SAy, file1 date1 file2 date2 file3 date3 file4 date4 file5 date5 file6 date6 i only want to concatenate the first three which are the most... (4 Replies)
Discussion started by: jlarios
4 Replies

9. Shell Programming and Scripting

Compare 2 sorted files

Hi all, please give me the commands using which i can compare 2 sorted files and get the difference in third file, indiating where the difference is from either file1 or file2. as: File1 (Original file) GARRY JOHN JULIE SAM --------------- File2 DEV GARRY JOHN JOHNIEE (7 Replies)
Discussion started by: varungupta
7 Replies

10. UNIX for Dummies Questions & Answers

Concatenate date to file name

Hi, I have written a script to rename a file, but I can not add the current date (in YYYYMMDD-HHMM format) can you please look at this cript, and help? thanks, #!/usr/local/bin/tcsh -f set server = "$1" set user = "$2" if (-f $server) then \mv $server $server.Saar. endif (4 Replies)
Discussion started by: sierra_aar
4 Replies
Login or Register to Ask a Question