Script to search specific folders dates /mm/dd/ structure


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Script to search specific folders dates /mm/dd/ structure
# 1  
Old 08-26-2011
Question Script to search specific folders dates /mm/dd/ structure

Hi,
I have a script that handles a huge amount of log files from many machines and copies it into a SAN location with the following directory structure:
Code:
/SAN/machinenames/yyyy/m/d

so for example
Code:
/SAN/hosta/2011/3/12/files*

Now I am writing a bash script to search for files between to date ranges and I do not care currently which hosts I search in so a find statement like:
Code:
find  /SAN/*/2011/2/3

that increments the days then months is what I am writing. The user inputs a start date and an end date, e.g. 04.01.2011 and I strip out leading zeros and convert the day, month and year into variables. User can only search between 1 years so we are left with a start day, start month, end day, end month and a year variable.

I been trying for several hours on and off to write the logic to get this implemented but every time I think I figured it out and test my script I notice my logic is flowed. Smilie help. There must be a better way to do this then using while and if statements.

Code:
WWstartmonth=$monS
	WWcstartday=$dayS
	while [ "$monS" -le "$monE" ] ##if start month is less or equal than end month
		do
			WWcurrentday=1 
			##fine-----------------------------------------------------------------
			if [ "$monS" == "$monE" ];then
				while [ "$dayS" -le "$dayE" ]
					do
						echo "find files for month and currentday and do gzcat into file with grep"
						echo "for logfile in find SAN/*/$yeaS/$monS/$dayS  ;"
					dayS=$((dayS+1))
					done
			#---------------------------------------------------------------------
			else
				if [ "$WWstartmonth" == "$monS" ];then
					while [ "$dayS" -le 31 ]
						do
							echo "find files for month and currentday and do gzcat into file with grep"
							echo "for logfile in find SAN/*/$yeaS/$monS/$dayS  ;"
						dayS=$((dayS+1))						
						done	
				fi
				while [ "$WWcurrentday" -le 31 ]
					do
						echo "find files for month and currentday and do gzcat into file with grep"
						echo "for logfile in find SAN/*/$yeaS/$monS/$WWcurrentday  ;"
					WWcurrentday=$((WWcurrentday+1))						
					done
			
			
			fi
		monS=$((monS+1))  ##increment start search month by one
		done

PS: doesnt matter if a day doesnt exists aka if I search for day 31 in February..
# 2  
Old 08-26-2011
If the user can only search within the same year and you have already edited the input data:
Code:
#!/usr/bin/ksh
typeset -i mMM mDD mDDto
mMM=${monS}
while [[ ${mMM} -le ${monE} ]]; do
  if [[ ${mMM} -eq ${monS} ]]; then
    mDD=${dayS}
  else
    mDD=1
  fi
  if [[ ${mMM} -eq ${monE} ]]; then
    mDDto=${dayE}
  else
    mDDto=31
  fi
  while [[ ${mDD} -le ${mDDto} ]]; do
    find ${yearS}/${mMM}/${mDD} -depth -type f -exec ls -l {} \; 2> /dev/null
    mDD=${mDD}+1
  done
  mMM=${mMM}+1
done

This User Gave Thanks to Shell_Life For This Post:
# 3  
Old 08-29-2011
unfortunately I do not have the ksh shell so I do not have a typeset option though it does look interesting.

also what does mMM=${monS} do?

I think mMM is a specific month format but what does it look like?
# 4  
Old 08-29-2011
Quote:
unfortunately I do not have the ksh shell so I do not have a typeset option though it does look interesting.
Then remove the reference to it: #!/usr/bin/ksh

Quote:
also what does mMM=${monS} do?
It assigns the value of your variable "monS" to variable "mMM".

Did you at least tried to run the script?
# 5  
Old 08-30-2011
will try tomorrow when I go to work. Holiday today in the UK Smilie

---------- Post updated 08-30-11 at 03:49 PM ---------- Previous update was 08-29-11 at 08:59 PM ----------

Hi,

it works Smilie

Had to change to +1 additions to mMM=$((mMM+1)) but that was it.

Thank you so much
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Copying specific file types to specific folders

I am trying to write a script that cycles through a folder containing many folders and when inside each one it's supposed to copy all the .fna.gz files to a folder elsewhere if the file and the respective folder have the same name. for fldr in /home/playground/genomes/* ; do find .... (8 Replies)
Discussion started by: Mr_Keystrokes
8 Replies

2. Shell Programming and Scripting

Search for specific file type in subdirectory with multiple folders

I have a directory that is in the below order (the --- is not part of the directory tree, only there to help illustrate: DATE --- main level Folder1 --- level under DATE plugin_out --- level under Folder1 variantCaller_out.40 --- level under plugin_out 001,002,003 --- level under... (3 Replies)
Discussion started by: cmccabe
3 Replies

3. Shell Programming and Scripting

Copying files from various folders to similar folder structure in another location

Hi, I need to write a script the has to copy the files from folders and subfolders to the same folder structure located in another location. Ex: mainfolder1 file1,file2,file3 subfolder1(file1,etc) subfolder2(file1,etc) to another folder location of same folder structure. rsync is not... (7 Replies)
Discussion started by: Raji Perumal
7 Replies

4. Shell Programming and Scripting

Search pattern on logfile and search for day/dates and skip duplicate lines if any

Hi, I've written a script to search for an Oracle ORA- error on a log file, print that line and the .trc file associated with it as well as the dateline of when I assumed the error occured. In most it is the first dateline previous to the error. Unfortunately, this is not a fool proof script.... (2 Replies)
Discussion started by: newbie_01
2 Replies

5. UNIX for Advanced & Expert Users

Copy Structure excluding some folders

Hi I want to copy the structure from one place to another. -> cd /hol/; -> find . -type d | cpio -pvdm /abc/cat; while copying the structure I want to exclude some directories like test1 and Test. I have read somewhere that this can be done with -prune option. Could anyone... (2 Replies)
Discussion started by: soumodeep123
2 Replies

6. Shell Programming and Scripting

Shell script to generate daily crontab entries between to specific dates

Hi, I am currently writing a shell script to enter daily reoccurring jobs into the crontab. I want to be able to specify any two dates for which I want these daily jobs to run between. Below is an example of what I need to be written to crontab. # Give a start day of 21/10/2009 09:00... (2 Replies)
Discussion started by: JM_86
2 Replies

7. Shell Programming and Scripting

Question on reading dates in folders/files

Hi All, I have some folder that are named "FOLDERYYYYMM". I'm trying to piece together a .sh script that will look for the folder with the date. How can I get shell to see the date? (3 Replies)
Discussion started by: bbbngowc
3 Replies

8. Shell Programming and Scripting

Need script to generate all the dates in DDMMYY format between 2 dates

Hello friends, I am looking for a script or method that can display all the dates between any 2 given dates. Input: Date 1 290109 Date 2 010209 Output: 300109 310109 Please help me. Thanks. :):confused: (2 Replies)
Discussion started by: frozensmilz
2 Replies

9. Shell Programming and Scripting

Copying specific files from remote m/c to specific folders

Hi All, I am trying to rsync some of the latest files from remote m/c to my local linux box. Folder structure in my remote m/c looks like this /pub/Nightly/Package/ROLL/WIN /pub/Nightly/Package/SOLL/sol /pub/Nightly/Package/SOLL/linux Each of the folder contains gzip files which on daily... (0 Replies)
Discussion started by: jhoomsharabi
0 Replies

10. Programming

Search attributes in one structure using the values from another structure

Hello Groups I am trying to find out ways of comparing a value from a 'c' structure to a value in another 'C' structure. the 'C' structure can be a List or liked list as it contains lot many records. if we loop it in both the structures it is going to consume time. I am looking for a simple... (3 Replies)
Discussion started by: dhanamurthy
3 Replies
Login or Register to Ask a Question