Gunzip,grep and zip across folders in a subdirectory
I have a simple function that greps within a folder into files that are already gunzipped. This was already written for me so from my understanding fn=$1 specifies that it will look into any file in that folder that is gunzipped and the rest of the script greps the data its looking for and puts it into a new file with '_woodsmoke'.
The problem is now I need to write something that greps across three different folders within a directory (part1_3b, part2_eb, part3_scsdmd). These 3 folders represent data in CA divided by 3 sections. The files in these folders ...although containing different data..have the same name which is why they aren't combined into 1 folder.
For every month of the year and 3 days of the week I have files written as such:
I need it to look across all 3 folders, gunzip & grep data for matching months and days of the week to produce one file per month per day of the week (36 files total) into preferentially a new sub-directory with medsarea.201001sat.v008a.asc_woodsmoke as an example of a Saturday in the first month...but I also need to make sure the original files get zipped back up at the end.
Any tips/pointers/ideas on how to go about this? I am new to shell scripting.
Thanks!
Last edited by Don Cragun; 10-04-2015 at 04:19 PM..
Reason: Add CODE and ICODE tags.
Are there always exactly 36 gzipped files in each of your three subdirectories?
Are the names of those 36 files in each of those three subdirectories always identical?
Is it important that lines containing each of the 5 patterns you are grepping from those files be grouped together, or is it OK to put data in the output file in the order in which those lines appear in the input files? If the order matters, is the output from the three different input files also required to be segregated, or can each input file be processed completely before processing the next input file?
There are more than 36 gzipped files in each folder
There are file names such as:
I don't need these particular gzipped files above I only want ones listed as:
36 is the total number of output files I need to have across all three folders.
1/month x 3 days of the week
The names of the particular files that I want are identical across all 3 folders.
Order doesn't matter. These are codes that are grepping for source classification codes or emission inventory codes for a particular category of emissions. Its likely most files will contain both of the bottom two codes over the top 3.
Each input file can be processed completely to check for the codes listed before processing the next input file.
It is important the 3 identical files that represent a given month and a given day of the week be outputted to one file that contains any and all data with the particular codes listed. This would give a complete inventory of that month on that day for all of CA.
Hi Folks -
Happy Friday and I hope you all are well!
What's the easiest way to tar.gz / zip all direct children directories in a folder that have the string _EBASE (suffix)?
Thank you! (6 Replies)
I have a directory that is in the below order (the --- is not part of the directory tree, only there to help illustrate:
DATE --- main level
Folder1 --- level under DATE
plugin_out --- level under Folder1
variantCaller_out.40 --- level under plugin_out
001,002,003 --- level under... (3 Replies)
Hi,
How do I write a script that will automatically find and zip any folder to the same name in a directory and affected recursively?. The zip file should be place in the same directory where the source folder is. Help is appreciated. Thanks in advance.
ex:
Orange Crayon
Blue Crayon... (5 Replies)
HI All
I have one master folder : ABCXYZ
I have sub folder in there :
AB
XY
AZ
AC
PR
AL
Now i want to copy AB , PR ,AL in to one new folder and zip it with time stamp.
like
Pre_02192013_12_32.zip
Zip folder should be in master folder. (2 Replies)
Hello,
I need a simple script to Auto-detect new files and folders in the directory.
And then I need to zip the new files and bzip2 new folders and move them out of that folder where I am detecting changes to the other folder.
Remember, I need simple one. If anyone could do it fast, I may... (1 Reply)
Hi all
i'm new in KSH,
i want to write a script to grep a logs files and redirecting the result into a relative subdirectory/file.txt that must be created near to each log file
my begin script is :
find ./logs -type f -name "*.log" -exec grep error {} \;
how i can perform that modest... (10 Replies)
Hi,
I have a zipped Oralce software (linux.x64_11gR1_database.zip) and I need to unzip it. On the Linux server, I only see the gunzip utility listed:
$ ls -ltr *zip*
-rwxr-xr-x 3 root root 60320 Jan 15 2008 gzip
-rwxr-xr-x 3 root root 60320 Jan 15 2008 gunzip
Can I use the command... (1 Reply)
Hi,
I have a folder that contains a few sub-folders. I would like to zip that folder and KEEP the subfolders. What it does at the moment is taking all the files within the subfolders and zipping them into one big file (zip -r ...). Does anyone know the UNIX command to keep the subfolders in the... (3 Replies)
I'm using the following command to zip a project file, but when it finishes, the resulting zip file contains all the directories above the file I wanted zipped, myapp.app, each one empty until you get to the actual app.
zip -r myapp.app.zip ... (0 Replies)