Find and split the list of files with suffiz of seg** | Unix Linux Forums | Shell Programming and Scripting

  Go Back    


Shell Programming and Scripting Post questions about KSH, CSH, SH, BASH, PERL, PHP, SED, AWK and OTHER shell scripts and shell scripting languages here.

Find and split the list of files with suffiz of seg**

Shell Programming and Scripting


Closed Thread    
 
Thread Tools Search this Thread Display Modes
    #1  
Old 01-29-2013
Satish Shettar Satish Shettar is offline
Registered User
 
Join Date: Jan 2013
Last Activity: 11 February 2013, 11:49 PM EST
Posts: 4
Thanks: 0
Thanked 0 Times in 0 Posts
Find and split the list of files with suffiz of seg**

Hi,.

I am writing a script to get the new files and split them.


Requirement
Find the new files under the path "/wload/scmp/app/data/OAS" (There are 5 sub folders).
Gunzip the files which are having .gz suffix.
Put the list of files in the filename in the format "fiels_to_split_2013012911300.txt where as the numeric part is date and time.
Perform the split command on the files to split each file containing 50000 lines.
The output of the split filename should have the same file with suffix seg** where ** can be either numbers or alphabets.


I got the code from Pikk45 and tried to add/modify as per the need.

Code:
find /wload/scmp/app/data/OAS -name "*.gz" -size +100000000 -type f -mtime -1 -exec gunzip {} \;
FILENAME=new_fiels_to_split$(date +"%Y%m%d%H%M").txt
find /wload/scmp/app/data/OAS -type f -mtime -1 > /wload/scmp/app/data/OAS/$FILENAME
FILES=/wload/scmp/app/data/OAS/$FILENAME
FIRST=1
FILE=1
SPL=50000
while read file; do
 split -l 50000 "$FILENAME" "$FILENAME".seg
 done < ${FILES}
FIRST=`expr $FIRST + 50000`
SPL=`expr $SPL + 50000`
FILE=`expr $FILE + 1`

for example
the source files


Code:
-rw-rw-r-- 1 scmpadm scmpuser 755861762 Jan 26 20:04 activity.log.20130125.gz
-rw-rw-r-- 1 scmpadm scmpuser 0 Jan 26 20:05 trigger.activity.log.20130125
-rw-rw-r-- 1 scmpadm scmpuser 913911244 Jan 27 20:24 activity.log.20130126.gz
-rw-rw-r-- 1 scmpadm scmpuser 0 Jan 27 20:25 trigger.activity.log.20130126
-rw-rw-r-- 1 scmpadm scmpuser 329351168 Jan 28 19:10 activity.log.20130127.gz

After gunzip

Code:
-rw-rw-r-- 1 scmpadm scmpuser 3012939863 Jan 28 19:10 activity.log.20130127

Files after split

Code:
-rw-rw---- 1 scmpadm scmpuser 49877675 Jan 29 07:04 activity.log.20130127.segaa
-rw-rw---- 1 scmpadm scmpuser 47552800 Jan 29 07:07 activity.log.20130127.segab
-rw-rw---- 1 scmpadm scmpuser 45867421 Jan 29 07:08 activity.log.20130127.segac
-rw-rw---- 1 scmpadm scmpuser 48985303 Jan 29 07:09 activity.log.20130127.segad
-rw-rw---- 1 scmpadm scmpuser 50846638 Jan 29 07:10 activity.log.20130127.segae
-rw-rw---- 1 scmpadm scmpuser 50946829 Jan 29 07:13 activity.log.20130127.segaf
-rw-rw---- 1 scmpadm scmpuser 50968144 Jan 29 07:14 activity.log.20130127.segag
-rw-rw---- 1 scmpadm scmpuser 51758156 Jan 29 07:18 activity.log.20130127.segah
-rw-rw---- 1 scmpadm scmpuser 52290589 Jan 29 07:23 activity.log.20130127.segai
-rw-rw---- 1 scmpadm scmpuser 19329024 Jan 29 07:26 activity.log.20130127.segaj


Last edited by Satish Shettar; 01-29-2013 at 06:20 AM.. Reason: Changed several icode tags to code tags
Sponsored Links
Closed Thread

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

More UNIX and Linux Forum Topics You Might Find Helpful
Thread Thread Starter Forum Replies Last Post
find list of files from a list and copy to a directory manishabh Shell Programming and Scripting 3 09-13-2009 06:03 PM
Find files and display only directory list containing those files r7p UNIX for Dummies Questions & Answers 4 06-05-2009 08:46 AM
I need a script to find socials in files and output a list of those files NewSolarisAdmin Shell Programming and Scripting 1 02-19-2009 12:01 PM
How to find the list of files senthilk615 Shell Programming and Scripting 2 06-19-2007 01:43 AM
list read only files using find vivekshankar UNIX for Dummies Questions & Answers 1 05-26-2005 04:47 PM



All times are GMT -4. The time now is 07:49 PM.