Find big file include current date


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Find big file include current date
# 8  
Old 05-20-2014
Do you want to have a list like this?

Code:
/dir1 2014-05-19
/dir2 2014-05-19
/dir3 2014-05-19

or do you want a list like:
Code:
2014-05-19
/dir1
/dir2
/dir3

Could you show the output of what you want?
# 9  
Old 05-20-2014
I want to view like this:
Code:
/dir1 20May2014 size 150MB
/dir2 20May2014 size 400MB
/dir3 20May2014 size 650MB

This folder together will put in tar.gz and split into 50MB each. This is run by crontab.
# 10  
Old 05-20-2014
That would produce a file named larger_than_100M.list which it has the information you are requesting, formatted the way you posted.

Code:
du -sm /mnt/* | awk -v d=`date "+%d%b%Y"` '$1 > 100 {printf "%s %s size %sMB\n", $2, d, $1}' > larger_than_100M.list



Quote:
This folder together will put in tar.gz and split into 50MB each. This is run by crontab.
Not quite sure what you want to tar. The above command only provides a list of files that are 100MB and larger, and the date that the query was performed.
This User Gave Thanks to Aia For This Post:
# 11  
Old 05-21-2014
Quote:
Originally Posted by Aia
That would produce a file named larger_than_100M.list which it has the information you are requesting, formatted the way you posted.

Code:
du -sm /mnt/* | awk -v d=`date "+%d%b%Y"` '$1 > 100 {printf "%s %s size %sMB\n", $2, d, $1}' > larger_than_100M.list


Not quite sure what you want to tar. The above command only provides a list of files that are 100MB and larger, and the date that the query was performed.
The result that list in larger_than_100M.list file, i want to tar all this folder and split. That is what i want to do.
# 12  
Old 05-21-2014
I tell you what. How about showing you some examples of how to tar and split and then you try by yourself? If you have trouble you can continue posting here.

This is how you tar and gzip a directory
Code:
tar vczf filename.tar.gz directory/

That creates a file named filename.tar.gz

Now you can divide it with a command properly named split
Code:
split -b 50m "filename.tar.gz" "filename.tar.gz.part-"

That would create as many files named filename.tar.gz.part-aa, filename.tar.gz.part-ab, filename.tar.gz.part-ac, 50MB at a time.

I suggest you do some test with something small first, you can always change the -b 50m to another size
# 13  
Old 05-21-2014
I know how to tar and split. But maybe you not read earlier post. I already told you, i want this bash script on crontab. It's mean it collect all data by running deamon.

If by manual, of course it's more easier. I can put all directory this have size more 100MB. And split them after put in tar.gz.

This script i want to put on cron and it's will run everyday. Seek new folder that have size more 100MB.

Our system have using that upload movie, document. The movie usually more that 100MB size.

The system also create directory and subdirectory. So we want to transfer big folder to other server that located on DC B, far away from our DC A.

Both DC connected using point-to-point fiber optic. It's mean not going to cloud or what ever public line. More than to dedicated line.

If i want to run manually, i will not asking expert here to do scripting. We have so many subdir. If using manually, it waste time. We want to reduce time, seek manually big folder.

---------- Post updated 05-21-14 at 09:14 PM ---------- Previous update was 05-20-14 at 10:59 PM ----------

From list folder below:
Code:
drwxrwxrwx 3 wwwrun  www     3896 May 20 15:36 LY2014000411
drwxrwxrwx 3 wwwrun  www     3896 May 20 15:42 LY2014000412
drwxrwxrwx 3 docuflo users   3896 May 20 15:54 AR2014000564
drwxrwxrwx 4 wwwrun  www     3896 May 20 17:00 LY2014000407
drwxrwxrwx 3 docuflo users   3896 May 20 17:00 LY2014000565
drwxrwxrwx 3 docuflo users   3896 May 20 17:00 LY2014000563
drwxrwxrwx 3 docuflo users   3896 May 20 17:38 LY2014000402
drwxrwxrwx 4 wwwrun  www     3896 May 20 17:39 LY2014000403
drwxrwxrwx 4 wwwrun  www     3896 May 20 17:51 LY2014000415
drwxrwxrwx 5 wwwrun  www     3896 May 21 09:50 LY2013000576
drwxrwxrwx 3 wwwrun  www     3896 May 21 12:28 LY2013000581
drwxrwxrwx 3 wwwrun  www     3896 May 21 12:43 LY2013000582
-rwxrwxrwx 1 root    root   16756 May 21 14:50 larger_than_100M.list
drwxrwxrwx 3 wwwrun  www     3896 May 21 14:59 AR2013001068
drwxrwxrwx 3 wwwrun  www     3896 May 21 15:07 AR2013001067
drwxrwxrwx 3 wwwrun  www     3896 May 21 15:17 AR2013001065
drwxrwxrwx 3 wwwrun  www     3896 May 21 15:21 AR2013001066
drwxrwxrwx 3 wwwrun  www     3896 May 21 15:34 FM2013000427

How to grep a date May 21 in bash script? List is directory. And i want to transfer using scp protocol.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Replace date in file every day with current date

I Have text like XXX_20190908.csv.gz need to replace Only date in this format with current date every day Thanks! (1 Reply)
Discussion started by: yamasani1991
1 Replies

2. UNIX for Beginners Questions & Answers

“sed” replace date in text file with current date

We want to call a parameter file (.txt) where my application read dynamic values when the job is triggered, one of such values are below: abc.txt ------------------ Code: line1 line2 line3 $$EDWS_DATE_INSERT=08-27-2019 line4 $$EDWS_PREV_DATE_INSERT=08-26-2019 I am trying to write a... (3 Replies)
Discussion started by: pradeepp
3 Replies

3. Shell Programming and Scripting

Find the file created on current date

Hi All, I'm trying to find a file which is created on current day.... I searched in unix.com and i found, below command. find /land/ -mtime -1 -type f -print | grep "FF_Member_STG.dat" The command checks if the file with name "FF_Member_STG.dat" is created today then exit else proceed. ... (3 Replies)
Discussion started by: ace_friends22
3 Replies

4. Shell Programming and Scripting

Get file name which include the current date

Hi, I'm trying to get the name of a file that has the current date in the name. The filename is on the form A<current date>01.DC, for example A2012110501.DC I have this script so far date +%y%m%d D=$(date +%y%m%d) N=A20$D echo $N N2={$N}01.DC echo $N2 And gets the following... (12 Replies)
Discussion started by: Immelstorn
12 Replies

5. UNIX for Dummies Questions & Answers

Delete a row from a file if one column containing a date is greater than the current system date

Hello gurus, I am hoping someone can help me with the required code/script to make this work. I have the following file with records starting at line 4: NETW~US60~000000000013220694~002~~IT~USD~2.24~20110201~99991231~01~01~20101104~... (4 Replies)
Discussion started by: chumsky
4 Replies

6. UNIX for Advanced & Expert Users

In ksh find out the timestamp of current date?

Normally we can use %s to find out the time in second since 1970. But in my ksh, this format option is not available. Example- date +%s 1268103151 above script command won't work in ksh. Can you guys provide its equivalent ? (3 Replies)
Discussion started by: boy18nj
3 Replies

7. Shell Programming and Scripting

How to find the create time of a file if current date is in next month

Hi All, I want to find the time diffrence between currnt time and "abc.txt" file create time. I have solve that but if the abc.txt file created last month then is there any process to find the difftent? Exp: Create time of abc.txt is "Apr 14 06:48" and currect date is "May 17 23:47".... (1 Reply)
Discussion started by: priyankak
1 Replies

8. Shell Programming and Scripting

How to find a date which is 7 days past when given current date

hii all. I have to get the date of the 7th day past from the current date. if i give the current date as sep 3 then i must get the date as 27th of august. can we get the values from the "cal" command. cal | awk '{print $2}' will this type of command work. actually my need is if today is... (17 Replies)
Discussion started by: ladtony
17 Replies

9. Shell Programming and Scripting

Perl: Extracting date from file name and comparing with current date

I need to extract the date part from the file name (20080221 in this ex) and compare it with the current date and delete it, if it is a past date. $file = exp_ABCD4_T-2584780_upto_20080221.dmp.Z really appreciate any help. thanks mkneni (4 Replies)
Discussion started by: MKNENI
4 Replies

10. UNIX for Dummies Questions & Answers

how to find files less than the current date

Hai, i have one directory contains 100 files .each file name like xvb_dateformat.i want find which file names are xvb_lessthan or equal to currentdate. any one give the solution. regards (4 Replies)
Discussion started by: mallikarjuna
4 Replies
Login or Register to Ask a Question