Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

deletion of multiple occurring files

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting deletion of multiple occurring files
# 1  
Old 10-19-2012
deletion of multiple occurring files
Hi all,
In the given file, what I need to do is to select double (or triple) occurring files and delete the one having smaller file size. And would like to keep the one with the big file size.

Code:
518t-rw-r--r-- 1 emily us_cms 101348458 Oct  8 16:43 vgtree_518_0_LHB.root
518t-rw-r--r-- 1 emily us_cms 101348458 Oct  8 17:04 vgtree_518_0_YTs.root
518t-rw-r--r-- 1 emily us_cms 101348200 Oct  8 17:04 vgtree_518_0_YTg.root
519t-rw-r--r-- 1 emily us_cms 101348900 Oct  8 17:04 vgtree_519_0_YTs.root
520t-rw-r--r-- 1 emily us_cms 101788200 Oct  8 17:04 vgtree_520_0_YTs.root

Here in the code,
1.) I would like to delete the following files
vgtree_518_0_YTs.root
vgtree_518_0_YTg.root

And would like to keep all other files as such. Cos there is no double occurance for them.

thanks in advance
emily.
# 2  
Old 10-19-2012
Code:
$ cat test
518t-rw-r--r-- 1 emily us_cms 101348458 Oct  8 16:43 vgtree_518_0_LHB.root
518t-rw-r--r-- 1 emily us_cms 101348458 Oct  8 17:04 vgtree_518_0_YTs.root
518t-rw-r--r-- 1 emily us_cms 101348200 Oct  8 17:04 vgtree_518_0_YTg.root
519t-rw-r--r-- 1 emily us_cms 101348900 Oct  8 17:04 vgtree_519_0_YTs.root
520t-rw-r--r-- 1 emily us_cms 101788200 Oct  8 17:04 vgtree_520_0_YTs.root

$ cat test | sort -rk5 | awk '{print $9}' | awk -F '_' 'x[$2]++'
vgtree_518_0_LHB.root
vgtree_518_0_YTg.root

Mine is a bit of a hack, but it works in the given example. As long as the file is continually growing it'll sort properly and the final awk statement will parse properly.
# 3  
Old 10-19-2012
Quote:
Originally Posted by emily
Hi all,
In the given file, what I need to do is to select double (or triple) occurring files and delete the one having smaller file size. And would like to keep the one with the big file size.

Code:
518t-rw-r--r-- 1 emily us_cms 101348458 Oct  8 16:43 vgtree_518_0_LHB.root
518t-rw-r--r-- 1 emily us_cms 101348458 Oct  8 17:04 vgtree_518_0_YTs.root
518t-rw-r--r-- 1 emily us_cms 101348200 Oct  8 17:04 vgtree_518_0_YTg.root
519t-rw-r--r-- 1 emily us_cms 101348900 Oct  8 17:04 vgtree_519_0_YTs.root
520t-rw-r--r-- 1 emily us_cms 101788200 Oct  8 17:04 vgtree_520_0_YTs.root

Here in the code,
1.) I would like to delete the following files
vgtree_518_0_YTs.root
vgtree_518_0_YTg.root

And would like to keep all other files as such. Cos there is no double occurance for them.

thanks in advance
emily.
I don't understand what constitutes multiple occurrences of a file. When I look at the data it seems that there is one file with a name that matches vgtree_*_LHB.root, one file with a name that matches vgtree_*_YTg.root, and three files with a name that matches
vgtree_*_YTs.root. If that is the case, why do you want to delete vgtree_518_0_YTg.root and why don't you want to delete vgtree_519_0_YTs.root?
# 4  
Old 10-20-2012
Quote:
Originally Posted by Don Cragun
I don't understand what constitutes multiple occurrences of a file. When I look at the data it seems that there is one file with a name that matches vgtree_*_LHB.root, one file with a name that matches vgtree_*_YTg.root, and three files with a name that matches
vgtree_*_YTs.root. If that is the case, why do you want to delete vgtree_518_0_YTg.root and why don't you want to delete vgtree_519_0_YTs.root?
Hi,
Multiple occurance meaning the REPITITION of digits, for example *518* is repeated thrice.

I hope it is clear now?

emily,

---------- Post updated at 05:18 AM ---------- Previous update was at 05:06 AM ----------

Quote:
Originally Posted by Vryali
Code:
$ cat test
518t-rw-r--r-- 1 emily us_cms 101348458 Oct  8 16:43 vgtree_518_0_LHB.root
518t-rw-r--r-- 1 emily us_cms 101348458 Oct  8 17:04 vgtree_518_0_YTs.root
518t-rw-r--r-- 1 emily us_cms 101348200 Oct  8 17:04 vgtree_518_0_YTg.root
519t-rw-r--r-- 1 emily us_cms 101348900 Oct  8 17:04 vgtree_519_0_YTs.root
520t-rw-r--r-- 1 emily us_cms 101788200 Oct  8 17:04 vgtree_520_0_YTs.root

$ cat test | sort -rk5 | awk '{print $9}' | awk -F '_' 'x[$2]++'
vgtree_518_0_LHB.root
vgtree_518_0_YTg.root

Mine is a bit of a hack, but it works in the given example. As long as the file is continually growing it'll sort properly and the final awk statement will parse properly.
Hi,
Thanks for the reply !
I tried it over the following file
Code:
 518t-rw-r--r-- 1 emily04 us_cms   8 Oct 20 05:07 vgtree_518_0_LHB.root
518t-rw-r--r-- 1 emily04 us_cms   8 Oct 20 05:08 vgtree_518_0_YTs.root
519t-rw-r--r-- 1 emily04 us_cms  63 Oct 20 05:13 vgtree_519_0_YTs.root
549t-rw-r--r-- 1 emily04 us_cms 322 Oct 20 05:08 vgtree_549_0_KTs.root
549t-rw-r--r-- 1 emily04 us_cms   9 Oct 20 05:08 vgtree_549_0_uTs.root

So I was expecting the answer as
Code:
 
518t-rw-r--r-- 1 emily04 us_cms   8 Oct 20 05:07 vgtree_518_0_LHB.root
519t-rw-r--r-- 1 emily04 us_cms  63 Oct 20 05:13 vgtree_519_0_YTs.root
549t-rw-r--r-- 1 emily04 us_cms 322 Oct 20 05:08 vgtree_549_0_KTs.root

But with your command line, I got the following
Code:
 $ cat DYJets50.log | sort -rk5 | awk '{print $9}' | awk -F '_' 'x[$2]++'
vgtree_518_0_LHB.root
vgtree_549_0_KTs.root

which is not what I want..Smilie
# 5  
Old 10-20-2012
Code:
sort -rk5 < file | awk -F_ '!x[$3]++'

# 6  
Old 10-20-2012
Hi puma,
It did work but not completely.
For example, in the elog file
Code:
 
-rw-r--r-- 1 emily04 us_cms   8 Oct 20 05:07 vgtree_518_0_LHB.root
-rw-r--r-- 1 emily04 us_cms   8 Oct 20 05:08 vgtree_518_0_YTs.root
-rw-r--r-- 1 emily04 us_cms   9 Oct 20 05:08 vgtree_549_0_uTs.root
-rw-r--r-- 1 emily04 us_cms 322 Oct 20 05:08 vgtree_549_0_KTs.root
-rw-r--r-- 1 emily04 us_cms  63 Oct 20 05:13 vgtree_519_0_YTs.root


After trying your comman,
Code:
 
$ sort -rk5 < elog | awk -F_ '!x[$3]++' > finallog
$ cat finallog 
549t-rw-r--r-- 1 pooja04 us_cms   9 Oct 20 05:08 vgtree_549_0_uTs.root
518t-rw-r--r-- 1 pooja04 us_cms   8 Oct 20 05:08 vgtree_518_0_YTs.root
519t-rw-r--r-- 1 pooja04 us_cms  63 Oct 20 05:13 vgtree_519_0_YTs.root

With file *549* it passes the small size file, but I want the size with the bigger size.

Thanks
emily,
# 7  
Old 10-20-2012
one small correction..

Code:
sort -nrk5 < file | awk -F_ '!x[$3]++'
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Three month old specific files deletion

Hi, I need to delete 3 month old files in my logpath. This path contains several logs and other important files The file names are be like this sl_details.env tomcatfiles_03062014.log application_zur_03.062014.log I need to delete only tomcatfiles logs. I wrote this command. can... (7 Replies)
Discussion started by: nag_sathi
7 Replies

2. Debian

Problem with files/dirs deletion

Hi, The other day i installed a PHP based CMS (modx) on my shell account and noticed that i couldn't delete any of files/dirs it created after. Also, i noticed that all that stuff is owned by username-www instead of username. I tried chown, chmod and using a PHP script to do the same wti... (4 Replies)
Discussion started by: pentago
4 Replies

3. UNIX for Dummies Questions & Answers

Periodical Deletion of files and folders

Hi All, how can Periodical Delete files and folders using shell script. I.e after 1 min i want to delete files and folderns from my home directory. Thanks, Arun (9 Replies)
Discussion started by: arun508.gatike
9 Replies

4. Shell Programming and Scripting

Comparison and deletion of lines between two files

Hi i need an help. I have two files list1 and list2, both contains the server names i want to delete the servers in list2 which were also found in list1. for an eg list2 list1 oradg1 oradg4 oradg2 oradg2 oradg3 ... (5 Replies)
Discussion started by: sudharson
5 Replies

5. Shell Programming and Scripting

Files Deletion After 20 Minutes

Hi, everyone. Could you help me with deletion of files, which are 20 minutes old. I found out how to make deletion for files in that way : find <dir> -mtime n -exec rm -rf "{}" Could you offer your suggestions. Many thanks in advance. (5 Replies)
Discussion started by: KReoN
5 Replies

6. Shell Programming and Scripting

Fake deletion of files

Hi, This is possibly an odd request to do with permissions as I seem to have tied myself up with these! I have the following directory (see below) that contains files that the 'usergrp' user needs to be able to 'delete' files from. drwxr-s--- 2 usergrp usergrp 512 16 Feb 14:37... (2 Replies)
Discussion started by: Peejay
2 Replies

7. Shell Programming and Scripting

Multiple Files deletion in perl

Hello Friends, I want to delete all the "*.trg" files in a directory but i don't want to do it by system("rm -r *.trg"); Can i do it thru unlink or by any other mean Thanks, Pulkit (1 Reply)
Discussion started by: pulkit
1 Replies

8. Shell Programming and Scripting

Script for automatic deletion of old files

Hi, I have a folder with limited space. I do not have provisions to increase the space for this folder. So i have to delete files which are more than 1 month old automatically. But, i need to maintain the files created by 4 users and delete all the other files automatically which is more than 1... (4 Replies)
Discussion started by: vivek_scv
4 Replies

9. UNIX for Dummies Questions & Answers

Deletion of log files.

We have log files dating back to 2004. I need to write an interative script that will request the user for how many months he needs to keep the log files and remove all the remaing log files after that month. Supposing we are now in June 2006 , if teh user request to keep log file for the last 3... (1 Reply)
Discussion started by: Geeta
1 Replies

10. Shell Programming and Scripting

Regarding deletion of old files

Hi, I have a list of directories which contain old files that are to be deleted. I put the list of all directories in a txt file and it is being read by a script Iam searching for the files older than 60 days using mtime and then deleting it But all the files are getting deleted... (3 Replies)
Discussion started by: Chidvilas
3 Replies
Login or Register to Ask a Question