Remove duplicate files in same directory


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Remove duplicate files in same directory
# 8  
Old 02-05-2010
Quote:
Originally Posted by cmf1985
I'm beginning to get a bit confused about what you want... Do you want to have the same files on the mail server and the backup server (as in, get a list of all the files currently on the mail server and delete from the backup server anything that's not in that list)?
Yes.

for e,g if I have xyz & abc on mailserver and xyzRS , xyzR , xyz , abcRSF , abcR, abc
on the backup server. then
xyzRS , xyzR , abcRSF, abcR must be deleted from the backup server.

Eventually on mailserver : xyz & abc
and on backup server : xyz & abc
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Remove all but newest two files (Not a duplicate post)

TARGET_DIR='/media/andy/MAXTOR_SDB1/Ubuntu_Mate_18.04/' REGEX='{4}-{2}-{2}_{2}:{2}' # regular expression that match to: date '+%Y-%m-%d_%H:%M' LATEST_FILE="$(ls "$TARGET_DIR" | egrep "^${REGEX}$" | tail -1)" find "$TARGET_DIR" ! -name "$LATEST_FILE" -type f -regextype egrep -regex... (7 Replies)
Discussion started by: drew77
7 Replies

2. Windows & DOS: Issues & Discussions

Remove duplicate lines from text files.

So, I have text files, one "fail.txt" And one "color.txt" I now want to use a command line (DOS) to remove ANY line that is PRESENT IN BOTH from each text file. Afterwards there shall be no duplicate lines. (1 Reply)
Discussion started by: pasc
1 Replies

3. Shell Programming and Scripting

Remove duplicate files

Hi, In a directory, e.g. ~/corpus is a lot of files and subdirectories. Some of the files are named: 12345___PP___0902___AA.txt 12346___PP___0902___AA. txt 12347___PP___0902___AA. txt The amount of files varies. I need to keep the highest (12347___PP___0902___AA. txt) and remove... (5 Replies)
Discussion started by: corfuitl
5 Replies

4. Shell Programming and Scripting

[uniq + awk?] How to remove duplicate blocks of lines in files?

Hello again, I am wanting to remove all duplicate blocks of XML code in a file. This is an example: input: <string-array name="threeItems"> <item>item1</item> <item>item2</item> <item>item3</item> </string-array> <string-array name="twoItems"> <item>item1</item> <item>item2</item>... (19 Replies)
Discussion started by: raidzero
19 Replies

5. Shell Programming and Scripting

perl/shell need help to remove duplicate lines from files

Dear All, I have multiple files having number of records, consist of more than 10 columns some column values are duplicate and i want to remove these duplicate values from these files. Duplicate values may come in different files.... all files laying in single directory.. Need help to... (3 Replies)
Discussion started by: arvindng
3 Replies

6. Shell Programming and Scripting

Remove Duplicate Files On Remote Servers

Hello, I wrote a basic script that works however I am was wondering if it could be sped up. I am comparing files over ssh to remove the file from the source server directory if a match occurs. Please Advise me on my mistakes. #!/bin/bash for file in `ls /export/home/podcast2/"$1" ` ; do ... (5 Replies)
Discussion started by: jaysunn
5 Replies

7. Shell Programming and Scripting

Remove duplicate files based on text string?

Hi I have been struggling with a script for removing duplicate messages from a shared mailbox. I would like to search for duplicate messages based on the “Message-ID” string within the messages files. I have managed to find the duplicate “Message-ID” strings and (if I would like) delete... (1 Reply)
Discussion started by: spangberg
1 Replies

8. Shell Programming and Scripting

remove all duplicate lines from all files in one folder

Hi, is it possible to remove all duplicate lines from all txt files in a specific folder? This is too hard for me maybe someone could help. lets say we have an amount of textfiles 1 or 2 or 3 or... maximum 50 each textfile has lines with text. I want all lines of all textfiles... (8 Replies)
Discussion started by: lowmaster
8 Replies

9. Shell Programming and Scripting

script that detects duplicate files in directory

I need help with a script which accepts one argument and goes through all the files under a directory and prints a list of possible duplicate files As its output, it prints zero or more lines, each one containing a space-separated list of filenames. All the files listed on one line have the same... (1 Reply)
Discussion started by: trueman82
1 Replies

10. Shell Programming and Scripting

remove duplicate files in a directory

Hi ppl. I have to check for duplicate files in a directory . the directory has following files /the/folder /containing/the/file a1.yyyymmddhhmmss a1.yyyyMMddhhmmss b1.yyyymmddhhmmss b2.yyyymmddhhmmss c.yyyymmddhhmmss d.yyyymmddhhmmss d.yyyymmddhhmmss where the date time stamp can be... (1 Reply)
Discussion started by: asinha63
1 Replies
Login or Register to Ask a Question
DELUSER(8)						      System Manager's Manual							DELUSER(8)

NAME
deluser, delgroup - remove a user or group from the system SYNOPSIS
deluser [options] [--force] [--remove-home] [--remove-all-files] [--backup] [--backup-to DIR] user deluser --group [options] group delgroup [options] [--only-if-empty] group deluser [options] user group COMMON OPTIONS [--quiet] [--system] [--help] [--version] [--conf FILE] DESCRIPTION
deluser and delgroup remove users and groups from the system according to command line options and configuration information in /etc/deluser.conf and /etc/adduser.conf. They are friendlier front ends to the userdel and groupdel programs, removing the home directory as option or even all files on the system owned by the user to be removed, running a custom script, and other features. deluser and del- group can be run in one of three modes: Remove a normal user If called with one non-option argument and without the --group option, deluser will remove a normal user. By default, deluser will remove the user without removing the home directory, the mail spool or any other files on the system owned by the user. Removing the home directory and mail spool can be achieved using the --remove-home option. The --remove-all-files option removes all files on the system owned by the user. Note that if you activate both options --remove-home will have no effect because all files including the home directory and mail spool are already covered by the --remove-all-files option. If you want to backup all files before deleting them you can activate the --backup option which will create a file username.tar(.gz|.bz2) in the directory specified by the --backup-to option (defaulting to the current working directory). Both the remove and backup options can also be activated for default in the configuration file /etc/deluser.conf. See deluser.conf(5) for details. If you want to remove the root account (uid 0), then use the --force parameter; this may prevent to remove the root user by accident. If the file /usr/local/sbin/deluser.local exists, it will be executed after the user account has been removed in order to do any local cleanup. The arguments passed to deluser.local are: username uid gid home-directory Remove a group If deluser is called with the --group option, or delgroup is called, a group will be removed. Warning: The primary group of an existing user cannot be removed. If the option --only-if-empty is given, the group won't be removed if it has any members left. Remove a user from a specific group If called with two non-option arguments, deluser will remove a user from a specific group. OPTIONS
--conf FILE Use FILE instead of the default files /etc/deluser.conf and /etc/adduser.conf --group Remove a group. This is the default action if the program is invoked as delgroup. --help Display brief instructions. --quiet Suppress progress messages. --system Only delete if user/group is a system user/group. This avoids accidentally deleting non-system users/groups. Additionally, if the user does not exist, no error value is returned. This option is mainly for use in Debian package maintainer scripts. --backup Backup all files contained in the userhome and the mailspool-file to a file named /$user.tar.bz2 or /$user.tar.gz. --backup-to Place the backup files not in / but in the directory specified by this parameter. This implicitly sets --backup also. --remove-home Remove the home directory of the user and its mailspool. If --backup is specified, the files are deleted after having performed the backup. --remove-all-files Remove all files from the system owned by this user. Note: --remove-home does not have an effect any more. If --backup is specified, the files are deleted after having performed the backup. --version Display version and copyright information. RETURN VALUE
0 The action was successfully executed. 1 The user to delete was not a system account. No action was performed. 2 There is no such user. No action was performed. 3 There is no such group. No action was performed. 4 Internal error. No action was performed. 5 The group to delete is not empty. No action was performed. 6 The user does not belong to the specified group. No action was performed. 7 You cannot remove a user from its primary group. No action was performed. 8 The required perl-package 'perl modules' is not installed. This package is required to perform the requested actions. No action was performed. 9 For removing the root account the parameter "--force" is required. No action was performed. FILES
/etc/deluser.conf SEE ALSO
deluser.conf(5), adduser(8), userdel(8), groupdel(8) COPYRIGHT
Copyright (C) 2000 Roland Bauerschmidt. Modifications (C) 2004 Marc Haber and Joerg Hoh. This manpage and the deluser program are based on adduser which is: Copyright (C) 1997, 1998, 1999 Guy Maor. Copyright (C) 1995 Ted Hajek, with a great deal borrowed from the original Debian adduser Copyright (C) 1994 Ian Murdock. deluser is free software; see the GNU General Public Licence version 2 or later for copying conditions. There is no warranty. Debian GNU/Linux Version 3.113+nmu3 DELUSER(8)