Sponsored Content
Top Forums UNIX for Advanced & Expert Users AIX find duplicate backup files Post 303043327 by MadeInGermany on Friday 24th of January 2020 09:09:45 AM
Old 01-24-2020
I guess you mean the file names?
Then consider sort and unique to sort out duplicates.
Show the duplicate file names:
Code:
$ sort your_example | uniq -d
server2_1-23-2020

Show the non-duplicates:
Code:
$ sort your_example | uniq -u
server1_1-20-2020
server1_1-21-2020
server1_1-22-2020
server2_1-20-2020
server2_1-21-2020
server2_1-22-2020

Show each file name once
Code:
$ sort your_example | uniq
$ sort -u your_example
server1_1-20-2020
server1_1-21-2020
server1_1-22-2020
server2_1-20-2020
server2_1-21-2020
server2_1-22-2020
server2_1-23-2020

 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

how to find duplicate files with find ?

hello all I like to make search on files , and the result need to be the files that are duplicated? (8 Replies)
Discussion started by: umen
8 Replies

2. Shell Programming and Scripting

Find Duplicate files, not by name

I have a directory with images: -rw-r--r-- 1 root root 26216 Mar 19 21:00 020109.210001.jpg -rw-r--r-- 1 root root 21760 Mar 19 21:15 020109.211502.jpg -rw-r--r-- 1 root root 23144 Mar 19 21:30 020109.213002.jpg -rw-r--r-- 1 root root 31350 Mar 20 00:45 020109.004501.jpg -rw-r--r-- 1 root... (2 Replies)
Discussion started by: Ikon
2 Replies

3. UNIX for Dummies Questions & Answers

Not Listing Duplicate Backup Files

I've searched about a bit and more than likely, I'm not entering the correct search phrase, but my question is as follows: When a user does an 'ls' (and whatever various flags needed) and in the results, we see duplicate or backup files from editing. IE: file1.txt file1.txt~ Is there a... (3 Replies)
Discussion started by: Brusimm
3 Replies

4. Shell Programming and Scripting

Find duplicate files

What utility do you recommend for simply finding all duplicate files among all files? (4 Replies)
Discussion started by: kiasas
4 Replies

5. Shell Programming and Scripting

Find duplicate files by file size

Hi! I want to find duplicate files (criteria: file size) in my download folder. I try it like this: find /Users/frodo/Downloads \! -type d -exec du {} \; | sort > /Users/frodo/Desktop/duplicates_1.txt; cut -f 1 /Users/frodo/Desktop/duplicates_1.txt | uniq -d | grep -hif -... (9 Replies)
Discussion started by: Dirk Einecke
9 Replies

6. Shell Programming and Scripting

find duplicate string in many different files

I have more than 100 files like this: SVEAVLTGPYGYT 2 SVEGNFEETQY 10 SVELGQGYEQY 28 SVERTGTGYT 6 SVGLADYNEQF 21 SVGQGYEQY 32 SVKTVLGYEQF 2 SVNNEQF 12 SVRDGLTNSPLH 3 SVRRDREGLEQF 11 SVRTSGSYEQY 17 SVSVSGSPLQETQY 78 SVVHSTSPEAF 59 SVVPGNGYT 75 (4 Replies)
Discussion started by: xshang
4 Replies

7. Shell Programming and Scripting

Find duplicate files but with different extensions

Hi ! I wonder if anyone can help on this : I have a directory: /xyz that has the following files: chsLog.107.20130603.gz chsLog.115.20130603 chsLog.111.20130603.gz chsLog.107.20130603 chsLog.115.20130603.gz As you ca see there are two files that are the same but only with a minor... (10 Replies)
Discussion started by: fretagi
10 Replies

8. Shell Programming and Scripting

Find duplicate rows between files

Hi champs, I have one of the requirement, where I need to compare two files line by line and ignore duplicates. Note, I hav files in sorted order. I have tried using the comm command, but its not working for my scenario. Input file1 srv1..development..employee..empname,empid,empdesg... (1 Reply)
Discussion started by: Selva_2507
1 Replies

9. Shell Programming and Scripting

Find help in shell - that clears away duplicate files

I am so frustrated!!! I want a nice command that clears away duplicate files: find . -type f -regex '.*{1,3}\..*' | xargs -I## rm -v '##' should work in my opinion. But it finds nothing even though I have files that have the file name: Scooby-Doo-1.txt Himalaya-2.jpg Camping... (8 Replies)
Discussion started by: Mr.Glaurung
8 Replies

10. Shell Programming and Scripting

To Find Duplicate files using latest in Linux

I have tried the following code and with that i couldnt achieve what i want. #!/usr/bin/bash find ./ -type f \( -iname "*.xml" \) | sort -n > fileList sed -i '/\.\/fileList/d' fileList NAMEOFTHISFILE=$(echo $0|sed -e 's/\/()$*.^|/\\&/g') sed -i "/$NAMEOFTHISFILE/d"... (2 Replies)
Discussion started by: gold2k8
2 Replies
RDIFF-BACKUP-FS(1)					      General Commands Manual						RDIFF-BACKUP-FS(1)

NAME
rdiff-backup-fs - Filesystem for accessing rdiff-backup archives. SYNOPSIS
rdiff-backup-fs <mount_point> <repository> [repositories ...] [-option ...] DESCRIPTION
rdiff-backup-fs is a filesystem in userspace that reads rdiff-backup archives and provides convenient access. OPTIONS
--debug <0-4> Run rdiff-backup-fs in foreground with given verbosity of debug messages. -f, --full Store information about all revisions in memory. CAUTION: this may take a lot of memory if your archive contains many revisions. -l, --last Displays files from the most recent increment as directories, each holding every version of the file. CAUTION: this stores informa- tion about all revisions in memory and therefore may take a lot of memory if archive contains many revisions. -c <n>, --caching <n> How many files retrieved from the rdiff-backup archive may be cached by filesystem. By default rdiff-backup-fs will cache up to 10 files. If this switch is set to 0, no caching will be done. -r <n>, --revisions <n> How many revisions should be stored in memory for on demand revision retrieval. By default rdiff-backup-fs will store up to 10 revi- sions in memory. -d, --directory <path> Set directory for directory with temporary files. By default rdiff-backup-fs uses /tmp. -v, --version Print version of rdiff-backup-fs and exit. SEE ALSO
rdiff-backup(1) COPYRIGHT
rdiff-backup-fs is Copyright (c) 2007-2011 Filip Gruszczyski. rdiff-backup-fs is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MER- CHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with this program. If not, see <http://www.gnu.org/licenses/>. AUTHORS
Filip Gruszczyski <gruszczy@gmail.com> RDIFF-BACKUP-FS(1)

Featured Tech Videos

All times are GMT -4. The time now is 10:43 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy