I have searched the FAQ - by using sort, duplicates, etc.... but I didn't get any articles or results on it.
Currently, I am using:
sort -u file1 > file2 to remove duplicates. For a file size of 1giga byte approx. time taken to remove duplicates is 1hr 21 mins.
Is there any other faster way... (15 Replies)
Hi!
I have thousands of sub-directories, and hundreds of thousands of files in them. What is the fast way to find out which files are older than a certain date? Is the "find" command the fastest? Or is there some other way?
Right now I have a C script that traverses through and checks... (5 Replies)
hello
i need help to remove directory . The directory is not empty ., it contains
several sub directories and files inside that..
total number of files in one directory is 12,24,446 .
rm -rf doesnt work . it is prompting for every file ..
i want to delete without prompting and... (6 Replies)
1)I am trying to write a script that works interactively lists duplicated records on certain field/column and asks user to delete one or more. And finally it deletes all the records the used has asked for.
I have an idea to store those line numbers in an array, not sure how to do this in... (3 Replies)
I have a log file and I am trying to run a script against it to search for key issues such as invalid users, errors etc. In one part, I grep for session closed and get a lot of the same thing,, ie. root username etc. I want to remove the multiple root and just have it do a count, like wc -l
... (5 Replies)
I have a 5 GB text file(log/debug)
I want to delete all lines containing 'TRACE'
Command used
sed -i '/TRACE/d' mylog.txt
Is there any other fastest way to do this? (1 Reply)
Hello,
i have the following problem:
there are two folders with a lot of files.
Example:
FolderA contains AAA, BBB, CCC
FolderB contains DDD, EEE, AAA
How can i via script identify AAA as duplicate in Folder B and delete it there? So that only DDD and EEE remain, in Folder B?
Thank you... (16 Replies)
I do have a big CA bundle certificate file and each time if i get request to add new certificate to the existing bundle i need to make sure it is not present already. How i can validate the duplicates.
The alignment of the certificate within the bundle seems to be different.
Example:
Cert 1... (7 Replies)
Hi,
i have another problem. I have been trying to solve it by myself but failed.
inputfile
;;
ID T08578
NAME T08578
SBASE 30696
EBASE 32083
TYPE P
func just test
func chronology
func cholesterol
func null
INT 30765-37333
INT 37154-37318
Link 5546
Link 8142 (4 Replies)
I am using the below script to delete duplicate files but it is not working for directories with more than 10k files "Argument is too long" is getting for ls -t. Tried to replace ls -t with
find . -type f \( -iname "*.xml" \) -printf '%T@ %p\n' | sort -rg | sed -r 's/* //' | awk... (8 Replies)
Discussion started by: gold2k8
8 Replies
LEARN ABOUT DEBIAN
puzzle-diff
PUZZLE-DIFF(1)PUZZLE-DIFF(1)NAME
puzzle-diff - Compare pictures with libpuzzle
SYNOPSIS
[-b <contrast barrier for cropping>] [-c] [-C <max cropping ratio>] [-e] [-E <similarity threshold>] [-h] [-H <max height>] [-l <lambdas>]
[-n <noise cutoff>] [-p <p ratio>] [-t] [-W <max width>] <file 1> <file 2>
DESCRIPTION
puzzle-diff compares two pictures and outputs the normalized distance.
Try puzzle-diff -h for more info.
EXAMPLES
Output distance between two images:
$ puzzle-diff pic-a-0.jpg pics-a-1.jpg
0.102286
Compare two images, exit with 10 if they look the same, exit with 20 if they don't (may be useful for scripts):
$ puzzle-diff -e pic-a-0.jpg pics-a-1.jpg
$ echo $?
10
Compute distance, without cropping and with computing the average intensity of the whole blocks:
$ puzzle-diff -p 1.0 -c pic-a-0.jpg pic-a-1.jpg
0.0523151
AUTHORS
Frank DENIS libpuzzle at pureftpd dot org
SEE ALSO libpuzzle(3), puzzle_set(3)
2012-05-09 PUZZLE-DIFF(1)