I need help with a script which accepts one argument and goes through all the files under a directory and prints a list of possible duplicate files As its output, it prints zero or more lines, each one containing a space-separated list of filenames. All the files listed on one line have the same... (1 Reply)
Hi,
is it possible to remove all duplicate lines from all txt files in a specific folder?
This is too hard for me maybe someone could help.
lets say we have an amount of textfiles 1 or 2 or 3 or... maximum 50
each textfile has lines with text.
I want all lines of all textfiles... (8 Replies)
Hi
I have been struggling with a script for removing duplicate messages from a shared mailbox.
I would like to search for duplicate messages based on the “Message-ID” string within the messages files.
I have managed to find the duplicate “Message-ID” strings and (if I would like) delete... (1 Reply)
Hi all.
Am doing continuous backup of mailboxes using rsync.
So whenever a new mail arrives it is automatically copied on backup server.
When a new mail arrives it is named as xyz:2, when it is read by the email client an S is appended xyz:2,S
Eventually , 2 copies of the same file exist on... (7 Replies)
Hello,
I wrote a basic script that works however I am was wondering if it could be sped up. I am comparing files over ssh to remove the file from the source server directory if a match occurs. Please Advise me on my mistakes.
#!/bin/bash
for file in `ls /export/home/podcast2/"$1" ` ; do
... (5 Replies)
Dear All,
I have multiple files having number of records, consist of more than 10 columns some column values are duplicate and i want to remove these duplicate values from these files.
Duplicate values may come in different files.... all files laying in single directory..
Need help to... (3 Replies)
Hello again, I am wanting to remove all duplicate blocks of XML code in a file. This is an example:
input:
<string-array name="threeItems">
<item>item1</item>
<item>item2</item>
<item>item3</item>
</string-array>
<string-array name="twoItems">
<item>item1</item>
<item>item2</item>... (19 Replies)
Hi,
In a directory, e.g. ~/corpus is a lot of files and subdirectories. Some of the files are named:
12345___PP___0902___AA.txt
12346___PP___0902___AA. txt
12347___PP___0902___AA. txt
The amount of files varies. I need to keep the highest (12347___PP___0902___AA. txt) and remove... (5 Replies)
So, I have text files,
one "fail.txt"
And one
"color.txt"
I now want to use a command line (DOS) to remove ANY line that is PRESENT IN BOTH from each text file.
Afterwards there shall be no duplicate lines. (1 Reply)
TARGET_DIR='/media/andy/MAXTOR_SDB1/Ubuntu_Mate_18.04/'
REGEX='{4}-{2}-{2}_{2}:{2}' # regular expression that match to: date '+%Y-%m-%d_%H:%M'
LATEST_FILE="$(ls "$TARGET_DIR" | egrep "^${REGEX}$" | tail -1)"
find "$TARGET_DIR" ! -name "$LATEST_FILE" -type f -regextype egrep -regex... (7 Replies)
Discussion started by: drew77
7 Replies
LEARN ABOUT HPUX
sum
sum(1) General Commands Manual sum(1)NAME
sum - print checksum and block or byte count of file(s)
SYNOPSIS
[file ...]
Remarks
is obsolescent and should not be used in new applications that are intended to be portable between systems. Use instead (see cksum(1)).
DESCRIPTION
calculates and prints to standard output a checksum for each named file, and also prints the size of the file in 512 byte blocks, rounded
up.
The default algorithm is a 16-bit sum of the bytes in which overflow is ignored. Alternate algorithms can be selected with the and
options.
Standard input is used if no file names are given.
is typically used to verify data integrity when copying files between systems.
Options
recognizes the following options:
Use an alternate algorithm in which the 16-bit sum is right rotated
with each byte in computing the checksum.
Use the 32-bit cyclical redundancy check (CRC) algorithm used by
RETURN VALUE
returns the following values upon completion:
All files were processed successfully.
One or more files could not be read or some other error occurred.
If an inaccessible file is encountered, continues processing any remaining files, but the final exit status is affected.
DIAGNOSTICS
Read error conditions are indistinguishable from end of file on most devices; check the block or byte count.
WARNINGS
This command is likely to be withdrawn from X/Open standards. Applications using this command might not be portable to other vendors'
platforms. The usage of cksum(1) is recommended.
SEE ALSO cksum(1), wc(1).
STANDARDS CONFORMANCE sum(1)