Remove Duplicate Filenames in 2 very large directories
Hello Gurus,
O/S RHEL4
I have a requirement to compare two linux based directories for duplicate filenames and remove them. These directories are close to 2 TB each. I have tried running a:
I have tried this as well:
I wanted to get the output of the above command and place in a variable for a deletion. This scenario does not work and the machines load goes to high for production. I have also thought of trying a rsync with the delete flag, and I am unsure if this will compare both directories successfully.
Can someone please point me in the right direction as to what commands or scenarios will best accomplish my task.
I have also tried to google this on unix.com as well as the web.
Your support and assistance is greatly appreciated.
Jaysunn
Last edited by jaysunn; 09-24-2009 at 11:56 AM..
Reason: Added O/S
Hi!
I have thousands of sub-directories, and hundreds of thousands of files in them. What is the fast way to find out which files are older than a certain date? Is the "find" command the fastest? Or is there some other way?
Right now I have a C script that traverses through and checks... (5 Replies)
I'm trying to put together a shell script that will append specific prefixes based on the content of filenames. I think I have this part down. However, I want to append before that part a process that will remove the current prefix before it renames the files with the new prefix.
For example,... (6 Replies)
I have a problem where tar is somehow creating duplicate filenames when tarring a directory. Doing an ls on the directory does not show any duplicate filenames, yet when the directory is tarred, you can see that there are duplicates:
bash-2.03# pwd
/var/log/cricket
bash-2.03# ls -1 | sort |... (2 Replies)
Hi,
I have file which users like
filename ->"readfile", following entries
peter
john
alaska
abcd
xyz
and i have directory /var/
i want to do first cat of "readfile" line by line and first read peter in variable and also cross check with /var/ how many directories are avaialble... (8 Replies)
I have a large list of filenames from an Excel sheet, which I then translate into a simple text file. I'd like to use this list, which contains various file extensions , to archive these files and then remove them recursively through multiple directories and subdirectories. So far, it looks like... (5 Replies)
below is the script to rename filenames ending with .pdf extension.
I want the script to enter directories and search for all pdf and then if it is in the format file_amb_2008.pdf , then change it to 2008_amb_file.pdf, and this script should work only for .pdf files.
help required to make the... (12 Replies)
Hi Gurus,
Do any kind souls encounter have the same script as mentioned here.
Find and compare filenames in different mount point and remove duplicates.
Thanks a million!!!
wanna13e (7 Replies)
I have noticed that the same folder (and contents) lives in
/u/public and /usr/public
Question was this put here intentionally or by accident?
Its 31Gb in size and on a 72Gb HDD that leaves little room for apps.
It is a nework shared drive for all to access e.g. p: points to... (0 Replies)
Hi,
I have files like below, In files coming as spaces. Before transfering those files into ftp server. I want to remove the spaces and then can transfer the files into unix server.
e.g: filenames are
1) SHmail _profile001_20120908.txt
2) SHmail_profile001 _20120908.txt
3) sh... (3 Replies)
Is there a way via some bash script or just cmd to find duplicate directories?
i have main folders:
TEST1
TEST2
In folder TEST1 is some amount of same folders as in folder TEST2
can be this done? i tried fdupe but it only search for dupe files not whle dirs
thx! (8 Replies)
Discussion started by: ZerO13
8 Replies
LEARN ABOUT CENTOS
openipmicmd
openipmicmd(1) An IPMI Command Interface openipmicmd(1)NAME
openipmicmd - An IPMI Command Interface
SYNOPSIS
openipmicmd [-k entry-to-execute] <connection parms>
DESCRIPTION
The openipmicmd program allows a user to execute direct IPMI commands. It can work with direct interface with the OpenIPMI driver or with
IPMI LAN interfaces.
OPTIONS -k entry-to-execute
Execute a single command an exit.
<connection parms>
The parameters for the connection depend on the connection type. These are all described in openipmi_conparms (7)
COMMANDS
Once up, you can execute commands in the user interface. Note that commands and responses are asynchronous, you issue a command and the
interface returns immediately. When the response comes back, it will be dumped on your console. That's a little strange looking, but IPMI
is asynchronous underneath. Note that the -k option is synchronous, it will wait for the response or a timeout before returning.
0f lun netfn cmd [data1 [data2 ...]]
Send a command to the BMC you are connected to.
channel [ipmb] IPMB-addr lun netfn [seq] cmd [data1 [data2 ...]]
Send a command to a device on the IPMB bus. The "ipmb" string is optional. The seq must be provided if the netfn is a response (an
odd number). It must be the same sequence number that came in on the command in the "Command Sequence = seq" part of the command.
channel 00 IPMB-addr lun netfn cmd [data1 [data2 ...]]
Send a broadcast command to a device on the IPMB bus.
channel lan handle remote-swid local-swid lun netfn cmd [data1 [data ...]]
Send a command to a device over a LAN channel. Note that this not the same as a LAN connection. This sends a message through a
local BMC to a remote system that is hooked up with a LAN connection.
test_lat count command
Executes the given commands (one of the previous commands) count times and gives the average time per command to execute. Note that
"count" is hexadecimal.
help Display some help.
regcmd netfn cmd
Register to receive the given command. If the driver receives an external command, it will print it out. This only works with sys-
tem interface connections, it will not work on LAN connections.
unregcmd netfn cmd
Remove a command registration.
SEE ALSO ipmi_ui(1), ipmilan(8), openipmi_conparms(7)KNOWN PROBLEMS
The asynchronous nature of the program can be annoying.
AUTHOR
Corey Minyard <cminyard@mvista.org>
OpenIPMI 05/13/03 openipmicmd(1)