12-14-2008
How to : Find duplicate number from file? with bash
Thanks
AVKlinux
10 More Discussions You Might Find Interesting
1. UNIX for Dummies Questions & Answers
Hello. First time poster here. I have a huge file of IP numbers. I am trying to output only the class b of the IPs and rank them by most common and output the total # of duplicate class b's before the class b. An example is below:
12.107.1.1
12.107.9.54
12.108.3.89
12.109.109.4
12.109.6.3
... (2 Replies)
Discussion started by: crunchtime
2 Replies
2. Shell Programming and Scripting
I am trying to write a housekeeping bash script. Part of it involves searching all of my attached storage media for photographs and moving them into a single directory. The problem occurs when files have duplicate names, obviously a file called 001.jpg will get overwritten with another file... (6 Replies)
Discussion started by: stumpyuk
6 Replies
3. Shell Programming and Scripting
Dear All,
I have one file which looks like :
account1:passwd1
account2:passwd2
account3:passwd3
account1:passwd4
account5:passwd5
account6:passwd6
you can see there're two records for account1. and is there any shell command which can find out : account1 is the duplicate record in... (3 Replies)
Discussion started by: tiger2000
3 Replies
4. Shell Programming and Scripting
Hi,
Need to find a duplicate records on the first column,
ANU4501710430989 0000000W20389390
ANU4501710430989 0000000W67065483
ANU4501130050520 0000000W80838713
ANU4501210170685 0000000W69246611... (3 Replies)
Discussion started by: Murugesh
3 Replies
5. Shell Programming and Scripting
Hi, here comes another newbie question:
How to find the number of non-duplicate names recursively?
For example, my files are stored in the folders like:
If I do
find . -depth -name "*.txt" | wc -l
This will gives out a result "4". One .txt file named "1.txt" in folder "1",
and... (2 Replies)
Discussion started by: jiapei100
2 Replies
6. UNIX for Dummies Questions & Answers
Hi Unix gurus,
Maybe it is too much to ask for but please take a moment and help me out. A very humble request to you gurus. I'm new to Unix and I have started learning Unix. I have this project which is way to advanced for me.
File format: CSV file
File has four columns with no header... (8 Replies)
Discussion started by: arvindosu
8 Replies
7. Shell Programming and Scripting
Hi,
In a file, I have to mark duplicate records as 'D' and the latest record alone as 'C'.
In the below file, I have to identify if duplicate records are there or not based on Man_ID, Man_DT, Ship_ID and I have to mark the record with latest Ship_DT as "C" and other as "D" (I have to create... (7 Replies)
Discussion started by: machomaddy
7 Replies
8. Shell Programming and Scripting
hello all
in my bash script I have a file and I only want to keep the lines that appear twice in the file.Is there a way to do this?
thanks in advance! (4 Replies)
Discussion started by: vlm
4 Replies
9. Shell Programming and Scripting
I am writing a bash script to find out all the files in a directory which are empty. I am running into multiple issues. I will really appreciate if someone can please help me.
#!/bin/bash
DATE=$(date +%m%d%y)
TIME=$(date +%H%M)
DIR="/home/statsetl/input/civil/test"
... (1 Reply)
Discussion started by: monasharma13
1 Replies
10. Shell Programming and Scripting
Ultimately, I'm looking to create a script that allows me to plug in a usb drive with lots of jpegs on it & copy them over to a folder on my hard drive. So in the process of copying I am looking to hash check them, record dupes to a file, copy only 1 of the identical files (if it doesn't exsist... (1 Reply)
Discussion started by: JonaQuinn
1 Replies
LEARN ABOUT DEBIAN
sysprofile
SYSPROFILE(8) System Manager's Manual SYSPROFILE(8)
NAME
sysprofile - modular centralized shell configuration
DESCRIPTION
sysprofile is a generic approach to configure shell settings in a modular and centralized way mostly aimed at avoiding work for lazy sysad-
mins. It has only been tested to work with the bash shell.
It basically consists of the small /etc/sysprofile shell script which invokes other small shell scripts having a .bash suffix which are
contained in the /etc/sysprofile.d/ directory. The system administrator can drop in any script he wants without any naming convention
other than that the scripts need to have a .bash suffix to enable automagic sourcing by /etc/sysprofile.
This mechanism is set up by inserting a small shell routine into /etc/profile for login shells and optionally into /etc/bashrc and/or
/etc/bash.bashrc for non-login shells from where the actual /etc/sysprofile script is invoked:
if [ -f /etc/sysprofile ]; then
. /etc/sysprofile
fi
For using "sysprofile" under X11, one can source it in a similar way from /etc/X11/Xsession or your X display manager's Xsession file to
provide the same shell environment as under the console in X11. See the example files in /usr/share/doc/sysprofile/ for illustration.
For usage of terminal emulators with a non-login bash shell under X11, take care to enable sysprofile via /etc/bash.bashrc. If not set
this way, your terminal emulators won't come up with the environment defined by the scripts in /etc/sysprofile.d/.
Users not wanting /etc/sysprofile to be sourced for their environment can easily disable it's automatic mechanism. It can be disabled by
simply creating an empty file called $HOME/.nosysprofile in the user's home directory using e.g. the touch(1) command.
Any single configuration file in /etc/sysprofile.d/ can be overridden by any user by creating a private $HOME/.sysprofile.d/ directory
which may contain a user's own version of any configuration file to be sourced instead of the system default. It's names have just to
match exactly the system's default /etc/sysprofile.d/ configuration files. Empty versions of these files contained in the $HOME/.syspro-
file.d/ directory automatically disable sourcing of the system wide version.
Naturally, users can add and include their own private script inventions to be automagically executed by /etc/sysprofile at login time.
OPTIONS
There are no options other than those dictated by shell conventions. Anything is defined within the configuration scripts themselves.
SEE ALSO
The README files and configuration examples contained in /etc/sysprofile.d/ and the manual pages bash(1), xdm(1x), xdm.options(5), and
wdm(1x). Recommended further reading is everything related with shell programming.
If you need a similar mechanism for executing code at logout time check out the related package syslogout(8) which is a very close compan-
ion to sysprofile.
BUGS
sysprofile in its current form is mainly restricted to bash(1) syntax. In fact it is actually a rather embarrassing quick and dirty hack
than anything else - but it works. It serves the practical need to enable a centralized bash configuration until something better
becomes available. Your constructive criticism in making this into something better" is very welcome. Before i forget to mention it: we
take patches... ;-)
AUTHOR
sysprofile was developed by Paul Seelig <pseelig@debian.org> specifically for the Debian GNU/Linux system. Feel free to port it to and use
it anywhere else under the conditions of either the GNU public license or the BSD license or both. Better yet, please help to make it into
something more worthwhile than it currently is.
SYSPROFILE(8)