|
|
Sponsored Content
Top Forums
Shell Programming and Scripting
How to find duplicate line in Linux?
Post 302838875 by rdrtx1 on Tuesday 30th of July 2013 04:58:08 PM
|
|
10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
hello all
I like to make search on files , and the result need to be the files that are duplicated? (8 Replies)
Discussion started by: umen
8 Replies
2. UNIX for Dummies Questions & Answers
Hi,
I have 3 lines in a text file that is similar to this (as a result of a diff between 2 files):
35,36d34
< DATA.EVENT.EVENT_ID.s = "3661208"
< DATA.EVENT.EVENT_ID.s = "3661208"
I am trying to get it down to just this:
DATA.EVENT.EVENT_ID.s = "3661208"
How can I do this?... (11 Replies)
Discussion started by: ocelot
11 Replies
3. Shell Programming and Scripting
I need a perl script, which will run every midnight via cronjob and e-mail few users once it finds any duplicated value in a file which is located /etc/hosts, the file name is called hosts and the format of the file has 3 colums and some time 2 columns. The script will look for duplicate IP or... (3 Replies)
Discussion started by: ricky007
3 Replies
4. Shell Programming and Scripting
Hi guys, I'm really happy to find this forum
I have a log file, and I have to find all lines that have "error" word, and then save this output in file, the output file has to have just only one line to any Duplicated lines and counter that show how many time this lines duplicated?
I already... (2 Replies)
Discussion started by: wax_light
2 Replies
5. Shell Programming and Scripting
Hi,
In a file, I have to mark duplicate records as 'D' and the latest record alone as 'C'.
In the below file, I have to identify if duplicate records are there or not based on Man_ID, Man_DT, Ship_ID and I have to mark the record with latest Ship_DT as "C" and other as "D" (I have to create... (7 Replies)
Discussion started by: machomaddy
7 Replies
6. Shell Programming and Scripting
Hi All ! I am trying to copy all files with extension .sh to one folder, following command I am using
find . -name \*.sh -print0 | xargs -I{} -0 cp -v {} Scripts/
above command working fine but I have some .sh file with same base name different directory, so I would copy all .sh file including... (5 Replies)
Discussion started by: Akshay Hegde
5 Replies
7. Shell Programming and Scripting
I have a script that builds a database ~30 million lines, ~3.7 GB .cvs file. After multiple optimzations It takes about 62 min to bring in and parse all the files and used to take 10 min to remove duplicates until I was requested to add another column. I am using the highly optimized awk code:
awk... (34 Replies)
Discussion started by: Michael Stora
34 Replies
8. Shell Programming and Scripting
Hi,
I am looking to extract linux version from /etc/*-release file.
I am specifically tring to avoid use of awk command here. would be great if can do done via sed or grep command.
Red Hat Enterprise Linux Server release 6.5 (Tikanga)
output must be 6
regards,
Litu (7 Replies)
Discussion started by: Litu1988
7 Replies
9. Shell Programming and Scripting
Dear folks
I have a map file of around 54K lines and some of the values in the second column have the same value and I want to find them and delete all of the same values. I looked over duplicate commands but my case is not to keep one of the duplicate values. I want to remove all of the same... (4 Replies)
Discussion started by: sajmar
4 Replies
10. Shell Programming and Scripting
I have tried the following code and with that i couldnt achieve what i want.
#!/usr/bin/bash
find ./ -type f \( -iname "*.xml" \) | sort -n > fileList
sed -i '/\.\/fileList/d' fileList
NAMEOFTHISFILE=$(echo $0|sed -e 's/\/()$*.^|/\\&/g')
sed -i "/$NAMEOFTHISFILE/d"... (2 Replies)
Discussion started by: gold2k8
2 Replies
LEARN ABOUT DEBIAN
fslint
fslint-gui(1) User Commands fslint-gui(1) NAME
fslint-gui - A GUI wrapper for the individual fslint command line tools SYNOPSIS
fslint-gui [OPTIONS] [PATHS] DESCRIPTION
fslint is a toolset to find various problems with filesystems, including duplicate files and problematic filenames etc. Individual command line tools are available in addition to the GUI and to access them, one can change to, or add to $PATH the /usr/share/fslint/fslint directory on a standard install. Each of these commands in that directory have a --help option which further details its parameters. findup - find DUPlicate files findnl - find Name Lint (problems with filenames) findu8 - find filenames with invalid utf8 encoding findbl - find Bad Links (various problems with symlinks) findsn - find Same Name (problems with clashing names) finded - find Empty Directories findid - find files with dead user IDs findns - find Non Stripped executables findrs - find Redundant Whitespace in files findtf - find Temporary Files findul - find possibly Unused Libraries zipdir - Reclaim wasted space in ext2 directory entries PARAMETERS
If [PATHS] are specified, they become the default search path, otherwise the current directory becomes the default. --version Display the fslint version and exit --help Display help for tool specific options AUTHOR
Written by Padraig Brady REPORTING BUGS
Report bugs to <P@draigBrady.com>. COPYRIGHT
Copyright (C) 2000-2009 Padraig Brady <P@draigBrady.com>. This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or any later version. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MER- CHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details, which is available at www.gnu.org fslint July 2009 fslint-gui(1)