09-20-2011
wicked solution vgersh99!
--ahamed
This User Gave Thanks to ahamed101 For This Post:
10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
Hi,
is it possible to remove all duplicate lines from all txt files in a specific folder?
This is too hard for me maybe someone could help.
lets say we have an amount of textfiles 1 or 2 or 3 or... maximum 50
each textfile has lines with text.
I want all lines of all textfiles... (8 Replies)
Discussion started by: lowmaster
8 Replies
2. UNIX for Dummies Questions & Answers
I have a data set that has 4 columns, I want to know if I can delete duplicate lines while ignoring one of the columns, for example
10 chr1 ASF 30
15 chr1 ASF 20
5 chr1 ASF 30
6 chr2 EBC 15
4 chr2 EBC 30
...
I want to know if I can delete duplicate lines while ignoring column 1, so the... (5 Replies)
Discussion started by: japaneseguitars
5 Replies
3. Shell Programming and Scripting
Input:
hello hello
hello hello
monkey
donkey
hello hello
drink
dance
drink
Output should be:
hello hello
monkey
donkey
drink
dance (9 Replies)
Discussion started by: cola
9 Replies
4. Shell Programming and Scripting
Dear All,
I have multiple files having number of records, consist of more than 10 columns some column values are duplicate and i want to remove these duplicate values from these files.
Duplicate values may come in different files.... all files laying in single directory..
Need help to... (3 Replies)
Discussion started by: arvindng
3 Replies
5. Shell Programming and Scripting
Hi,
I came to know that using awk '!x++' removes the duplicate lines. Can anyone please explain the above syntax. I want to understand how the above awk syntax removes the duplicates.
Thanks in advance,
sudvishw :confused: (7 Replies)
Discussion started by: sudvishw
7 Replies
6. Shell Programming and Scripting
Hi,
I am on a Solaris8 machine
If someone can help me with adjusting this awk 1 liner (turning it into a real awkscript) to get by this "event not found error"
...or
Present Perl solution code that works for Perl5.8 in the csh shell ...that would be great.
******************
... (3 Replies)
Discussion started by: andy b
3 Replies
7. Windows & DOS: Issues & Discussions
So, I have text files,
one "fail.txt"
And one
"color.txt"
I now want to use a command line (DOS) to remove ANY line that is PRESENT IN BOTH from each text file.
Afterwards there shall be no duplicate lines. (1 Reply)
Discussion started by: pasc
1 Replies
8. Shell Programming and Scripting
Hi All
I have a list of files which will have duplicate list of blocks of text. Following is a sample of the file, I have removed the sensitive information from the file.
All the code samples starts from <TR BGCOLOR="white"> and Ends with IP address and two html tags like this.
10.14.22.22... (3 Replies)
Discussion started by: mahasona
3 Replies
9. Shell Programming and Scripting
I have two large files (~250GB) that I am trying to remove the where GT: 0/0 or 1/1 or 2/2 for both files. I was going to use a bash with the below awk, which I think will find each line but how do I remove that line is that condition is found? Thank you :).
Input
20 60055 . A ... (4 Replies)
Discussion started by: cmccabe
4 Replies
10. Shell Programming and Scripting
I create a CGI in bash/html.
My awk script looks like :
echo "<table>"
for fn in /var/www/cgi-bin/LPAR_MAP/*;
do
echo "<td>"
echo "<PRE>"
awk -F',|;' -v test="$test" '
NR==1 {
split(FILENAME ,a,"");
}
$0 ~ test {
if(!header++){
... (12 Replies)
Discussion started by: Tim2424
12 Replies
LEARN ABOUT DEBIAN
dptest
DPTEST(1) Quick Database Manager DPTEST(1)
NAME
dptest - test cases for QDBM Depot
SYNOPSIS
dptest write [-s] name rnum bnum
dptest read [-wb] name
dptest rcat [-c] name rnum bnum pnum align
dptest combo name
dptest wicked [-c] name rnum
DESCRIPTION
The command `dptest' is a utility for facility test and performance test. Check a database generated by the command or measure the execu-
tion time of the command. This command is used in the following format. `name' specifies a database name. `rnum' specifies the number of
the records. `bnum' specifies the number of the elements of the bucket array. `pnum' specifies the number of patterns of the keys.
`align' specifies the basic size of alignment. `fbpsiz' specifies the size of the free block pool.
dptest write [-s] name rnum bnum
Store records with keys of 8 bytes. They change as `00000001', `00000002'...
dptest read [-wb] name
Retrieve all records of the database above.
dptest rcat [-c] name rnum bnum pnum align fbpsiz
Store records with partway duplicated keys using concatenate mode.
dptest combo name
Perform combination test of various operations.
dptest wicked [-c] name rnum
Perform updating operations selected at random.
Options feature the following.
-s : make the file sparse.
-wb : use the function `dpgetwb' instead of the function `dpget'.
-c : perform comparison test with map of Cabin.
This command returns 0 on success, another on failure. The environment variable `QDBMDBGFD' specifies the file descriptor to output the
history of updating the variable `dpecode'.
SEE ALSO
qdbm(3), depot(3)
AUTHOR
QDBM was written by Mikio Hirabayashi <mikio@fallabs.com>.
Man Page 2005-06-01 DPTEST(1)