02-16-2012
Thank you
both them worked great and i actually learned something
9 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
Hi
Just wondering whether or not I can remove duplicated lines without sort
For example, I use the command who, which shows users who are logging on. In some cases, it shows duplicated lines of users who are logging on more than one terminal.
Normally, I would do
who | cut -d" " -f1 |... (6 Replies)
Discussion started by: lalelle
6 Replies
2. Shell Programming and Scripting
Hi Guys...
Please Could you help me with the following ?
aaaa bbbb cccc sdsd
aaaa bbbb cccc qwer
as you can see, the 2 lines are matched in three fields...
how can I delete this pupicate ? I mean to delete the second one if 3 fields were duplicated ?
Thanks (14 Replies)
Discussion started by: yahyaaa
14 Replies
3. UNIX for Dummies Questions & Answers
Hello all,
I've got a strange behaviour of sort and uniq commands: they do not recognise apparently duplicated lines in a file (already sorted). The lines are identical by eye, but they must differ in smth, because when they are put in two files, those have slightly different size.
What can make... (8 Replies)
Discussion started by: roussine
8 Replies
4. Shell Programming and Scripting
We have an input file as follows:
2010-09-15-12.41.15
2010-09-15-12.41.15
2010-09-15-12.41.24
2010-09-15-12.41.24
2010-09-15-12.41.24
2010-09-15-12.41.24
2010-09-15-12.41.25
2010-09-15-12.41.26
2010-09-15-12.41.26
2010-09-15-12.41.26
2010-09-15-12.41.26
2010-09-15-12.41.26... (3 Replies)
Discussion started by: ux4me
3 Replies
5. Shell Programming and Scripting
Dear All,
Suppose I have a file:
1 1 1 1
2 2 2 2
3 3 3 3I want to insert new line under each old line so that the file would become:
1 1 1 1
1 1 1 1
2 2 2 2
2 2 2 2
3 3 3 3
3 3 3 3How can this be accomplished using awk (or sed)? (5 Replies)
Discussion started by: littlewenwen
5 Replies
6. Shell Programming and Scripting
Hi, if i have a file like this:
Query=1
a
a
b
c
c
c
d
Query=2
b
b
b
c
c
e
.
.
. (7 Replies)
Discussion started by: the_simpsons
7 Replies
7. Shell Programming and Scripting
Hello,
I am trying to eliminate rows where the first field is duplicated, leaving the row where the last field is "NET".
Data file:
345234|22.34|LST
546543|55.33|LST
793929|98.23|LST
793929|64.69|NET
149593|49.22|LST
Desired output:
345234|22.34|LST
546543|55.33|LST... (2 Replies)
Discussion started by: palex
2 Replies
8. UNIX for Dummies Questions & Answers
Hi Everybody! First post! Totally noobie.
I'm using the terminal to read a poorly formatted book.
The text file contains, in the middle of paragraphs, hyphenation to split words that are supposed to be on multiple pages. It looks ve -- ry much like this.
I was hoping to use grep -v " -- "... (5 Replies)
Discussion started by: AxeHandle
5 Replies
9. Shell Programming and Scripting
Dear all,
I always appreciate your help.
I would like to delete lines containing duplicated strings in the second column.
test.txt
658 invert_d2e_q_reg_0_/Qalu_ecl_zlow_e 0.825692
659 invert_d2e_q_reg_0_/Qalu_byp_rd_data_e 0.825692
660 invert_d2e_q_reg_0_/Qalu_byp_rd_data_e 0.825692... (1 Reply)
Discussion started by: jypark22
1 Replies
LEARN ABOUT DEBIAN
bup-margin
bup-margin(1) General Commands Manual bup-margin(1)
NAME
bup-margin - figure out your deduplication safety margin
SYNOPSIS
bup margin [options...]
DESCRIPTION
bup margin iterates through all objects in your bup repository, calculating the largest number of prefix bits shared between any two
entries. This number, n, identifies the longest subset of SHA-1 you could use and still encounter a collision between your object ids.
For example, one system that was tested had a collection of 11 million objects (70 GB), and bup margin returned 45. That means a 46-bit
hash would be sufficient to avoid all collisions among that set of objects; each object in that repository could be uniquely identified by
its first 46 bits.
The number of bits needed seems to increase by about 1 or 2 for every doubling of the number of objects. Since SHA-1 hashes have 160 bits,
that leaves 115 bits of margin. Of course, because SHA-1 hashes are essentially random, it's theoretically possible to use many more bits
with far fewer objects.
If you're paranoid about the possibility of SHA-1 collisions, you can monitor your repository by running bup margin occasionally to see if
you're getting dangerously close to 160 bits.
OPTIONS
--predict
Guess the offset into each index file where a particular object will appear, and report the maximum deviation of the correct answer
from the guess. This is potentially useful for tuning an interpolation search algorithm.
--ignore-midx
don't use .midx files, use only .idx files. This is only really useful when used with --predict.
EXAMPLE
$ bup margin
Reading indexes: 100.00% (1612581/1612581), done.
40
40 matching prefix bits
1.94 bits per doubling
120 bits (61.86 doublings) remaining
4.19338e+18 times larger is possible
Everyone on earth could have 625878182 data sets
like yours, all in one repository, and we would
expect 1 object collision.
$ bup margin --predict
PackIdxList: using 1 index.
Reading indexes: 100.00% (1612581/1612581), done.
915 of 1612581 (0.057%)
SEE ALSO
bup-midx(1), bup-save(1)
BUP
Part of the bup(1) suite.
AUTHORS
Avery Pennarun <apenwarr@gmail.com>.
Bup unknown- bup-margin(1)