10 More Discussions You Might Find Interesting
1. UNIX for Beginners Questions & Answers
In the awk piped to sed below I am trying to format file by removing the odd xxxx_digits and whitespace after, then move the even xxxx_digit to the line above it and add a space between them. There may be multiple lines in file but they are in the same format. The Filename_ID line is the last line... (4 Replies)
Discussion started by: cmccabe
4 Replies
2. UNIX for Beginners Questions & Answers
In the awk below I am trying to remove all lines above and including the pattern Test or Test2. Each block is seperated by a newline and Test2 also appears in the lines to keep but it will always have additional text after it. The Test to remove will not. The awk executed until the || was added... (2 Replies)
Discussion started by: cmccabe
2 Replies
3. UNIX for Beginners Questions & Answers
Hi,
I'd be grateful for your help with the following. I have a file (file.txt) with 10 columns and about half a million lines, which in simplified form looks like this:
ID Col1 Col2 Col3....
a 4 2 8
b 5 6 1
c 8 4 1
d... (4 Replies)
Discussion started by: aberg
4 Replies
4. Shell Programming and Scripting
I am trying to remove each line in which $2 is FP or RFP. I believe the below will remove one instance but not both. Thank you :).
file
12
123 FP
11
10 RFP
awk
awk -F'\t' '
$2 != "FP"' file
desired output
12
11 (6 Replies)
Discussion started by: cmccabe
6 Replies
5. Shell Programming and Scripting
I am trying to remove lines in the target.txt file if $5 before the - in that file matches sorted_list. I have tried grep and awk. Thank you :).
grep
grep -v -F -f targets.bed sort_list
grep -vFf sort_list targets
awk
awk -F, '
> FILENAME == ARGV {to_remove=1; next}
> ! ($5 in... (2 Replies)
Discussion started by: cmccabe
2 Replies
6. UNIX for Dummies Questions & Answers
I have a file like this - I want to remove the 2015 (or any four digit #) from column $4 so I can get:
Nov 05 1997 /ifs/inventory2/
for example. Im not sure how. Should I use an if statement with awk?
Jan 16 2015 23:45 /ifs/sql_file
Jan 16 2015 23:45 /ifs/sql_file
Nov 05 2015 1997... (4 Replies)
Discussion started by: newbie2010
4 Replies
7. Shell Programming and Scripting
I need to send email to receipient in each block of data in a file which has the sender address under TO and just send that block of data where it ends as COMPANY.
I tried to work this out by getting line numbers of the string HELLO but unable to grab the next block of data to send the next... (5 Replies)
Discussion started by: loggedout
5 Replies
8. Shell Programming and Scripting
Hi,
I need to get specific parts in a large file.
I need to:
Get a line containing an IP address, and read from there to another line saying ***SNMP-END***
So, I have the start and the end well defined, but the problem is that apparently the awk command using the -F option doesn't work... (17 Replies)
Discussion started by: ocramas
17 Replies
9. Shell Programming and Scripting
hi,,
i hav a file with many lines.i need to remove all lines before a line begginning with a specific pattern from the file because these lines are not required.
Can u help me out with either a perl script or shell script
example:-
if file initially contains lines:
a
b
c
d
.1.2
d
e
f... (2 Replies)
Discussion started by: raksha.s
2 Replies
10. Shell Programming and Scripting
I want to search a file for a string and then if the string is found I need the line that the string is on - but also the previous two lines from the file (that the pattern will not be found in)
This is on solaris
Can you help? (2 Replies)
Discussion started by: frustrated1
2 Replies
bup-margin(1) General Commands Manual bup-margin(1)
NAME
bup-margin - figure out your deduplication safety margin
SYNOPSIS
bup margin [options...]
DESCRIPTION
bup margin iterates through all objects in your bup repository, calculating the largest number of prefix bits shared between any two
entries. This number, n, identifies the longest subset of SHA-1 you could use and still encounter a collision between your object ids.
For example, one system that was tested had a collection of 11 million objects (70 GB), and bup margin returned 45. That means a 46-bit
hash would be sufficient to avoid all collisions among that set of objects; each object in that repository could be uniquely identified by
its first 46 bits.
The number of bits needed seems to increase by about 1 or 2 for every doubling of the number of objects. Since SHA-1 hashes have 160 bits,
that leaves 115 bits of margin. Of course, because SHA-1 hashes are essentially random, it's theoretically possible to use many more bits
with far fewer objects.
If you're paranoid about the possibility of SHA-1 collisions, you can monitor your repository by running bup margin occasionally to see if
you're getting dangerously close to 160 bits.
OPTIONS
--predict
Guess the offset into each index file where a particular object will appear, and report the maximum deviation of the correct answer
from the guess. This is potentially useful for tuning an interpolation search algorithm.
--ignore-midx
don't use .midx files, use only .idx files. This is only really useful when used with --predict.
EXAMPLE
$ bup margin
Reading indexes: 100.00% (1612581/1612581), done.
40
40 matching prefix bits
1.94 bits per doubling
120 bits (61.86 doublings) remaining
4.19338e+18 times larger is possible
Everyone on earth could have 625878182 data sets
like yours, all in one repository, and we would
expect 1 object collision.
$ bup margin --predict
PackIdxList: using 1 index.
Reading indexes: 100.00% (1612581/1612581), done.
915 of 1612581 (0.057%)
SEE ALSO
bup-midx(1), bup-save(1)
BUP
Part of the bup(1) suite.
AUTHORS
Avery Pennarun <apenwarr@gmail.com>.
Bup unknown- bup-margin(1)