Thanks RudiC. I took your suggestions into consideration and combined all those commands into one awk command. Thanks so much.
In doing the above, i discovered the code i originally pasted in this thread is not the reason why the script was slow. I found out that it is the for loop below that takes at least 4 seconds to complete.
Hi,
I have this following script below. Its searching a log file for 2 string and if found then write the strings to success.txt and If not found write strings to failed.txt . if one found and not other...then write found to success.txt and not found to failed.txt.
I want to optimize this... (3 Replies)
can we optimize this command ?
sed 's#AAAA##g' /study/i.txt | sed '1,2d' | tr -d '\n\' > /study/i1.txt;
as here i am using two files ...its overhead..can we optimise to use only 1 file
sed 's#AAAA##g' /study/i.txt | sed '1,2d' | tr -d '\n\' > /study/i.txt;
keeping them same but it... (9 Replies)
#!/usr/bin/perl
use strict;
use warnings;
use Date::Manip;
my $date_converted = UnixDate(ParseDate("3 days ago"),"%e/%h/%Y");
open FILE,">$ARGV";
while(<DATA>){
my @tab_delimited_array = split(/\t/,$_);
$tab_delimited_array =~ s/^\ =~ s/^\-//;
my $converted_date =... (2 Replies)
Pl help to me to write the below code in a simple way ...
i suupose to use this code 3 to 4 places in my makefile(gnu) ..
****************************************
@for i in $(LIST_A); do \
for j in $(LIST_B); do\
if ;then\
echo "Need to sign"\
echo "List A = $$i , List B =$$j"\
... (2 Replies)
Hi,
I have to assign a value for a varaiable based on a Input. I have written the below code:
if
then
nf=65
elif
then
nf=46
elif
then
nf=164
elif
then
nf=545
elif
then
nf=56
elif
then (3 Replies)
Here is my code. What it does is it reads an input file (input.txt which contains roughly 2,000 search phrases) and searches a directory for files that contains the search phrase. The directory contains roughly 1900 files and 84 subdirectories. The output is a file (output.txt) that shows only the... (23 Replies)
Hello,
I'm wondering if there is a quicker way of doing this.
Here is my mv script.
d=/conversion/program/out
cd $d
ls $d > /home/tempuser/$$tmp
while read line ; do
a=`echo $line|cut -c1-5|sed "s/_//g"`
b=`echo $line|cut -c16-21`
if ;then mkdir... (13 Replies)
Hi guys,
I feel a bit comfortable now doing bash scripting but I am worried that the way I do it is not optimized and I can do much better as to how I code.
e.g.
I have a whole line in a file from which I want to extract some values.
Right now what I am doing is :
STATE=`cat... (5 Replies)
how can i optimize the following:
TOTALRESULT="total1=4
total2=9
total3=89
TMEMORY=1999"
TOTAL1=$(echo "${TOTALRESULT}" | egrep "total1=" | awk -F"=" '{print $NF}')
TOTAL2=$(echo "${TOTALRESULT}" | egrep "total2=" | awk -F"=" '{print $NF}')
TOTAL3=$(echo... (4 Replies)
Discussion started by: SkySmart
4 Replies
LEARN ABOUT DEBIAN
bup-margin
bup-margin(1) General Commands Manual bup-margin(1)NAME
bup-margin - figure out your deduplication safety margin
SYNOPSIS
bup margin [options...]
DESCRIPTION
bup margin iterates through all objects in your bup repository, calculating the largest number of prefix bits shared between any two
entries. This number, n, identifies the longest subset of SHA-1 you could use and still encounter a collision between your object ids.
For example, one system that was tested had a collection of 11 million objects (70 GB), and bup margin returned 45. That means a 46-bit
hash would be sufficient to avoid all collisions among that set of objects; each object in that repository could be uniquely identified by
its first 46 bits.
The number of bits needed seems to increase by about 1 or 2 for every doubling of the number of objects. Since SHA-1 hashes have 160 bits,
that leaves 115 bits of margin. Of course, because SHA-1 hashes are essentially random, it's theoretically possible to use many more bits
with far fewer objects.
If you're paranoid about the possibility of SHA-1 collisions, you can monitor your repository by running bup margin occasionally to see if
you're getting dangerously close to 160 bits.
OPTIONS --predict
Guess the offset into each index file where a particular object will appear, and report the maximum deviation of the correct answer
from the guess. This is potentially useful for tuning an interpolation search algorithm.
--ignore-midx
don't use .midx files, use only .idx files. This is only really useful when used with --predict.
EXAMPLE
$ bup margin
Reading indexes: 100.00% (1612581/1612581), done.
40
40 matching prefix bits
1.94 bits per doubling
120 bits (61.86 doublings) remaining
4.19338e+18 times larger is possible
Everyone on earth could have 625878182 data sets
like yours, all in one repository, and we would
expect 1 object collision.
$ bup margin --predict
PackIdxList: using 1 index.
Reading indexes: 100.00% (1612581/1612581), done.
915 of 1612581 (0.057%)
SEE ALSO bup-midx(1), bup-save(1)BUP
Part of the bup(1) suite.
AUTHORS
Avery Pennarun <apenwarr@gmail.com>.
Bup unknown-bup-margin(1)