Thanks mr.bean, but what do you do when the keywords in column two contains spaces and filler has multiple entries? Like if there was a "ITEM 1" in the following? Is there a way to go by byte position?
Hi
I have the fixed width flat file having the following data
12345aaaaaaaaaabbbbbbbbbb
12365sssssssssscccccccccc
12365sssss
12367ddddddddddvvvvvvvvvv
12367 vvvvv
Here the first column is length 5 second is length 10 third is length 10
if the second or third column exceeds... (3 Replies)
The symbols are \t and \t\t (note: not tab)
If the line starts with \t merge them into a single line upto symbol \t\t
\t\t to end and start new line
I able to join in a single line but not ending at \t\t and I completely confused
help would be appreciated:b::D
Input
\ta tab XXXXXXXXXX
\te... (5 Replies)
Hello,
I have searched forum trying to find a solution to my problem, but could not find anything or I did not understand the examples....
I should say, I am very inexperienced with text processing.
I have a text file with approx 60k lines in it.
I need to merge lines based on the number... (8 Replies)
Hi Guys,
After windows died on my netbook I installed Lubuntu and discovered Gawk about a month ago. After using Excel for 10+ years I'm amazed how quick and easily Gawk can process data but I'm stuck with a little problem merging data from multiple lines.
I'm an SEO Consultant and provide... (9 Replies)
This is a variation of an earlier post found here:
unixcom/shell-programming-scripting/159821-merge-two-non-consecutive-lines.html
User Bartus11 was kind enough to solve that example.
Previously, I needed help combining two lines that are non-consecutive in a file. Now I need to do the... (7 Replies)
Hello Everyone,
I have two files I created in a format similar to the ones found below (character position is important):
File 1:
21 Cat Y N S Y Y N N
FOUR LEGS
TAIL
WHISKERS
30 Dog N N 1 Y Y N N
FOUR LEGS
TAIL
33 Fish Y N 1 Y Y N N
FINS
43 CAR Y N S Y Y N N
WHEELS
DOORS... (7 Replies)
Hi,
I have a requirement to merge multiple lines based on search pattern. The search criteria is : it will search for CONSTRAINT and when it found CONSTRAINT, it will merge all lines to 1 line till it founds blank line.
For Example:
CREATE TABLE "AMS_DISTRIBUTOR_XREF"
(
"SOURCE"... (5 Replies)
Hi all.
This is the content of the text file used for the e-mail:
TM ICP-EDW BILLING REGISTER USAGE BREAKDOWN_01062014.csv
TM_ICP_EDWH_FICL_13062014.TXT
TM_ICP_EDWH_FICL_16062014.TXT
TM_ICP_EDW_Detailed Payment Journal Report_13062014.txt
TM_ICP_EDW_Detailed Payment Journal... (9 Replies)
Hi,
trying to knock something together to create one line entries based on whether the first word on each line matches a particular value.
eg.
Link,"Name=""Something\something"","Timeout=""1800""",
"Target=""\\thing\thing\thing""","State=""ONLINE""",something,... (0 Replies)
I am trying to merge two lines to one based on some matching condition.
The file is as follows:
Matches filter:
'request ', timestamp, <HTTPFlow
request=<GET:
Matches filter:
'request ', timestamp, <HTTPFlow
request=<GET:
Matches filter:
'request ', timestamp, <HTTPFlow
... (8 Replies)
Discussion started by: jamie_123
8 Replies
LEARN ABOUT DEBIAN
bup-margin
bup-margin(1) General Commands Manual bup-margin(1)NAME
bup-margin - figure out your deduplication safety margin
SYNOPSIS
bup margin [options...]
DESCRIPTION
bup margin iterates through all objects in your bup repository, calculating the largest number of prefix bits shared between any two
entries. This number, n, identifies the longest subset of SHA-1 you could use and still encounter a collision between your object ids.
For example, one system that was tested had a collection of 11 million objects (70 GB), and bup margin returned 45. That means a 46-bit
hash would be sufficient to avoid all collisions among that set of objects; each object in that repository could be uniquely identified by
its first 46 bits.
The number of bits needed seems to increase by about 1 or 2 for every doubling of the number of objects. Since SHA-1 hashes have 160 bits,
that leaves 115 bits of margin. Of course, because SHA-1 hashes are essentially random, it's theoretically possible to use many more bits
with far fewer objects.
If you're paranoid about the possibility of SHA-1 collisions, you can monitor your repository by running bup margin occasionally to see if
you're getting dangerously close to 160 bits.
OPTIONS --predict
Guess the offset into each index file where a particular object will appear, and report the maximum deviation of the correct answer
from the guess. This is potentially useful for tuning an interpolation search algorithm.
--ignore-midx
don't use .midx files, use only .idx files. This is only really useful when used with --predict.
EXAMPLE
$ bup margin
Reading indexes: 100.00% (1612581/1612581), done.
40
40 matching prefix bits
1.94 bits per doubling
120 bits (61.86 doublings) remaining
4.19338e+18 times larger is possible
Everyone on earth could have 625878182 data sets
like yours, all in one repository, and we would
expect 1 object collision.
$ bup margin --predict
PackIdxList: using 1 index.
Reading indexes: 100.00% (1612581/1612581), done.
915 of 1612581 (0.057%)
SEE ALSO bup-midx(1), bup-save(1)BUP
Part of the bup(1) suite.
AUTHORS
Avery Pennarun <apenwarr@gmail.com>.
Bup unknown-bup-margin(1)