Thanks for your inputs, can this be modified in my code which runs in bash
The tail command in RudiC's suggestion copies your source files to their destinations removing "$HDCNT" lines of headers during the copy.
Your code above will do two things:
The shell will see the redirection and truncate the file named by the expansion of $file to size 0, and then
The sed command will delete the 1st line (of the empty file) named by the expansion of $file and copy the remaining (non-existent) lines to the output.
Do you really want to change your input files before you copy their contents to their destinations, or do you just need to remove the header lines from the copied files? If you don't need to modify the source files:
will do what you want. If you need to change the source files too:
is a safe way to do what you want without breaking links to your source file (if any existed).
Hi,
I need some help in removing the header (first line) and the trailer (last line) in a give file...
The data file actually comes in EBCDIC format and I converted it into ASCII..
Now I need to strip off the first line and the last line..
I think we can use sed to do something like this:... (2 Replies)
All,
I am new to unix and i have the following requirement.
I have file(s) landing into input directory with timestamp, first i want to copy all these files into seperate directory then i want to rename these files without timestamp and also remove header,trailer from that file..
Could... (35 Replies)
Hi,
Here we have a situation where we have to check the file size and if the file size is greater than 0 bytes then remove the files from the directory.
1)EdwTrxn 2)EdwPost 3)EdwTndr 4)EdwSls 5)EdwSlsRej 6)EdwTndrRej
Files will be created in the directory in the following manner.
... (5 Replies)
Hi
I am writing a perl script which checks for the specific column values from a file and writes to the OUT file.
So the feed file has a header information and footer information.
I header information isaround107 lines i.e.
Starts with
START-OF-FILE
.......
so on ....
... (11 Replies)
Hi All,
I am trying write a simple command using AWK and SED to this but without any success.
Here is what I am using:
head -1 test1.txt>test2.txt|sed '1d;$d' test1.txt|awk '{print substr($0,0,(length($0)-2))}' >>test2.txt|tail -1 test1.txt>>test2.txt
Input:
Header
1234567
abcdefgh... (2 Replies)
Hi All,
I want to remove the content based on the header information .
Please find the example below.
File1.txt
Name|Last|First|Location|DepId|Depname|DepLoc
naga|rr|tion|hyd|1|wer|opr
Nava|ra|tin|gen|2|wera|opra
I have to search for the DepId and remove the data from the... (5 Replies)
Hi,
In my server, the hostname and IP addresses are added to the known_hosts which then be hashed by ssh-keygen -H -f. Is it possible to check if an IP address has already been added to the hashed known_hosts to avoid duplications (I think there will be problems if there are duplicated IP... (3 Replies)
Hi Folks,
I have a requirement to develop a shell script. PFB my requirement,
Requirement:
I need to check an empty line after the end of each header in respective file and if a empty line is present simply echo file OK and if empty line is not present echo "Adding empty line" and add an... (6 Replies)
I've been struggling with this one for quite a while and cannot seem to find a solution for this find/replace scenario. Perhaps I'm getting rusty.
I have a file that contains a number of metrics (exactly 3 fields per line) from a few appliances that are collected in parallel. To identify the... (3 Replies)
Discussion started by: verdepollo
3 Replies
LEARN ABOUT DEBIAN
bup-margin
bup-margin(1) General Commands Manual bup-margin(1)NAME
bup-margin - figure out your deduplication safety margin
SYNOPSIS
bup margin [options...]
DESCRIPTION
bup margin iterates through all objects in your bup repository, calculating the largest number of prefix bits shared between any two
entries. This number, n, identifies the longest subset of SHA-1 you could use and still encounter a collision between your object ids.
For example, one system that was tested had a collection of 11 million objects (70 GB), and bup margin returned 45. That means a 46-bit
hash would be sufficient to avoid all collisions among that set of objects; each object in that repository could be uniquely identified by
its first 46 bits.
The number of bits needed seems to increase by about 1 or 2 for every doubling of the number of objects. Since SHA-1 hashes have 160 bits,
that leaves 115 bits of margin. Of course, because SHA-1 hashes are essentially random, it's theoretically possible to use many more bits
with far fewer objects.
If you're paranoid about the possibility of SHA-1 collisions, you can monitor your repository by running bup margin occasionally to see if
you're getting dangerously close to 160 bits.
OPTIONS --predict
Guess the offset into each index file where a particular object will appear, and report the maximum deviation of the correct answer
from the guess. This is potentially useful for tuning an interpolation search algorithm.
--ignore-midx
don't use .midx files, use only .idx files. This is only really useful when used with --predict.
EXAMPLE
$ bup margin
Reading indexes: 100.00% (1612581/1612581), done.
40
40 matching prefix bits
1.94 bits per doubling
120 bits (61.86 doublings) remaining
4.19338e+18 times larger is possible
Everyone on earth could have 625878182 data sets
like yours, all in one repository, and we would
expect 1 object collision.
$ bup margin --predict
PackIdxList: using 1 index.
Reading indexes: 100.00% (1612581/1612581), done.
915 of 1612581 (0.057%)
SEE ALSO bup-midx(1), bup-save(1)BUP
Part of the bup(1) suite.
AUTHORS
Avery Pennarun <apenwarr@gmail.com>.
Bup unknown-bup-margin(1)