Apologies, my bad. I should've uploaded the file. Attached is a masked .dat file renamed as .txt for uploading.
On opening it with notepad++ in Windows, the null characters show up as boxes. In a Linux vi the nulls are ^@.
All the records in this file are in one row. This particular file has 2 records followed by the trailer record.
First record = starts at the beginning of the file 00000230 (this field gives the length of the record in bytes)
Second record = starts at the next 00000230 (it is a coincidence, here both records have same length)
Trailer record = starts at 0000096 (the trailer length is of 96 bytes and it also has 80 delimiters of ^@ or null characters. Ignore my earlier post saying trailer has 20 delimiters. It has 80 actually)
As the field lengths are variable so we cannot define a record in terms of total length of its fields or total bytes. This is why we are defining a record as effectively having length of 80 ^@ delimiters.
I require the 1st record in one row, 2nd record in next row and so on till the end of the file, with the trailer in the last row. If there is a way of adding a newline after every 80th ^@ from the beginning till the eof, then perhaps it will work?
The only unprintable character is null ^@, no TABS or other spaces, all other characters are alphanumeric.
Please let me know if any questions. Thanks for the help
Hi,
I need add leading zeroes to a field in a file based on the character count. The field can be of 1 character to 6 character length. I need to make the field 14bytes.
eg:
8351,20,1
8351,234,6
8351,2,0
8351,1234,2
8351,123456,1
8351,12345,2
This should become.
... (3 Replies)
Hi,
I have a No Delimiter variable length text file with following schema -
Column Name Data length
Firstname 5
Lastname 5
age 3
phoneno1 10
phoneno2 10
phoneno3 10
sample data - ... (16 Replies)
Hi not sure if this is possible but I need some help with a bash script, I have a text file and on the first line that starts with 7150230 I need it to put a 1 at position 79 and a 2 at position 88, this is where it gets complicated, on the next line it finds that starts with 7150230 I then need it... (8 Replies)
Hi,
I have a requirement where in I need to insert delimiters before the last column of the total delimiters is less than a specified number.
Say if the delimiters is less than 139, I need to insert 2 columns ( with blanks) before the last field
awk -F 'Ç' '{ if (NF-1 < 139)} END { "Insert 2... (5 Replies)
I have file listed in my directory in following format
-rwxrwxr-x+ 1 test test 4.9M Oct 3 16:06 test20141002150108.txt
-rwxrwxr-x+ 1 test test 4.9M Oct 4 16:06 test20141003150108.txt
-rwxrwxr-x+ 1 test test 4.9M Oct 5 16:06 test20141005150108.txt
-rwxrwxr-x+ 1 test ... (2 Replies)
Hi,
I have tried to remove dublicate lines based on first column with pipe delimiter . but i ma not able to get some uniqu lines
Command : sort -t'|' -nuk1 file.txt
Input :
38376KZ|09/25/15|1.057
38376KZ|09/25/15|1.057
02006YB|09/25/15|0.859
12593PS|09/25/15|2.803... (2 Replies)
Hi Folks,
I have a file with fields as follows which has last field in multiple lines. I would like to combine a line which has three fields with single field line for as shown in expected output. Please help.
INPUT
hname01 windows appnamec1eda_p1, ... (5 Replies)
I want to count the number of lines, I need this result be a number, and sum the last numeric column, I had done to make this one at time, but I need to make this for a crontab, so, it has to be an script, here is my lines:
It counts the number of lines:
egrep -i String file_name_201611* |... (5 Replies)
I have a huge file (around 4-5 GB containing 20 million rows) which has text like:
<EOFD>11<EOFD>22<EORD>2<EOFD>2222<EOFD>3333<EORD>3<EOFD>44<EOFD>55<EORD>66<EOFD>888<EOFD>9999<EORD>
Actually above is an extracted file from a Sql Server with each field delimited by <EOFD> and each row ends... (8 Replies)
Discussion started by: amvip
8 Replies
LEARN ABOUT DEBIAN
cdb
cdb(5) File Formats Manual cdb(5)NAME
cdb - Constant DataBase file format
DESCRIPTION
A cdb database is a single file used to map `keys' to `values', having records of (key,value) pairs. File consists of 3 parts: toc (table
of contents), data and index (hash tables).
Toc has fixed length of 2048 bytes, containing 256 pointers to hash tables inside index sections. Every pointer consists of position of a
hash table in bytes from the beginning of a file, and a size of a hash table in entries, both are 4-bytes (32 bits) unsigned integers in
little-endian form. Hash table length may have zero length, meaning that corresponding hash table is empty.
Right after toc section, data section follows without any alingment. It consists of series of records, each is a key length, value (data)
length, key and value. Again, key and value length are 4-byte unsigned integers. Each next record follows previous without any special
alignment.
After data section, index (hash tables) section follows. It should be looked to in conjunction with toc section, where each of max 256
hash tables are defined. Index section consists of series of hash tables, with starting position and length defined in toc section. Every
hash table is a sequence of records each holds two numbers: key's hash value and record position inside data section (bytes from the begin-
ning of a file to first byte of key length starting data record). If record position is zero, then this is an empty hash table slot,
pointed to nowhere.
CDB hash function is
hv = ((hv << 5) + hv) ^ c
for every single c byte of a key, starting with hv = 5381.
Toc section indexed by (hv % 256), i.e. hash value modulo 256 (number of entries in toc section).
In order to find a record, one should: first, compute the hash value (hv) of a key. Second, look to hash table number hv modulo 256. If
it is empty, then there is no such key exists. If it is not empty, then third, loop by slots inside that hash table, starting from slot
with number hv divided by 256 modulo length of that table, or ((hv / 256) % htlen), searching for this hv in hash table. Stop search on
empty slot (if record position is zero) or when all slots was probed (note cyclic search, jumping from end to beginning of a table). When
hash value in question is found in hash table, look to key of corresponding record, comparing it with key in question. If them of the same
length and equals to each other, then record is found, overwise, repeat with next hash table slot. Note that there may be several records
with the same key.
SEE ALSO cdb(1), cdb(3).
AUTHOR
The tinycdb package written by Michael Tokarev <mjt@corpit.ru>, based on ideas and shares file format with original cdb library by Dan
Bernstein.
LICENSE
Public domain.
Apr, 2005 cdb(5)