Sponsored Content
Top Forums Shell Programming and Scripting reformat one record from two records Post 302591035 by menglm on Wednesday 18th of January 2012 09:45:00 AM
Old 01-18-2012
reformat one record from two records

I have not get much answer/solution for the posting. Here I break down the question and hope to get some help.

1. How can I use AWK to read in two records at the same time and keep loop to next two when the condition is meet?








position 1-10 --> Unique to identity whether there is secondary record or not
--> If there is more than one record with the same value for this portion,
it means there is secondary record;
otherwise, there is only primary record
position 11 ---> 0 means the record is the secondary
1 means the record is the primary
Output file ----> starting from position 12
Segment definition --->starting from position 36 (TTTT)

XXXX###
XXXX---> Segment ID 4 bytes , eg TTTT or SH01
### ---> Total length of segment 020 means segment is 20 bytes long
See below string has two segment,
first one id is TTTT and length is 15 bytes long;
second one is is SH01 and length is 8 bytes long
TTTT015cvsdbfffSH01008X

ENDS segment format --->ENDS010###
ENDS010 --> segment id and length
### represents the total number of segment in current records.
For example ENDS010004 means there is 4 segments in the record including ENDS010 segment
-----------------
rules
1. if group by position 1-10 have one record, then reformat the string by cutting off first 11 bytes and output
2. if group by position 1-10 have two record, then
for the record with value as 1 in position 11,
then reformat string by
a. cutting of the first 11 bytes
b. recount the number of segments
c. append ENDS010### segment at the end of string

for the record with values as 0 in position 11,
then reformat string by
a. cut the first two segments from primary records and append them at the beginning of the output string
b. recount the number of segments
c. append ENDS010### segment at the end of string
----------------
Attached are two example files, one for input and one for output

Last edited by menglm; 01-23-2012 at 10:55 AM.. Reason: simplify the question
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

How to extract duplicate records with associated header record

All, I have a task to search through several hundred files and extract duplicate detail records and keep them grouped with their header record. If no duplicate detail record exists, don't pull the header. For example, an input file could look like this: input.txt HA D1 D2 D2 D3 D4 D4... (17 Replies)
Discussion started by: run_eim
17 Replies

2. UNIX for Dummies Questions & Answers

Parsing out records from one huge record

Hi, I have one huge record and know that each record in the file is 550 bytes long. How do I parse out individual records from the single huge record. Thanks, (4 Replies)
Discussion started by: bwrynz1
4 Replies

3. UNIX for Advanced & Expert Users

Parsing records from one record

Hi, I got a file which is one huge record. I know each record should be 550 bytes long. How do I parse out the records from the one huge record. (1 Reply)
Discussion started by: bwrynz1
1 Replies

4. Shell Programming and Scripting

converting all records of a file in to one record

I have a file temp.dat. The contents of this file is as follows abcdefgh abcdefgh abcdefgh abcdefgh abcdefgh abcdefgh The multiple records in this file needs to be converted in to a single record. abcdefgh abcdefgh abcdefgh abcdefgh abcdefgh abcdefgh (2 Replies)
Discussion started by: rsriramiyer
2 Replies

5. Shell Programming and Scripting

Multiple records based on ';' in the record

Hi All, I have a *.csv files in a die /pro/lif/dow, (pipe delimiter file), these files are having 8 columns and 6 column(CDR_LOGIC) records are populated as below, I need to incorporate the below logic in all the *.csv files. 11||:ColumnA||:ColumnB 123||:ColumnA IIF(:ColumnA = :ColumnC then... (6 Replies)
Discussion started by: shruthidwh
6 Replies

6. UNIX for Dummies Questions & Answers

Split single record to multiple records

Hi Friends, source .... col1,col2,col3 a,b,1;2;3 here colom delimeter is comma(,). here we dont know what is the max length of col3 means now we have 1;2;3 next time i will receive 1;2;3;4;5;etc... required output .............. col1,col2,col3 a,b,1 a,b,2 a,b,3 please give me... (5 Replies)
Discussion started by: bab.galary
5 Replies

7. Shell Programming and Scripting

Multiple Records from 1 Record

I need to make one record to multiple records based on occurence column in the record and change the date.For example below first record has 5 ,so need to create 5 records from one and change the date to 5 months.Occurence can be any number. I am unable to come with a script.Can some one help ... (5 Replies)
Discussion started by: traininfa
5 Replies

8. Shell Programming and Scripting

How to split one record to multiple records?

Hi, I have one tab delimited file which is having multiple store_ids in first column seprated by pipe.I want to split the file on the basis of store_id(separating 1st record in to 2 records ). I tried some more options like below with using split,awk etc ,But not able to get proper output. can... (1 Reply)
Discussion started by: jaggy
1 Replies

9. Shell Programming and Scripting

Sorting group of records and loading last record

Hi Everyone, I have below record set. File is fixed widht file 101newjersyus 20150110 101nboston us 20150103 102boston us 20140106 102boston us 20140103 I need to group record based on first 3 letters in our case(101 and 102) and sort last 8 digit in ascending order and print only... (4 Replies)
Discussion started by: patricjemmy6
4 Replies

10. UNIX for Beginners Questions & Answers

Help in printing records where there is a 'header' in the first record ???

Hi, I have a backup report that unfortunately has some kind of hanging indent thing where the first line contains one column more than the others I managed to get the output that I wanted using awk, but just wanting to know if there is short way of doing it using the same awk Below is what... (2 Replies)
Discussion started by: newbie_01
2 Replies
cdb(5)								File Formats Manual							    cdb(5)

NAME
cdb - Constant DataBase file format DESCRIPTION
A cdb database is a single file used to map `keys' to `values', having records of (key,value) pairs. File consists of 3 parts: toc (table of contents), data and index (hash tables). Toc has fixed length of 2048 bytes, containing 256 pointers to hash tables inside index sections. Every pointer consists of position of a hash table in bytes from the beginning of a file, and a size of a hash table in entries, both are 4-bytes (32 bits) unsigned integers in little-endian form. Hash table length may have zero length, meaning that corresponding hash table is empty. Right after toc section, data section follows without any alingment. It consists of series of records, each is a key length, value (data) length, key and value. Again, key and value length are 4-byte unsigned integers. Each next record follows previous without any special alignment. After data section, index (hash tables) section follows. It should be looked to in conjunction with toc section, where each of max 256 hash tables are defined. Index section consists of series of hash tables, with starting position and length defined in toc section. Every hash table is a sequence of records each holds two numbers: key's hash value and record position inside data section (bytes from the begin- ning of a file to first byte of key length starting data record). If record position is zero, then this is an empty hash table slot, pointed to nowhere. CDB hash function is hv = ((hv << 5) + hv) ^ c for every single c byte of a key, starting with hv = 5381. Toc section indexed by (hv % 256), i.e. hash value modulo 256 (number of entries in toc section). In order to find a record, one should: first, compute the hash value (hv) of a key. Second, look to hash table number hv modulo 256. If it is empty, then there is no such key exists. If it is not empty, then third, loop by slots inside that hash table, starting from slot with number hv divided by 256 modulo length of that table, or ((hv / 256) % htlen), searching for this hv in hash table. Stop search on empty slot (if record position is zero) or when all slots was probed (note cyclic search, jumping from end to beginning of a table). When hash value in question is found in hash table, look to key of corresponding record, comparing it with key in question. If them of the same length and equals to each other, then record is found, overwise, repeat with next hash table slot. Note that there may be several records with the same key. SEE ALSO
cdb(1), cdb(3). AUTHOR
The tinycdb package written by Michael Tokarev <mjt@corpit.ru>, based on ideas and shares file format with original cdb library by Dan Bernstein. LICENSE
Public domain. Apr, 2005 cdb(5)
All times are GMT -4. The time now is 08:55 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy