11-23-2016
Quote:
Originally Posted by
digitalnirvana
Hi Don,
Thanks and sorry for not responding earlier. I've tried this out on our test Linux box (running redhat) and it's outputting a blank file.
I'll try and give you a properly formatted expected output file tonight.
What do you mean by field separators vs field terminators please? In our case each record has 80 fields separated by 80 NULL (^@) characters.
In a CSV formatted file, you have a character that is a field separator and a record is terminated by a <newline> character. In a CSV record with 80 field separators, there are 81 fields. If your file has field separators (instead of field terminators), there is no way to know where the last field of one record ends and the 1st field of the next record starts without counting bytes based on the number that you say is in the 1st field of each record. If you have field terminators (instead of field separators), we just have to add a <newline> after the last field terminator and don't ever have to look at the first field and count bytes to find the end of a record.
From the sample you downloaded before, it appears that the NUL characters are field terminators and that there are 80 NUL terminated fields per record and that the sample contained three records. The script I suggested in post #8 inserts a <newline> character into the output after finding 80 field terminators.
10 More Discussions You Might Find Interesting
1. UNIX for Dummies Questions & Answers
Hi Guys,
I want to write a perl/shell script do parse the following file
input file content
NPA-NXX SC
2084549 45
2084552 45
2084563 2007
2084572 45
2084580 45
3278411 45
3278430 45
3278493 530
3278507 530... (3 Replies)
Discussion started by: pistachio
3 Replies
2. Shell Programming and Scripting
Hi,
I need add leading zeroes to a field in a file based on the character count. The field can be of 1 character to 6 character length. I need to make the field 14bytes.
eg:
8351,20,1
8351,234,6
8351,2,0
8351,1234,2
8351,123456,1
8351,12345,2
This should become.
... (3 Replies)
Discussion started by: gpaulose
3 Replies
3. Shell Programming and Scripting
Hi,
I have a No Delimiter variable length text file with following schema -
Column Name Data length
Firstname 5
Lastname 5
age 3
phoneno1 10
phoneno2 10
phoneno3 10
sample data - ... (16 Replies)
Discussion started by: Gaurav Martha
16 Replies
4. Shell Programming and Scripting
Hi not sure if this is possible but I need some help with a bash script, I have a text file and on the first line that starts with 7150230 I need it to put a 1 at position 79 and a 2 at position 88, this is where it gets complicated, on the next line it finds that starts with 7150230 I then need it... (8 Replies)
Discussion started by: firefox2k2
8 Replies
5. Shell Programming and Scripting
Hi,
I have a requirement where in I need to insert delimiters before the last column of the total delimiters is less than a specified number.
Say if the delimiters is less than 139, I need to insert 2 columns ( with blanks) before the last field
awk -F 'Ç' '{ if (NF-1 < 139)} END { "Insert 2... (5 Replies)
Discussion started by: arunkesi
5 Replies
6. Shell Programming and Scripting
I have file listed in my directory in following format
-rwxrwxr-x+ 1 test test 4.9M Oct 3 16:06 test20141002150108.txt
-rwxrwxr-x+ 1 test test 4.9M Oct 4 16:06 test20141003150108.txt
-rwxrwxr-x+ 1 test test 4.9M Oct 5 16:06 test20141005150108.txt
-rwxrwxr-x+ 1 test ... (2 Replies)
Discussion started by: krish2014
2 Replies
7. Shell Programming and Scripting
Hi,
I have tried to remove dublicate lines based on first column with pipe delimiter . but i ma not able to get some uniqu lines
Command : sort -t'|' -nuk1 file.txt
Input :
38376KZ|09/25/15|1.057
38376KZ|09/25/15|1.057
02006YB|09/25/15|0.859
12593PS|09/25/15|2.803... (2 Replies)
Discussion started by: parithi06
2 Replies
8. Shell Programming and Scripting
Hi Folks,
I have a file with fields as follows which has last field in multiple lines. I would like to combine a line which has three fields with single field line for as shown in expected output. Please help.
INPUT
hname01 windows appnamec1eda_p1, ... (5 Replies)
Discussion started by: shunya
5 Replies
9. Shell Programming and Scripting
I want to count the number of lines, I need this result be a number, and sum the last numeric column, I had done to make this one at time, but I need to make this for a crontab, so, it has to be an script, here is my lines:
It counts the number of lines:
egrep -i String file_name_201611* |... (5 Replies)
Discussion started by: Elly
5 Replies
10. Shell Programming and Scripting
I have a huge file (around 4-5 GB containing 20 million rows) which has text like:
<EOFD>11<EOFD>22<EORD>2<EOFD>2222<EOFD>3333<EORD>3<EOFD>44<EOFD>55<EORD>66<EOFD>888<EOFD>9999<EORD>
Actually above is an extracted file from a Sql Server with each field delimited by <EOFD> and each row ends... (8 Replies)
Discussion started by: amvip
8 Replies
LEARN ABOUT DEBIAN
srec_emon52
srec_emon52(5) File Formats Manual srec_emon52(5)
NAME
srec_emon52 - Elektor Monitor (EMON52) file format
DESCRIPTION
This format is used by the monitor EMON52, developed by the European electronics magazine Elektor (Elektuur in Holland). Elektor wouldn't
be Elektor if they didn't try to reinvent the wheel. It's a mystery why they didn't use an existing format for the project. Only the
Elektor Assembler will produce this file format, reducing the choice of development tools dramatically.
Records
All data lines are called records, and each record contains the following four fields:
+---+------+---+-----------+------+
|cc | aaaa | : | dd ... dd | ssss |
The field are defined as follows: +---+------+---+-----------+------+
cc The byte count. A two digit hex value (1 byte), counting the actual data bytes in the record. The byte count is separated from
the next field by a space.
aaaa The address field. A four hex digit (2 byte) number representing the first address to be used by this record.
: The address field and the data field are separated by a colon.
dd The actual data of this record. There can be 1 to 255 data bytes per record (see cc) All bytes in the record are separated from
each other (and the checksum) by a space.
ssss Data Checksum, adding all bytes of the data line together, forming a 16 bit checksum. Covers only all the data bytes of this
record.
Please note that there is no End Of File record defined.
Byte Count
The byte count cc counts the actual data bytes in the current record. Usually records have 16 data bytes. I don't know what the maximum
number of data bytes is. It depends on the size of the data buffer in the EMON52.
Address Field
This is the address where the first data byte of the record should be stored. After storing that data byte, the address is incremented by
1 to point to the address for the next data byte of the record. And so on, until all data bytes are stored.
The address is represented by a 4 digit hex number (2 bytes), with the MSD first.
Data Field
The payload of the record is formed by the Data field. The number of data bytes expected is given by the Byte Count field.
Checksum
The checksum is a 16 bit result from adding all data bytes of the record together.
Size Multiplier
In general, binary data will expand in sized by approximately 3.8 times when represented with this format.
EXAMPLE
Here is an example of an EMON52 file:
10 0000:57 6F 77 21 20 44 69 64 20 79 6F 75 20 72 65 61 0564
10 0010:6C 6C 79 20 67 6F 20 74 68 72 6F 75 67 68 20 61 05E9
10 0020:6C 6C 20 74 68 69 73 20 74 72 6F 75 62 6C 65 20 05ED
10 0030:74 6F 20 72 65 61 64 20 74 68 69 73 20 73 74 72 05F0
04 0040:69 6E 67 21 015F
SEE ALSO
http://sbprojects.fol.nl/knowledge/fileformats/emon52.htm
AUTHOR
This man page was taken from the above Web page. It was written by San Bergmans <sanmail@bigfoot.com>
Reference Manual SRecord srec_emon52(5)