Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

awk get length of a binary file.

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting awk get length of a binary file.
# 1  
Old 05-07-2012
awk get length of a binary file.
Hi..

Just got a problem with awk and cannot figure out whats the issue. I am not really expert in awk so looking for help/opinion from experts

Command is

Code:
$ cat abinaryfile | sed -e "s/@@//g;s/@$//"  | awk ' begin {}    {        printf(length($0));    }'

Well its a binary file and I am looking to get a count of this file. If I use awk version 3.1.3 . it counts the whole binary file and give me desired result (9126) but we are roling out to RHEL6 and awk version in this is 3.1.7 and it seems there is some change in this version as now it counts only I suppose first column.(8)
Code:
--------------------------------------------
$ cat TT120T0-inbound | sed -e "s/@@//g;s/@$//"  |  awk ' begin {}    {        printf(length());    }'
8 

(Awk 3.1.7) 
------------------------------------------------------
$ cat TT120T0-inbound | sed -e "s/@@//g;s/@$//"  |  ./awk ' begin {}    {        printf(length());    }'
9126 

( Awk 3.1.3)
----------------------------------------------------
I might be doing wrong so it anyone can tell me how to count the length of whole binary file..

Any help would be great.


Cheers
Sam

Quote:
---------- Post updated at 11:42 AM ---------- Previous update was at 11:29 AM ----------

Binary File contains this data
------------------------------------------
J1644�697040010502500112050300000013606033010122001T00000001�1644�DŽ
165186007600090864000000120412063056C10101C5434C6036341701100000000279900000000000023953339221030000 0000007406013606069996740059622080002003MCC0003003MCC014801239204582840201490063920000158029MCC10401 01 12050303 NNNNN0165030M 01910012022800111000060Automatic response message for DE031 95333922103000000000074016 MCC 0419 00000002060136060699967406013606�1644�DŽ
165186007600090864000000120412063056C10101C5434C6036322701100000000279900000000000023953339221030000 0000019906013606069996740059622080002003MCC0003003MCC014801239204582840201490063920000158029MCC10401 01 12050303 NNNNN0165030M 01910012022800111000060Automatic response message for DE031 95333922103000000000199016 MCC 0419 00000003060136060699967406013606�1644�DŽ
165186007600090864000000120412063311C10101C5434C603630570110000000026200000000@@00000239533392210300 000000011606013606069996740059642080002003MCC0003003MCC014801282624582840201490068260000158029MCC104 0101 12050303 NNNNN0165030M 01910012022800111000060Automatic response message for DE031 95333922103000000000116016 MCC 0419 00000004060136060699967406013606�1644�DŽ
1651860076000908640000001204120631532209992592226036342701100000000379700000000000023953339221030000 00000066060136060699967400596Z2040002003MCC0003003MCC01480088402458201490068400000158029MCC1040101 12050303 NNNNN0165030M 01910012022800111000060Automatic response message for DE031 95333922103000000000066016 MCC 0419 00000005060136060699967406013606�1644�DŽ
1651860076000908640000001204120631532209992592226036322701100000000379700000000000023953339221030000 00000181060136060699967400596Z2040002003MCC0003003MCC01480088402458201490068400000158029MCC1040101 12050303 NNNNN0165030M 01910012@@022800111000060Automatic response message for DE031 95333922103000000000181016 MCC 0419 00000006060136060699967406013606�1442�DŽ!�16518600760009086400000000000000350400000001058663021118120 412063056C10101C5434C4504849701100000000279900000000000023953339221030000000001080601360606999674005 96263TRAVELEX MERCHANT\JALANTUN PERAK\MENARA MAYBAN\50050 KULMYS3440002003MCC0003003MCC0023003NA 0146036002901840000000000039458000000000118014801239204582840201490063920000158029MCC104010173120503 03 NNNNN01590672485 6870582456 1AP00000004M12050303120503010165030M 0177002YN01910012026200101000060Automatic response message for DE031 953339221030000000001088404580200058458D000000010000016 MCC 0419 00000007060136060699967410001606924306013606�1442�DŽ!�16518600760009086400000000000000425500000001285 463020916120412063311C10101C5434C4504807701100000000262000000000000023953339221030000000000250601360 60699967400596463TRAVELEX@@ MERCHANT\JALANTUN PERAK\MENARA MAYBAN\50050 KULMYS3440002003MCC0003003MCC0023003NA 0146036002901840000000000047458000000000142014801282624582840201490068260000158029MCC104010173120503 03 NNNNN01590672485 6870582456 1AP00000004M12050303120503010165030M 0177002YN01910012026200101000060Automatic response message for DE031 953339221030000000000258404580200058458D000000012000016 MCC 0419 00000008060136060699967410001606924506013606�1442�DŽ!�16518600760009086400000000000000425500000001285 463020916120412063311C10101C5434C4504849701100000000262000000000000023953339221030000000001400601360 60699967400596463TRAVELEX MERCHANT\JALANTUN PERAK\MENARA MAYBAN\50050 KULMYS3440002003MCC0003003MCC0023003NA 0146036002901840000000000047458000000000142014801282624582840201490068260000158029MCC104010173120503 03 NNNNN01590672485 6870582456 1AP00000004M12050303120503010165030M 0177002YN019@@10012026200101000060Automatic response message for DE031 953339221030000000001408404580200058458D000000012000016 MCC 0419 00000009060136060699967410001606924706013606�1442�DŽ!�16518600760009086400000000000000379700000001147 1630210691204120631532209992592224504808701100000000379700000000000023953339221030000000000900601360 60699967400596Z63TRAVELEX MERCHANT\JALANTUN PERAK\MENARA MAYBAN\50050 KULMYS3400002003MCC0003003MCC0023003NA 014603600290184000000000004245800000000012701480088402458201490068400000158029MCC10401017312050303 NNNNN01590672485 6870582456 1AP00000004M12050303120503010165030M 0177002YN01910012026200101000060Automatic response message for DE031 953339221030000000000908404580200058458D000000011000016 MCC 0419 00000010060136060699967410001606924906013606�1442�DŽ!�16518600760009086400000000000000175200000000529 363021118120412063056C10101C5434C45348077011000000002799000000000000239533392210300@@000000013206013 6060699967400596263TRAVELEX MERCHANT\JALANTUN PERAK\MENARA MAYBAN\50050 KULMYS3440002003MCC0003003MCC0023003NA 0146036002901840000000000019458000000000057014801239204582840201490063920000158029MCC104010173120503 03 NNNNN01590672485 6870582456 1AP00000004M12050303120503010165030M 0177002YN01910012026200101000060Automatic response message for DE031 953339221030000000001328404580200058458D000000010000016 MCC 0419 00000011060136060699967410001606924406013606�1442�DŽ!�16518600760009086400000000000000141800000000428 463021156120412063311C10101C5434C4534808701100000000262000000000000023953339221030000000000580601360 60699967400596463TRAVELEX MERCHANT\JALANTUN PERAK\MENARA MAYBAN\50050 KULMYS3440002003MCC0003003MCC0023003NA 0146036002901840000000000016458000000000048014801282624582840201490068260000158029MCC104010173120503 03 NNNNN01590672485 6870582456 1AP00000004M12050303120503010165030@@M 0177002YN01910012026200101000060Automatic response message for DE031 953339221030000000000588404580200058458D000000012000016 MCC 0419 00000012060136060699967410001606924606013606�1442�DŽ!�16518600760009086400000000000000212700000000642 663021156120412063311C10101C5434C4534849701100000000262000000000000023953339221030000000001730601360 60699967400596463TRAVELEX MERCHANT\JALANTUN PERAK\MENARA MAYBAN\50050 KULMYS3440002003MCC0003003MCC0023003NA 0146036002901840000000000023458000000000069014801282624582840201490068260000158029MCC104010173120503 03 NNNNN01590672485 6870582456 1AP00000004M12050303120503010165030M 0177002YN01910012026200101000060Automatic response message for DE031 953339221030000000001738404580200058458D000000012000016 MCC 0419 00000013060136060699967410001606924806013606�1442�DŽ!�16518600760009086400000000000000094900000000286 7630210741204120631532209992592224534@@8087011000000003797000000000000239533392210300000000012406013 6060699967400596Z63TRAVELEX MERCHANT\JALANTUN PERAK\MENARA MAYBAN\50050 KULMYS3400002003MCC0003003MCC0023003NA 014603600290184000000000001045800000000003001480088402458201490068400000158029MCC10401017312050303 NNNNN01590672485 6870582456 1AP00000004M12050303120503010165030M 0177002YN01910012026200101000060Automatic response message for DE031 953339221030000000001248404580200058458D000000011000016 MCC 0419 00000014060136060699967410001606925006013606�644��68068621660148004840203000250011205030000001360603 3010380017D00000000000220570381017C00000000000000000384017D00000000000220570400010000000000804010100 00000000004020100000000008840000000150601360606013606g1644��6856862302014800445820165001M03000250011 205030000001360603301037200714424500374002000378001O0390017D00000000000477650391017C0000000000000000 039201800D000000000000000039301800C00000000@@00005290394017D00000000000477650395016C0000000000005290 396017D000000000004723604000100000000004040101000000000000402010000000000445800000016060136060601360 6g1644��6856862302014800445820165001M03000250011205030000001360603301037200714424530374002000378001O 0390017D00000000000188700391017C0000000000000000039201800D000000000000000039301800C00000000000020403 94017D00000000000188700395016C0000000000002040396017D00000000000186660400010000000000404010100000000 00004020100000000004458000000170601360606013606h1644�69507001050250011205030000001360603301030101600 0000000002205703060080000001800000018@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ @@@@@@@@@@@@@@@@@@@@
# 2  
Old 05-07-2012
Code:
tr -d "@" | wc -m

or

Code:
cat abinaryfile | sed -e s/@@//g;s/@$// | wc -m

# 3  
Old 05-07-2012
Binary files are just that - all possible acsii characters from 0 - 255

awk is not meant to read binary files, it is meant to read carriage control, human readable files. Because one version "works" and another does not is not unexpected because awk is NOT meant to work well with your data. sed is not guaranteed to work with that kind of data either.

There are tools that will work. What are you trying to do? count bytes in a file?
# 4  
Old 05-08-2012
Hi


Thanks for your responses..

Yes I am trying to count bytes in a binary files. What tools you are talking abut.. Would you mind giving me some info ..



Thanks again

Regards
Farrukh


Also I am not a programmer and its a problem of a programmer here and I am system engineer there so have no idea what that SED is doing in that command...



Hi..

This is the actual script I got and need to run

Code:
cat TT120T0 | sed -e "s/@@//g;s/@$//"  | ./awk ' begin {}    {        x = substr($0,1,length($0) - 4);        printf("%s", x);    }' >>  AB.TEST

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Using awk to add length of matching characters between field in file

The awk below produces the current output, which will add +1 to $3. However, I am trying to add the length of the matching characters between $5 and $6 to $3. I have tried using sub as a variable to store the length but am not able to do so correctly. I added comments to each line and the... (4 Replies)
Discussion started by: cmccabe
4 Replies

2. Shell Programming and Scripting

Delimit file based on character length using awk

Hi, I need help with one problem, I came across recently. I have one input file which I need to delimit based on character length. $ cat Input.txt 12345sda231453 asd760kjol62569 sdasw4g76gdf57 And, There is one comma separated file which mentions "start of the field" and "length... (6 Replies)
Discussion started by: Prathmesh
6 Replies

3. Shell Programming and Scripting

Awk: Matching Pattern From other file with length

Hi, I have input file whose first column needs(match.txt) to be matched with the first column of the input file with min & max length as defined in match.txt. But conditions are not matching. Please help on the changes in the code below as for multiple enteries in match.txt complete match.txt will... (3 Replies)
Discussion started by: siramitsharma
3 Replies

4. Shell Programming and Scripting

Splitting fixed length file using awk

Hi, I need to split a fixed length file of 160 characters based on value of a column. Example: ABC 456780001 DGDG SDFSF BCD 444440002 SSSS TTTTT ABC 777750003 HHHH UUUUU THH 888880001 FFFF LLLLLL HHH 999990002 GGGG OOOOO I need to split this file on basis of column from... (7 Replies)
Discussion started by: Neelkanth
7 Replies

5. Shell Programming and Scripting

Convert binary file to csv and then back to the binary format

Hello *nix specialists, Im working for a non profit organisation in Germany to transport DSL over WLAN to people in areas without no DSL. We are using Linksys WRT 54 router with DD-WRT firmware There are at the moment over 180 router running but we have to change some settings next time. So my... (7 Replies)
Discussion started by: digidax
7 Replies

6. Shell Programming and Scripting

file column length using awk

1|BANG|KINR|3456 2|BANG2222|KINR|347 3|BANG|KINR|347 4|BANG|KINR|347 5|BANG|KINR|347 6|BANG|KINR|347 awk -F"|" ' length($2)>=4||length($4)>=4 {print $0 >"above.txt";next}' test1.txt i want required output,because if the 2nd column more than 4 character or 4th column more than 4... (5 Replies)
Discussion started by: bmk
5 Replies

7. Shell Programming and Scripting

Flat file-make field length equal to header length

Hello Everyone, I am stuck with one issue while working on abstract flat file which i have to use as input and load data to table. Input Data- ------ ------------------------ ---- ----------------- WFI001 Xxxxxx Control Work Item A Number of Records ------ ------------------------... (5 Replies)
Discussion started by: sonali.s.more
5 Replies

8. Shell Programming and Scripting

how to fix the column length in a file using Awk Prog

Hi I use the following code to read the file and to fix the length of the column of the record in the file 'Sample.txt' ls Samp* | awk ' { a=$1 } END{ FS="n" for(i=1;i<=NR;i++) { while( getline < a ) { f1=$0; print("Line::",f1); f2=substr(f1,1,10) print("Field1::",f2);... (10 Replies)
Discussion started by: meva
10 Replies

9. Shell Programming and Scripting

Need awk script to compare 2 fields in fixed length file.

Need a script that manipulates a fixed length file that will compare 2 fields in that file and if they are equal write that line to a new file. i.e. If fields 87-93 = fields 119-125, then write the entire line to a new file. Do this for every line in the file. After we get only the fields... (1 Reply)
Discussion started by: Muga801
1 Replies

10. UNIX for Dummies Questions & Answers

Convert a tab delimited/variable length file to fixed length file

Hi, all. I need to convert a file tab delimited/variable length file in AIX to a fixed lenght file delimited by spaces. This is the input file: 10200002<tab>US$ COM<tab>16/12/2008<tab>2,3775<tab>2,3783 19300978<tab>EURO<tab>16/12/2008<tab>3,28523<tab>3,28657 And this is the expected... (2 Replies)
Discussion started by: Everton_Silveir
2 Replies
Login or Register to Ask a Question