Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

PROBLEM with another binary file

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting PROBLEM with another binary file
# 15  
Old 02-03-2015
This should strip out all characters not between ascii 32 and 127 inclusive.

Code:
tr -d '\000-\037\200-\377' < inputfile > outputfile

# 16  
Old 02-03-2015
How about using strings: -n 1 will detect a sequence of 1 or more printable characters:

Code:
$ printf "%s" $(strings -n 1 20150118_1920.txt ) | fold
00002F6D00002F5S00002F6800002F6J00002F6800002F5P00002F6HBREAK00000BZ4Generique_d
ebut_19_20100002F6LTR1_MANIF_MONTPELLIER300002F6MTR2_ST_LAURENT_LE_MINIER400002F
6NTR3_JEANE_MANSON500002F5ZS_DOUBLE_MANIF_MONTPELLIER700002F60S_DEBAT_AGORA_LA_P
AILLADE900002F5U%M_OFF_GREVE_CHAUFFEURS_ROUTIERS_0_22_1100002F6CS_OFF_USAP_0_23_
1300002F5X!M_OFF_FOOT_METZ_MONTPELLIER_0_33_1700002F6OCARTE_ST_LAURENT_LE_MINIER
1900002F61.ST_LAURENT_LE_MINIER_NETTOYAGE_RIVIERES_NIMES_2000002F21L_ECOFERME220
0002F6PCARTE_AUMONT_AUBRAC_48_2400002F69LRM_2_ATTRAZIC_ET_VIGROUX2500002F6ALRM_J
EANE_MANSON270000245KVoix_gene_fin_MIXEE_F3Regions_29BREAK00002F6K00002F5O00002F
5T00002F6IBREAK

This User Gave Thanks to Chubler_XL For This Post:
protocomm
# 17  
Old 02-03-2015
Hi Chubler_XL...

Your code works except it strips all (white)spaces...
Code:
Last login: Tue Feb  3 22:06:33 on ttys000
AMIGA:barrywalker~> cd Desktop/Code/Shell
AMIGA:barrywalker~/Desktop/Code/Shell> hexdump -C binary2
00000000  00 00 00 0a 0a 0a 09 09  09 20 20 20 00 0a 09 5c  |.........   ...\|
00000010  20 0a 00 5c 20 09 20 20  20 0a 0a 0a              | ..\ .   ...|
0000001c
AMIGA:barrywalker~/Desktop/Code/Shell> 
AMIGA:barrywalker~/Desktop/Code/Shell> printf "%s" $(strings -n 1 binary2 ) | fold
\\AMIGA:barrywalker~/Desktop/Code/Shell> 
AMIGA:barrywalker~/Desktop/Code/Shell> _

---------- Post updated at 10:36 PM ---------- Previous update was at 10:28 PM ----------

Hi Corona688...

Sadly your code does not work:-
It hits an "Illegal byte sequence" and crashes out...

Code:
Last login: Tue Feb  3 22:24:05 on ttys000
AMIGA:barrywalker~> cd Desktop/Code/Shell
AMIGA:barrywalker~/Desktop/Code/Shell> hexdump -C binary.bin
00000000  00 01 02 03 04 05 06 07  08 09 0a 0b 0c 0d 0e 0f  |................|
00000010  10 11 12 13 14 15 16 17  18 19 1a 1b 1c 1d 1e 1f  |................|
00000020  20 21 22 23 24 25 26 27  28 29 2a 2b 2c 2d 2e 2f  | !"#$%&'()*+,-./|
00000030  30 31 32 33 34 35 36 37  38 39 3a 3b 3c 3d 3e 3f  |0123456789:;<=>?|
00000040  40 41 42 43 44 45 46 47  48 49 4a 4b 4c 4d 4e 4f  |@ABCDEFGHIJKLMNO|
00000050  50 51 52 53 54 55 56 57  58 59 5a 5b 5c 5d 5e 5f  |PQRSTUVWXYZ[\]^_|
00000060  60 61 62 63 64 65 66 67  68 69 6a 6b 6c 6d 6e 6f  |`abcdefghijklmno|
00000070  70 71 72 73 74 75 76 77  78 79 7a 7b 7c 7d 7e 7f  |pqrstuvwxyz{|}~.|
00000080  80 81 82 83 84 85 86 87  88 89 8a 8b 8c 8d 8e 8f  |................|
00000090  90 91 92 93 94 95 96 97  98 99 9a 9b 9c 9d 9e 9f  |................|
000000a0  a0 a1 a2 a3 a4 a5 a6 a7  a8 a9 aa ab ac ad ae af  |................|
000000b0  b0 b1 b2 b3 b4 b5 b6 b7  b8 b9 ba bb bc bd be bf  |................|
000000c0  c0 c1 c2 c3 c4 c5 c6 c7  c8 c9 ca cb cc cd ce cf  |................|
000000d0  d0 d1 d2 d3 d4 d5 d6 d7  d8 d9 da db dc dd de df  |................|
000000e0  e0 e1 e2 e3 e4 e5 e6 e7  e8 e9 ea eb ec ed ee ef  |................|
000000f0  f0 f1 f2 f3 f4 f5 f6 f7  f8 f9 fa fb fc fd fe ff  |................|
00000100  00 01 02 03 04 05 06 07  08 09 0a 0b 0c 0d 0e 0f  |................|
00000110  10 11 12 13 14 15 16 17  18 19 1a 1b 1c 1d 1e 1f  |................|
00000120  20 21 22 23 24 25 26 27  28 29 2a 2b 2c 2d 2e 2f  | !"#$%&'()*+,-./|
00000130  30 31 32 33 34 35 36 37  38 39 3a 3b 3c 3d 3e 3f  |0123456789:;<=>?|
00000140  40 41 42 43 44 45 46 47  48 49 4a 4b 4c 4d 4e 4f  |@ABCDEFGHIJKLMNO|
00000150  50 51 52 53 54 55 56 57  58 59 5a 5b 5c 5d 5e 5f  |PQRSTUVWXYZ[\]^_|
00000160  60 61 62 63 64 65 66 67  68 69 6a 6b 6c 6d 6e 6f  |`abcdefghijklmno|
00000170  70 71 72 73 74 75 76 77  78 79 7a 7b 7c 7d 7e 7f  |pqrstuvwxyz{|}~.|
00000180  80 81 82 83 84 85 86 87  88 89 8a 8b 8c 8d 8e 8f  |................|
00000190  90 91 92 93 94 95 96 97  98 99 9a 9b 9c 9d 9e 9f  |................|
000001a0  a0 a1 a2 a3 a4 a5 a6 a7  a8 a9 aa ab ac ad ae af  |................|
000001b0  b0 b1 b2 b3 b4 b5 b6 b7  b8 b9 ba bb bc bd be bf  |................|
000001c0  c0 c1 c2 c3 c4 c5 c6 c7  c8 c9 ca cb cc cd ce cf  |................|
000001d0  d0 d1 d2 d3 d4 d5 d6 d7  d8 d9 da db dc dd de df  |................|
000001e0  e0 e1 e2 e3 e4 e5 e6 e7  e8 e9 ea eb ec ed ee ef  |................|
000001f0  f0 f1 f2 f3 f4 f5 f6 f7  f8 f9 fa fb fc fd fe ff  |................|
00000200
AMIGA:barrywalker~/Desktop/Code/Shell> 
AMIGA:barrywalker~/Desktop/Code/Shell> tr -d '\000-\037\200-\377' < binary.bin > text.txt
tr: Illegal byte sequence
AMIGA:barrywalker~/Desktop/Code/Shell> cat text.txt
 !"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~AMIGA:barrywalker~/Desktop/Code/Shell> _

Note that the second string is missing...
This User Gave Thanks to wisecracker For This Post:
Chubler_XL
# 18  
Old 02-03-2015
Striping whitespace may be considered a feature, however here is another method conserving whitespace (except newlines):

Code:
strings -n 1 filename | LC_CTYPE=C tr -d '\n' | fold

Fold above is optional, I just like 80 chars per line for readability

And try this for Corona688's solution:

Code:
LC_CTYPE=C tr -d '\000-\037\177-\377' < binary.bin > text.txt

Last edited by Chubler_XL; 02-03-2015 at 07:17 PM..
# 19  
Old 02-03-2015
Hi Chubler_XL...

Almost but 2 0x0C characters have been added...
Code:
Last login: Tue Feb  3 23:01:00 on ttys000
AMIGA:barrywalker~> cd Desktop/Code/Shell
AMIGA:barrywalker~/Desktop/Code/Shell> strings -n 1 binary.bin | tr -d '\n' > text.txt
AMIGA:barrywalker~/Desktop/Code/Shell> cat text.txt

 !"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~
                !"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~AMIGA:barrywalker~/Desktop/Code/Shell> 
AMIGA:barrywalker~/Desktop/Code/Shell> hexdump -C text.txt
00000000  0c 20 21 22 23 24 25 26  27 28 29 2a 2b 2c 2d 2e  |. !"#$%&'()*+,-.|
00000010  2f 30 31 32 33 34 35 36  37 38 39 3a 3b 3c 3d 3e  |/0123456789:;<=>|
00000020  3f 40 41 42 43 44 45 46  47 48 49 4a 4b 4c 4d 4e  |?@ABCDEFGHIJKLMN|
00000030  4f 50 51 52 53 54 55 56  57 58 59 5a 5b 5c 5d 5e  |OPQRSTUVWXYZ[\]^|
00000040  5f 60 61 62 63 64 65 66  67 68 69 6a 6b 6c 6d 6e  |_`abcdefghijklmn|
00000050  6f 70 71 72 73 74 75 76  77 78 79 7a 7b 7c 7d 7e  |opqrstuvwxyz{|}~|
00000060  0c 20 21 22 23 24 25 26  27 28 29 2a 2b 2c 2d 2e  |. !"#$%&'()*+,-.|
00000070  2f 30 31 32 33 34 35 36  37 38 39 3a 3b 3c 3d 3e  |/0123456789:;<=>|
00000080  3f 40 41 42 43 44 45 46  47 48 49 4a 4b 4c 4d 4e  |?@ABCDEFGHIJKLMN|
00000090  4f 50 51 52 53 54 55 56  57 58 59 5a 5b 5c 5d 5e  |OPQRSTUVWXYZ[\]^|
000000a0  5f 60 61 62 63 64 65 66  67 68 69 6a 6b 6c 6d 6e  |_`abcdefghijklmn|
000000b0  6f 70 71 72 73 74 75 76  77 78 79 7a 7b 7c 7d 7e  |opqrstuvwxyz{|}~|
000000c0
AMIGA:barrywalker~/Desktop/Code/Shell> _

It's not as easy as it looks...

---------- Post updated at 11:17 PM ---------- Previous update was at 11:11 PM ----------

Hi Chubler_XL...

Your modification to Corona688's code works on binary.bin now... ;o)

<thumbs up>

Now to see if I can crash it... ;oD

HTH.
# 20  
Old 02-03-2015
Strange about the 0x0C character on my system I see a 0x09 in that position which is a TAB character, and technically as whitespace it should be kept:

Code:
$ xxd text.txt
0000000: 0920 2122 2324 2526 2728 292a 2b2c 2d2e  . !"#$%&'()*+,-.
0000010: 2f30 3132 3334 3536 3738 393a 3b3c 3d3e  /0123456789:;<=>
0000020: 3f40 4142 4344 4546 4748 494a 4b4c 4d4e  ?@ABCDEFGHIJKLMN
0000030: 4f50 5152 5354 5556 5758 595a 5b5c 5d5e  OPQRSTUVWXYZ[\]^
0000040: 5f60 6162 6364 6566 6768 696a 6b6c 6d6e  _`abcdefghijklmn
0000050: 6f70 7172 7374 7576 7778 797a 7b7c 7d7e  opqrstuvwxyz{|}~
0000060: 0920 2122 2324 2526 2728 292a 2b2c 2d2e  . !"#$%&'()*+,-.
0000070: 2f30 3132 3334 3536 3738 393a 3b3c 3d3e  /0123456789:;<=>
0000080: 3f40 4142 4344 4546 4748 494a 4b4c 4d4e  ?@ABCDEFGHIJKLMN
0000090: 4f50 5152 5354 5556 5758 595a 5b5c 5d5e  OPQRSTUVWXYZ[\]^
00000a0: 5f60 6162 6364 6566 6768 696a 6b6c 6d6e  _`abcdefghijklmn
00000b0: 6f70 7172 7374 7576 7778 797a 7b7c 7d7e  opqrstuvwxyz{|}~

# 21  
Old 02-04-2015
Hi Chubler_XL...
(OSX 10.7.5, default bash terminal.)
Good morning all...
Using your newer code it still has a FF, (Form Feed), character. Effectively this is a page of whitespaces. ;o)
Code:
Last login: Wed Feb  4 07:34:09 on ttys000
AMIGA:barrywalker~> cd Desktop/Code/Shell
AMIGA:barrywalker~/Desktop/Code/Shell> strings -n 1 binary.bin | LC_CTYPE=C tr -d '\n' > text.txt
AMIGA:barrywalker~/Desktop/Code/Shell> cat text.txt
 !"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~
                !"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~AMIGA:barrywalker~/Desktop/Code/Shell> 
AMIGA:barrywalker~/Desktop/Code/Shell> hexdump -C text.txt
00000000  0c 20 21 22 23 24 25 26  27 28 29 2a 2b 2c 2d 2e  |. !"#$%&'()*+,-.|
00000010  2f 30 31 32 33 34 35 36  37 38 39 3a 3b 3c 3d 3e  |/0123456789:;<=>|
00000020  3f 40 41 42 43 44 45 46  47 48 49 4a 4b 4c 4d 4e  |?@ABCDEFGHIJKLMN|
00000030  4f 50 51 52 53 54 55 56  57 58 59 5a 5b 5c 5d 5e  |OPQRSTUVWXYZ[\]^|
00000040  5f 60 61 62 63 64 65 66  67 68 69 6a 6b 6c 6d 6e  |_`abcdefghijklmn|
00000050  6f 70 71 72 73 74 75 76  77 78 79 7a 7b 7c 7d 7e  |opqrstuvwxyz{|}~|
00000060  0c 20 21 22 23 24 25 26  27 28 29 2a 2b 2c 2d 2e  |. !"#$%&'()*+,-.|
00000070  2f 30 31 32 33 34 35 36  37 38 39 3a 3b 3c 3d 3e  |/0123456789:;<=>|
00000080  3f 40 41 42 43 44 45 46  47 48 49 4a 4b 4c 4d 4e  |?@ABCDEFGHIJKLMN|
00000090  4f 50 51 52 53 54 55 56  57 58 59 5a 5b 5c 5d 5e  |OPQRSTUVWXYZ[\]^|
000000a0  5f 60 61 62 63 64 65 66  67 68 69 6a 6b 6c 6d 6e  |_`abcdefghijklmn|
000000b0  6f 70 71 72 73 74 75 76  77 78 79 7a 7b 7c 7d 7e  |opqrstuvwxyz{|}~|
000000c0
AMIGA:barrywalker~/Desktop/Code/Shell> _
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Web Development

Problem in printing binary tree using php and mysql

Database Structure Root Table ID Root_ Node Level 1 A 0 2 B 1 3 C 1 Child Table ID Left_Node Right_Node Root_Node Root_ID 1 B C A 1 ... (1 Reply)
Discussion started by: Deepak Tiwari
1 Replies

2. Shell Programming and Scripting

Problem with binary file

I have a binary file with all content is text, i want convert this binary to a regular file because i want to parse it. Thx. ---------- Post updated at 05:01 AM ---------- Previous update was at 04:34 AM ---------- I have found the solution cat file | tr -d '\0' (8 Replies)
Discussion started by: protocomm
8 Replies

3. Shell Programming and Scripting

Convert binary file to csv and then back to the binary format

Hello *nix specialists, Im working for a non profit organisation in Germany to transport DSL over WLAN to people in areas without no DSL. We are using Linksys WRT 54 router with DD-WRT firmware There are at the moment over 180 router running but we have to change some settings next time. So my... (7 Replies)
Discussion started by: digidax
7 Replies

4. UNIX for Dummies Questions & Answers

[AIX] Binary file warning for text file.

Hello guys, We had to move from a DC to another, and we are now facing an "issue" with some text files. Looks like that some of our log files are set as binary: file TuxConnectorURA.20121012 TuxConnectorURA.20121012: data or International Language text less TuxConnectorURA.20121012... (2 Replies)
Discussion started by: EnioMarques
2 Replies

5. Shell Programming and Scripting

Output redirection of c binary file to a file in shell script is failing

I am struck up with a problem and that is with output redirection. I used all the ways for the redirection of the output of c binary to a file, still it is failing. Here are the different ways which I have used: ./a.out | tee -a /root/tmp.txt 2>&1 ./a.out | tee -a /root/tmp.txt 1>&1 ./a.out |... (2 Replies)
Discussion started by: Maya29988
2 Replies

6. UNIX for Dummies Questions & Answers

Pipe binary file matches grep results to file

I am using grep to match a pattern, but the output is strange. $ grep -r -o "pattern" * Gives me: Binary file foo1 matches Binary file foo2 matches Binary file foo3 matches To find the lines before/after, I then have to use the following on each file: $ strings foo1 | grep -A1 -B1... (0 Replies)
Discussion started by: chipperuga
0 Replies

7. SCO

SCO unix binary compatibility problem

Hi I am looking for sco xenix binary compatibility utility "cvtomf",(convert omf object to COFF object) Would you please help me ? tnx (3 Replies)
Discussion started by: javad1_maroofi
3 Replies

8. Solaris

problem when decoding a binary file

I tried to decode a binary script using the command 'uudecode'. but it is giving error as 'No begining line'. 'uudecode -o <outfile name> <binary file>' Please help me in resolving this. (4 Replies)
Discussion started by: vamshikrishnab
4 Replies

9. Filesystems, Disks and Memory

UNIX problem concerning Microsoft Office Binary

I just started using Terminal on my Mac this summer, and right now I am having problems making a shell script. This shell should run the "Microsoft Setup Assistant" once I run the Automator Script. Alas, I still have problems. Here is the offending scriptlet: Sudo Tcsh root# cd... (2 Replies)
Discussion started by: chrisfrap
2 Replies

10. Solaris

compiled binary file gives "cannot execute binary file"

Hi, I have two Solaris machines. 1. SunOS X 5.8 Generic_108528-29 sun4u sparc SUNW,Sun-Blade-1500 2. SunOS Y 5.8 Generic_108528-13 sun4u sparc SUNW,Ultra-60 I am trying to buiild a project on both these machines. The Binary output file compiled on machine 2 runs on both the machines. Where... (0 Replies)
Discussion started by: scgupta
0 Replies
Login or Register to Ask a Question