Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

parsing a hex file

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting parsing a hex file
# 1  
Old 04-07-2009
parsing a hex file
Goodmorning,

I have a hex encoded file that needs to be parsed to remove some bytes and write the remaining ones to a new file. Here is the top of the file as shown by a hex editor:

Image

The first parse exercise is to remove bytes 0-87 such that the resulting file begins with byte 88 (in this file's case the first byte would contain 22 as shown).

The second exercise is to capture 512 bytes and then discard 16 bytes and then capture 512 and then discard 16 and so on until the end of the file.

The third and final exercise is to handle the bottom of the file where the last block after the discarded 16 bytes is less then 512 bytes but not a fixed number.

I have done scripts similar to this to handle text files, but I'm not sure even where to begin with a hex file working with byte positions. This is for a unix emulator (MKS Toolkit) running on a windows server.

Any assistance that can be provided to assist me in producing a script that does this would be greatly appreciated.

Thanks,
Last edited by philplasma; 04-07-2009 at 12:19 PM..
# 2  
Old 04-07-2009
here it is in C. Most of this can be translated into perl with the same function
names. Lemme know if that's necessary.

Write this into a file called my_reader.c

Code:
#include <stdio.h>
#include <stdlib.h>

int main( int nArgc, char **ppszArgv )
{
int nCounter;
FILE *Fin, *Fout;
char acBuffer[ 256 ];

/*--------------------------------------------------------------------*/
/* Argument check.                                                    */
/*--------------------------------------------------------------------*/
if ( nArgc != 3 ){
  (void)fprintf( stderr, "usage: %s file-in file-out \n",
    ppszArgv[0] );
  exit(9);
  }

/*--------------------------------------------------------------------*/
/* Open files in binary mode...                                       */
/*--------------------------------------------------------------------*/
Fin = fopen( ppszArgv[1], "rb" );
Fout = fopen( ppszArgv[2], "wb" );

/*--------------------------------------------------------------------*/
/* Skip over first 88 bytes.                                          */
/*--------------------------------------------------------------------*/
(void)fseek( Fin, 88, SEEK_SET );

nCounter = 0;

/*--------------------------------------------------------------------*/
/* Read the file...                                                   */
/*--------------------------------------------------------------------*/
while( 1 == fread( acBuffer, 16, 1, Fin ) && !(feof)( Fin )){

  nCounter += 16;

/*--------------------------------------------------------------------*/
/* Skip the record past 512 bytes...                                  */
/*--------------------------------------------------------------------*/
  if ( nCounter > 512 ){
    nCounter = 0;
    continue;
    }

/*--------------------------------------------------------------------*/
/* Write stuff.                                                       */
/*--------------------------------------------------------------------*/
  (void)fwrite( acBuffer, 16, 1, Fout );

  } /*** while reading the input file. . .. ***/

(void)fclose( Fin );
(void)fclose( Fout );

return 0;
}

compile with:

cc my_reader.c -o my_reader

Lemme know if you have to have it in perl.
# 3  
Old 04-07-2009
hmmm....

i did a little more testing and it only worked on files ending in exactly 16 byte segments.

i suppose we need to modify the counting mechanism.
# 4  
Old 04-07-2009
try this:

Code:
#include <stdio.h>
#include <stdlib.h>

int main( int nArgc, char **ppszArgv )
{
int nCounter;
FILE *Fin, *Fout;
char acBuffer[ 256 ];

/*--------------------------------------------------------------------*/
/* Argument check.                                                    */
/*--------------------------------------------------------------------*/
if ( nArgc != 3 ){
  (void)fprintf( stderr, "usage: %s file-in file-out \n",
    ppszArgv[0] );
  exit(9);
  }

/*--------------------------------------------------------------------*/
/* Open files in binary mode...                                       */
/*--------------------------------------------------------------------*/
Fin = fopen( ppszArgv[1], "rb" );
Fout = fopen( ppszArgv[2], "wb" );

/*--------------------------------------------------------------------*/
/* Skip over first 88 bytes.                                          */
/*--------------------------------------------------------------------*/
(void)fseek( Fin, 88, SEEK_SET );

nCounter = 0;

/*--------------------------------------------------------------------*/
/* Read the file...                                                   */
/*--------------------------------------------------------------------*/
while( ( 1 == fread( acBuffer, 1, 1, Fin )) && !(feof)( Fin )){

  nCounter++;

/*--------------------------------------------------------------------*/
/* Skip the record past 512 bytes...                                  */
/*--------------------------------------------------------------------*/
  if ( nCounter > 512 ){
    nCounter = 0;
    while( ( 1 == fread( acBuffer, 1, 1, Fin )) && !(feof)( Fin )){
      nCounter++;
      if ( nCounter > 16 ) break;
      }
    nCounter = 0;
    }

/*--------------------------------------------------------------------*/
/* Write stuff.                                                       */
/*--------------------------------------------------------------------*/
  (void)fwrite( acBuffer, 1, 1, Fout );

  } /*** while reading the input file. . .. ***/

(void)fclose( Fin );
(void)fclose( Fout );

return 0;
}

# 5  
Old 04-07-2009
Thanks, I'll give it a try and report back...
# 6  
Old 04-07-2009
Perl please?
The compiler in MKS Toolkit is not working, I don't think it is a problem with the code, here is the error message I am getting:
Code:
$ cc my_reader.c -o my_reader
Warning: Include' variable is undefined.
Compiler "cl" not found.

Since you offered a perl version, would you mind providing that version?

Thanks!!!!
# 7  
Old 04-07-2009
had a feeling. here it is in perl. not tooooo different:

Code:
#!perl


my $counter;
my $bytes_read;

#----------------------------------------------------------------------#
#  Argument check.                                                     #
#----------------------------------------------------------------------#
if ( scalar( @ARGV ) != 2 ){
  print "usage: $0 file-in file-out \n";
  exit(9);
  }

#----------------------------------------------------------------------#
#  Open files in binary mode...                                        #
#----------------------------------------------------------------------#
unless ( sysopen FIN, $ARGV[0], O_RDONLY ){
  print "cannot read file: $ARGV[0] \n";
  exit(9);
  }

unless ( sysopen FOUT, $ARGV[1], O_WRONLY ){
  print "cannot write to file: $ARGV[1] \n";
  exit(9);
  }

#----------------------------------------------------------------------#
#  Skip over first 88 bytes.                                           #
#----------------------------------------------------------------------#
sysseek FIN, 88, SEEK_SET;

$counter = 0;

#----------------------------------------------------------------------#
#  Read the file...                                                    #
#----------------------------------------------------------------------#
while ( $bytes_read = sysread FIN, $line, 1 ){

  $counter++;

#----------------------------------------------------------------------#
#  Skip the record past 512 bytes...                                   #
#----------------------------------------------------------------------#
  if ( $counter > 512 ){
    $counter = 0;
    while ( $bytes_read = sysread FIN, $line, 1 ){
      $counter++;
      last if ( $counter > 16 );
      }
    $counter = 0;
    }

#----------------------------------------------------------------------#
#  Write stuff.                                                        #
#----------------------------------------------------------------------#
  syswrite FOUT, $line, 1;

  } #-** while reading the input file. . .. **-#

close FIN;
close FOUT;

exit(0);

Last edited by quirkasaurus; 04-07-2009 at 03:18 PM.. Reason: error messages....
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Find and delete a certain HEX and its following value in a file

Hello there, I've been trying to do this half of the day and it's like I haven't come a single step further, so I hope you guys can help me with my problem: I have a text file that contains strings that should not be there and which I want to delete automatically from the command line. The... (4 Replies)
Discussion started by: surfi
4 Replies

2. Shell Programming and Scripting

Convert Binary File To Hex In Linux

dHi, I have the attached file(actual file can be extracted post unzipping it) & i am trying to use the following code for coversion to hex format. Starting hex value is 84 which is start of the record & termination is done using 00 00 followed by 84(hex) which i can see in the dump clearly using... (14 Replies)
Discussion started by: siramitsharma
14 Replies

3. HP-UX

Replacing Hex Characters In A File Using awk?

Hi guys, First off, i'm a complete noob to UNIX and LINUX so apologies if I don't understand the basics! I have a file which contains a hex value of '0D' at the end of each line when I look at it in a hex viewer. I need to change it so it contains a hex value of '0D0A0A' I thought... (10 Replies)
Discussion started by: AndyBSG
10 Replies

4. Shell Programming and Scripting

Compare Hex Value from CSV File

I have a one CSV File Contain Hex Value here is a sample file 6300, 0x0, 0x60d0242c6, , 0x728e5806, unnamedImageEntryPoint_0x728e5806, 0x728e$ 6300, 0x0, 0x60d024c52, , 0x728e8cb7, unnamedImageEntryPoint_0x728e8cb7, 0x728e$ 6300, 0x0, 0x60d025638, , 0x728e82da,... (2 Replies)
Discussion started by: rakesh_arxmind
2 Replies

5. Shell Programming and Scripting

Perl: Parse Hex file into fields

Hi, I want to split/parse certain bits of the hex data into another field. Example: Input data is Word1: 4f72abfd Output: Parse bits (5 to 0) into field word1data1=0x00cd=205 decimal Parse bits (7 to 6) into field word1data2=0x000c=12 decimal etc. Word2: efff3d02 Parse bits (13 to... (1 Reply)
Discussion started by: morrbie
1 Replies

6. Programming

What is the difference between ios::hex and std::hex?

Hi, Is there really a difference between these two, std::hex and ios::hex?? I stumbled upon reading a line, "std::ios::hex is a bitmask (8 on gcc) and works with setf(). std::hex is the operator". Is this true? Thanks (0 Replies)
Discussion started by: royalibrahim
0 Replies

7. Programming

After converting the hexstr to Hex and storing the Hex in a char*

Hi All, My main intension of is to convert the Hexstring stored in a char* into hex and then prefixing it with "0x" and suffix it with ',' This has to be done for all the hexstring char* is NULL. Store the result prefixed with "0x" and suffixed with ',' in another char* and pass it to... (1 Reply)
Discussion started by: rvan
1 Replies

8. Shell Programming and Scripting

Parsing of file for Report Generation (String parsing and splitting)

Hey guys, I have this file generated by me... i want to create some HTML output from it. The problem is that i am really confused about how do I go about reading the file. The file is in the following format: TID1 Name1 ATime=xx AResult=yyy AExpected=yyy BTime=xx BResult=yyy... (8 Replies)
Discussion started by: umar.shaikh
8 Replies

9. UNIX for Dummies Questions & Answers

hex value in a file + perl

Am not able to display the corresponding character for the hex value using the format specifier into a file Could you please help me with that >cat other a|\xc2\xbo >cat write.pl #! /opt/third-party/bin/perl open(FILE2, "< other") || die "Unable to open file other\n"; while (... (7 Replies)
Discussion started by: matrixmadhan
7 Replies

10. HP-UX

Hex characters of ascii file

Hi, Whats the command or how do you display the hexadecimal characters of an ascii file. thanks Bud (2 Replies)
Discussion started by: budrito
2 Replies
Login or Register to Ask a Question