Sponsored Content
Top Forums Programming Reading a binary file in text or ASCII format Post 82877 by jim mcnamara on Wednesday 7th of September 2005 11:08:17 AM
Old 09-07-2005
This comes up all too often. Windows has text files. Unix does not. Unix is not Windows. In Unix a file is a file is a file. It's a bag of bytes. Period.

Because a standard C string uses ASCII zero (nul character) as the end of string, that data from files that contain nuls (in Windows these are binary files, in Unix they are just files) cannot be parsed as strings because the nuls confuse everything.

To the OP: try
Code:
od -c filename

to find out what is in the file. Then you will know if you can read it using standard string C calls like fgets(). Or if you will have to use fread().

After you've programmed for a while you tend to bypass fread and fgets, especially when you're dealing with large files that may contain interesting stuff.
This reads an entire file containing anything into a buffer:
Code:
#include <stddef.h>
#include <sys/types.h>
#include <unistd.h>
#include <fcntl.h>
#include <stdlib.h>
#include <stdio.h>
#include <sys/stat.h>
#include <errno.h>
#include <assert.h>

#define ck(x) \
if( (x) == (-1) ){ perror("");exit(EXIT_FAILURE);}

/* read a buffer from a file */
ssize_t readall(int fd, void *buf, size_t *bytes){
     ssize_t nread = 0, n=0;
     size_t nbyte = *bytes;

     do {
         if ((n = read(fd, &((char *)buf)[nread], nbyte - nread)) == -1) {
             if (errno == EINTR)
                 continue;
             else
                 return (-1);
         }
         if (n == 0)
             return nread;
         nread += n;
     } while (nread < nbyte);
     return nread;
}

/* read control */
void readfile(char *fname, char *buffer, size_t *size, mode_t *mode)
{
   int fd=0;   
   struct stat st;
   
   ck(fd=open(fname,O_RDONLY) );
   ck(fstat(fd,&st) );
   *size=st.st_size;   
   *mode=st.st_mode;
   buffer=calloc(1,*size+1);
   ck(readall(fd, buffer, size) );    
   ck(close(fd) );
}


int main(int argc, char *argv[])
{
   char *buffer=NULL;
   size_t size;
   mode_t mode;
 
   readfile(argv[1],buffer,&size,&mode);
   /* play with buffer here  */
   free(buffer);
   return 0;  
}

This User Gave Thanks to jim mcnamara For This Post:
 

10 More Discussions You Might Find Interesting

1. Programming

Binary to text format conversion

Hi, Please can any one tell me how to convert binary data to text format and vice versa. If possible give me the algorithm or C program. Thanks in advance Waiting for reply Bye:o (5 Replies)
Discussion started by: manjunath
5 Replies

2. Shell Programming and Scripting

ftp - determine ascii or binary file

Hello, How to i determine via ftp commandline if files on ftp server is ascii or binary files. Like every other comon windows ftp program does it automatically. regards Thomas (5 Replies)
Discussion started by: congo
5 Replies

3. UNIX for Dummies Questions & Answers

To convert multi format file to a readable ascii format

Hi I have a file which has ascii , binary, binary decimal coded,decimal & hexadecimal data with lot of special characters (like öƒ.ƒ.„İİ¡Š·œƒ.„İİ¡Š· ) in it. I want to standardize the file into ASCII format & later use that as source . Can any one suggest a way a logic to convert such... (5 Replies)
Discussion started by: gaur.deepti
5 Replies

4. Shell Programming and Scripting

Binary or ascii file

I want to verify the file is Binary or ascii file and accordingly I want to switch the program with ret code ie 0 or success and 1 for failure Can any one help me is this a correct syntex...i am getting error #!/bin/ksh $file filename if echo "ascii fie Found" else echo " binary... (6 Replies)
Discussion started by: u263066
6 Replies

5. Shell Programming and Scripting

Binary to ASCII(TEXT converion)

Hi all, I have been trying to convert a binary file to TEXT/ASCII file in linux/solaries.and commands like string are no good.Also i am not sure how the how output of the file looks like... I am attaching the binary file as zip since i couldnt load it in its original form in the post incase... (1 Reply)
Discussion started by: pistachio
1 Replies

6. Shell Programming and Scripting

Convert binary file to csv and then back to the binary format

Hello *nix specialists, Im working for a non profit organisation in Germany to transport DSL over WLAN to people in areas without no DSL. We are using Linksys WRT 54 router with DD-WRT firmware There are at the moment over 180 router running but we have to change some settings next time. So my... (7 Replies)
Discussion started by: digidax
7 Replies

7. Shell Programming and Scripting

Difference between ascii and binary file -

what is the diff between ascii and binary file. my understand is that.. ascii file - has only line feed - \n in it where as binary file - has both line feed and carriage return in it- \r\n is that correct. also,what is the ksh command to identify whether it is a binary or ascii... (1 Reply)
Discussion started by: billpeter3010
1 Replies

8. Shell Programming and Scripting

Reading the text file for particular format

Hi All, Need your help!! I have particular host file with below format: 172.34.45.67 Host1 Host2 134.45.56.67 Host3 Host4 Host5 I need shell script snippet which read this file and change the format of the file to the below format 172.34.45.67 Host1 172.34.45.67 ... (9 Replies)
Discussion started by: sharsour
9 Replies

9. Shell Programming and Scripting

Converting a binary file to ascii and vice versa?

Hi All, I have a binary file which is being exported from a Database, and i need to convert that to ASCII format. How can i achieve that? And this solution should work for any file which is given to us; means they will give different files from different tables. Thanks in advance. (8 Replies)
Discussion started by: baranisachin
8 Replies

10. Shell Programming and Scripting

Base32 decoding binary file to ascii

I need to convert a binary file which in encoded using base32 encoding technique and convert that into readible ASCII so that i can load the same in DB. is there any command to do the same. sample from the binary file lools like : ... (18 Replies)
Discussion started by: krk
18 Replies
FOPENCOOKIE(3)						     Linux Programmer's Manual						    FOPENCOOKIE(3)

NAME
fopencookie - opening a custom stream SYNOPSIS
#define _GNU_SOURCE #include <stdio.h> FILE *fopencookie(void *cookie, const char *mode, cookie_io_functions_t io_funcs); DESCRIPTION
The fopencookie() function allows the programmer to create a custom implementation for a standard I/O stream. This implementation can store the stream's data at a location of its own choosing; for example, fopencookie() is used to implement fmemopen(3), which provides a stream interface to data that is stored in a buffer in memory. In order to create a custom stream the programmer must: * Implement four "hook" functions that are used internally by the standard I/O library when performing I/O on the stream. * Define a "cookie" data type, a structure that provides bookkeeping information (e.g., where to store data) used by the aforementioned hook functions. The standard I/O package knows nothing about the contents of this cookie (thus it is typed as void * when passed to fopencookie()), but automatically supplies the cookie as the first argument when calling the hook functions. * Call fopencookie() to open a new stream and associate the cookie and hook functions with that stream. The fopencookie() function serves a purpose similar to fopen(3): it opens a new stream and returns a pointer to a FILE object that is used to operate on that stream. The cookie argument is a pointer to the caller's cookie structure that is to be associated with the new stream. This pointer is supplied as the first argument when the standard I/O library invokes any of the hook functions described below. The mode argument serves the same purpose as for fopen(3). The following modes are supported: r, w, a, r+, w+, and a+. See fopen(3) for details. The io_funcs argument is a structure that contains four fields pointing to the programmer-defined hook functions that are used to implement this stream. The structure is defined as follows struct cookie_io_functions_t { cookie_read_function_t *read; cookie_write_function_t *write; cookie_seek_function_t *seek; cookie_close_function_t *close; }; The four fields are as follows: cookie_read_function_t *read This function implements read operations for the stream. When called, it receives three arguments: ssize_t read(void *cookie, char *buf, size_t size); The buf and size arguments are, respectively, a buffer into which input data can be placed and the size of that buffer. As its function result, the read function should return the number of bytes copied into buf, 0 on end of file, or -1 on error. The read function should update the stream offset appropriately. If *read is a NULL pointer, then reads from the custom stream always return end of file. cookie_write_function_t *write This function implements write operations for the stream. When called, it receives three arguments: ssize_t write(void *cookie, const char *buf, size_t size); The buf and size arguments are, respectively, a buffer of data to be output to the stream and the size of that buffer. As its func- tion result, the write function should return the number of bytes copied from buf, or -1 on error. The write function should update the stream offset appropriately. If *write is a NULL pointer, then output to the stream is discarded. cookie_seek_function_t *seek This function implements seek operations on the stream. When called, it receives three arguments: int seek(void *cookie, off64_t *offset, int whence); The *offset argument specifies the new file offset depending on which of the following three values is supplied in whence: SEEK_SET The stream offset should be set *offset bytes from the start of the stream. SEEK_CUR *offset should be added to the current stream offset. SEEK_END The stream offset should be set to the size of the stream plus *offset. Before returning, the seek function should update *offset to indicate the new stream offset. As its function result, the seek function should return 0 on success, and -1 on error. If *seek is a NULL pointer, then it is not possible to perform seek operations on the stream. cookie_close_function_t *close This function closes the stream. The hook function can do things such as freeing buffers allocated for the stream. When called, it receives one argument: int close(void *cookie); The cookie argument is the cookie that the programmer supplied when calling fopencookie(). As its function result, the close function should return 0 on success, and EOF on error. If *close is NULL, then no special action is performed when the stream is closed. RETURN VALUE
On success fopencookie() returns a pointer to the new stream. On error, NULL is returned. CONFORMING TO
This function is a nonstandard GNU extension. EXAMPLE
The program below implements a custom stream whose functionality is similar (but not identical) to that available via fmemopen(3). It implements a stream whose data is stored in a memory buffer. The program writes its command-line arguments to the stream, and then seeks through the stream reading two out of every five characters and writing them to standard output. The following shell session demonstrates the use of the program: $ ./a.out 'hello world' /he/ / w/ /d/ Reached end of file Note that a more general version of the program below could be improved to more robustly handle various error situations (e.g., opening a stream with a cookie that already has an open stream; closing a stream that has already been closed). Program source #define _GNU_SOURCE #include <sys/types.h> #include <stdio.h> #include <stdlib.h> #include <unistd.h> #include <string.h> #define INIT_BUF_SIZE 4 struct memfile_cookie { char *buf; /* Dynamically sized buffer for data */ size_t allocated; /* Size of buf */ size_t endpos; /* Number of characters in buf */ off_t offset; /* Current file offset in buf */ }; ssize_t memfile_write(void *c, const char *buf, size_t size) { char *new_buff; struct memfile_cookie *cookie = c; /* Buffer too small? Keep doubling size until big enough */ while (size + cookie->offset > cookie->allocated) { new_buff = realloc(cookie->buf, cookie->allocated * 2); if (new_buff == NULL) { return -1; } else { cookie->allocated *= 2; cookie->buf = new_buff; } } memcpy(cookie->buf + cookie->offset, buf, size); cookie->offset += size; if (cookie->offset > cookie->endpos) cookie->endpos = cookie->offset; return size; } ssize_t memfile_read(void *c, char *buf, size_t size) { ssize_t xbytes; struct memfile_cookie *cookie = c; /* Fetch minimum of bytes requested and bytes available */ xbytes = size; if (cookie->offset + size > cookie->endpos) xbytes = cookie->endpos - cookie->offset; if (xbytes < 0) /* offset may be past endpos */ xbytes = 0; memcpy(buf, cookie->buf + cookie->offset, xbytes); cookie->offset += xbytes; return xbytes; } int memfile_seek(void *c, off64_t *offset, int whence) { off64_t new_offset; struct memfile_cookie *cookie = c; if (whence == SEEK_SET) new_offset = *offset; else if (whence == SEEK_END) new_offset = cookie->endpos + *offset; else if (whence == SEEK_CUR) new_offset = cookie->offset + *offset; else return -1; if (new_offset < 0) return -1; cookie->offset = new_offset; *offset = new_offset; return 0; } int memfile_close(void *c) { struct memfile_cookie *cookie = c; free(cookie->buf); cookie->allocated = 0; cookie->buf = NULL; return 0; } int main(int argc, char *argv[]) { cookie_io_functions_t memfile_func = { .read = memfile_read, .write = memfile_write, .seek = memfile_seek, .close = memfile_close }; FILE *fp; struct memfile_cookie mycookie; ssize_t nread; long p; int j; char buf[1000]; /* Set up the cookie before calling fopencookie() */ mycookie.buf = malloc(INIT_BUF_SIZE); if (mycookie.buf == NULL) { perror("malloc"); exit(EXIT_FAILURE); } mycookie.allocated = INIT_BUF_SIZE; mycookie.offset = 0; mycookie.endpos = 0; fp = fopencookie(&mycookie,"w+", memfile_func); if (fp == NULL) { perror("fopencookie"); exit(EXIT_FAILURE); } /* Write command-line arguments to our file */ for (j = 1; j < argc; j++) if (fputs(argv[j], fp) == EOF) { perror("fputs"); exit(EXIT_FAILURE); } /* Read two bytes out of every five, until EOF */ for (p = 0; ; p += 5) { if (fseek(fp, p, SEEK_SET) == -1) { perror("fseek"); exit(EXIT_FAILURE); } nread = fread(buf, 1, 2, fp); if (nread == -1) { perror("fread"); exit(EXIT_FAILURE); } if (nread == 0) { printf("Reached end of file "); break; } printf("/%.*s/ ", nread, buf); } exit(EXIT_SUCCESS); } SEE ALSO
fclose(3), fmemopen(3), fopen(3), fseek(3), feature_test_macros(7) COLOPHON
This page is part of release 3.25 of the Linux man-pages project. A description of the project, and information about reporting bugs, can be found at http://www.kernel.org/doc/man-pages/. Linux 2008-12-05 FOPENCOOKIE(3)
All times are GMT -4. The time now is 05:05 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy