Sponsored Content
Top Forums UNIX for Advanced & Expert Users Conversion from ASCII to binary for physical simulation code in C/C++ Post 302507396 by Cybertib on Wednesday 23rd of March 2011 05:59:37 PM
Old 03-23-2011
Conversion from ASCII to binary for physical simulation code in C/C++

Good evening, everybody

A good math friend told me that it would be possible to shrink the size of the numerical datas I produce with a physical simulation code I programmed for my PhD.

It usually writes at least 100 GB to complete the simulation, and it seems that it is too high. There are some quotas to respect, and I have been told that it would possible to use "binary" datas instead of ASCII datas.

Here are the inputs of the problem.
- I calculate and produce the datas using a C/C++ simulation code.
- I make post treatments using bash and awk.
- I make plots using GNUplot 4.4 [splot works great with pm3d now Smilie Have a look to "Not so frequently asked question" website to be aware of GNUplot possibilities]

I found how to use binary mode in Gawk and GNUplot, but the main point is missing: which conversion should I make to decrease the volume of the datas in a loseless way ? They are all numerical datas, then I would convert double precision number coded in ASCII (visible with "more" command) into "binary" (which I assume to be a language abuse, because converting datafile into binary using "od" command just multiply the size of the initial file... :-( ).

Example. Each line of my generated main data file is:
3000 -3.9e-13 -4.24661e-05 0 299.964 300 1.50018e+16 1.50005e+16 0 00 1 0 0

What conversion do you recommand to optimize space needed ?

By writing this post, I feel that I should convert double precision numbers coded with characters in a ascii file into double precision datas coded with numbers only. How can I do that? "od" command is sufficient ?

Glad of any help,

Cheers from France,
Thibault

Last edited by Cybertib; 03-23-2011 at 07:18 PM.. Reason: added example
 

10 More Discussions You Might Find Interesting

1. UNIX for Advanced & Expert Users

Convert ASCII to BINARY

Here is what I did . . . . I FTP'd several *.pdf files from a web site to a UNIX server, and did not set the transfer mode to BIN, now Adobe thinks that the documents are corrupted. Is there a way to convert the *.pdf files to Binary so that Adobe can open them again. I would just re-download... (2 Replies)
Discussion started by: pc9456
2 Replies

2. Shell Programming and Scripting

ascii conversion

after converting my ebcidic file to ascii i get the following output 2097152+0 records in 1797345+1 records out Why is there a difference in number of records. Is the converson chopping off any records. All i am doing is just a conversion using the following script dd if=xaa cbs=152 ... (0 Replies)
Discussion started by: rintingtong
0 Replies

3. SCO

ascii to binary conversion in sco 5.0.5

Here is what I did . . . . I FTP'd several *.dbf zipped files from a SCO 5.0.5 server to winXP machine, and did not set the transfer mode to BIN, now when i was uncompressing these files in SCO 5.0.5 , it was giving "Bad Decode Table error. Is there a way to convert the *.dbf.Z files to Binary so... (1 Reply)
Discussion started by: sameek1211
1 Replies

4. Shell Programming and Scripting

binary to ascii

Hi, Is there a way to convert the binary file to ascii . the binary file is pipe delimited. from source the file(pipe delimited) is ftped to mainframe and from mainframe it is ftped to the unix box using binary format. Is there a way to change it back to ascii and view it? Thanks! (3 Replies)
Discussion started by: dnat
3 Replies

5. UNIX for Dummies Questions & Answers

Ascii or Binary?

Hello all, I am working with ftp servers in unix, and always I have to get and put files but I don't know exactly if I have to get or put them as an ascii or binary. Some files that I use are: .txt, .sav, .fmb, .pct, .sh, .ksh, .dat, .log. Somebody can tell me what is the difference between... (2 Replies)
Discussion started by: Geller
2 Replies

6. Shell Programming and Scripting

binary to ascii conversion

Hi, I have got a library file, created by compiling C code. The file information with "file" command, gives it a "application/x-archive" type file. I want to extract the release string of my software from this file, so that i can know which version of C files were used to create the lib. Can... (3 Replies)
Discussion started by: atulmt
3 Replies

7. Solaris

EBCDIC to ASCII Binary conversion issue on Solaris i-series Unix

Hi All, I am facing EBCDIC to ASCII Binary conversion on Solaris i-series Unix system. However this is working fine on Solaris Sparc Unix system. Input file having EBCDIC format does not work on Solaris i-series Unix system. Could you please tell me, what will be the root cause for same? (14 Replies)
Discussion started by: amodkavi
14 Replies

8. Shell Programming and Scripting

Bash - binary data to ascii code

Hello, With bash-script (ubunto server) I'm trying to read a binary file and, for each character, give back its ascii code (including extended ascii). For example: HEX => ASCII => PRINT f5 => 245 => õ 50 => 80 => P To load the binary file into a variable I tried in this way: ... (2 Replies)
Discussion started by: math4
2 Replies

9. Solaris

ASN Binary to ASCII

Dears, I need help to convert the binary file into ASCII format. Actually we have CDRs which is generated by telecom switch at this is in ASN1 format or binary format, I need to convert those binary formatted file into ASCII format using Perl, or shell scripting. Is there any way to solve... (3 Replies)
Discussion started by: PRINCESS_RORO
3 Replies

10. Shell Programming and Scripting

File conversion from Binary to ASCII though UNIX command

Hi All , I have a mainframe file which contains the data in EBCDIC format.I have downloaded this file from mainframe to windows in binary format(unreadable raw data).Now I want convert this file to ASCII format(readable format data) through Unix command.I have tried iconv but that is not working... (2 Replies)
Discussion started by: STCET22
2 Replies
datas(3U)						    InterViews Reference Manual 						 datas(3U)

NAME
ColorData, GSData, MobilityData, MoveData, VoidData - Data subclasses for storing common undo information SYNOPSIS
#include <Unidraw/Commands/datas.h> DESCRIPTION
The Unidraw library predefines several Data subclasses for storing common undo information: ColorData stores two PSColors (usually repre- senting foreground and background colors), while GSData stores a complete set of graphics state in a FullGraphic object. MobilityData stores the mobility and graphics state information needed to undo a change in mobility. MoveData stores a horizontal and vertical distance. VoidData stores a void pointer as a useful alternative to deriving trivially from Data just to store one word of information. COLORDATA PUBLIC OPERATIONS
ColorData(PSColor*, PSColor*) Create a ColorData instance, supplying the PSColor objects it stores. GSDATA PUBLIC OPERATIONS
GSData(Graphic*) Create a GSData instance, supplying the Graphic object from which to copy a complete set of graphics state. The GSData constructor copies this information into its _gs member, from whence it can be retrieved later. MOBILITYDATA PUBLIC OPERATIONS
MobilityData(Mobility, Graphic*) Create a MobilityData instance, supplying the mobility and a Graphic object from which to copy a complete set of graphics state. The MobilityData constructor copies the graphic into its _gs member, from whence it can be retrieved later. MOVEDATA PUBLIC OPERATIONS
MoveData(float, float) Create a MoveData instance, supplying two floating point numbers that normally represent horizontal and vertical movement. VOIDDATA PUBLIC OPERATIONS
VoidData(void*) Create a VoidData instance, supplying a pointer to an object of unknown type. SEE ALSO
Data(3U), Graphic(3U), globals(3U) Unidraw 24 January 1991 datas(3U)
All times are GMT -4. The time now is 08:14 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy