Like I said: read them in with fgets and sscanf, write them back out as binary with fwrite(). To make an example that works I'll need to see what your data looks like. No doubt: It does the precise opposite, dumping binary files in a variety of ASCII forms. Could be. Also means that if you make a mistake in your C program, you've !$@^ed up 100 gigs of data faster than you ever could before.

It's just occurred to me that doing it in double-precision is pointless anyway; you've already processed it with single-precision awk before this point.

The gnuplot "matrix" format is out. It stores everything as floating point numbers, even the number of rows, which means you get 8 million rows max before it starts expressing the number of rows in exponential notation and ending up with slightly too few or too many. Their documentation in this area seems especially impenetrable. I'm working on something.

---------- Post updated at 06:17 PM ---------- Previous update was at 06:07 PM ----------

Code:

// Generate a raw binary file for gnuplot to work with.
// compile with -lm

#include <stdio.h>
#include <math.h>

int main(void)
{
        int n, points=100;
        FILE *fp=fopen("sin.bin", "wb");

        for(n=0; n<points; n++)
        {
                float v[3]= { (2*3.14159*n)/(points-1) };
                v[1]=sin(v[0]);         v[2]=cos(v[0]);
                // writes three floats in a row.  x, sin(x), cos(x)
                fwrite(v, 3, sizeof(float), fp);
        }
        fclose(fp);
        return(0);
}

Code:

$ gcc graph.c -lm
$ ./a.out
$ ls -l sin.bin
-rw-r--r-- 1 monttyle monttyle 1200 Mar 23 18:15 sin.bin
$ gnuplot
> plot "sin.bin" binary format='%f%f%f' using 1:2;
(pops up a picture of a sine wave)
>

How to cram that into a surface plot or whatever I'm not sure but it's something to work from.

For doubles, use %lf.