Search Results

BASH: because it's the most universal. I like to use and stay familiar with the tools I'm most likely to run into on my next assignment. That pretty much means BASH or Window's command.

You should probably find out what the + is referring to before you delete it. That may remind you there was a reason for it. will do it. Look at the man page for explanations because the output is...

Quite right. But I've found there's still a sweet spot, and I'm gonna use it. I didn't know seeks had gotten that fast. Interesting. But somebody please tell me what AU is.

I have finished my...

I am. What would not be fun is re-writing the merge section of GNU sort to make special use of an SSD, and verifying that it is correct and robust. I don't have a spare SSD anyway, so it wouldn't...

I may be dense, but I don't see the point in such suggestions. My files are huge, much bigger than any SSD I can afford. No matter how you cut it, the file won't fit in RAM or my SSD drive, and...

I'm not going to load a database, because the results of the sort will be used just once, and as a practical matter may be passed in a pipe without ever hitting the filesystem. For testing, there's...

That makes no sense to me. If I tell it to make 1GB temporaries, my 13GB test file will make 13 of them and probably merge just once. If I tell it to make 1MB temporaries, it will make 13,000 of...

I'm testing with my idea of a small file: 13 GB. My targets are more like TB-sized. These are the cases that motivate optimizing GNU sort. For modest-sized files, I wouldn't bother.


Size and...

Nice to know. I tried e4defrag and it showed a fragmentation score of 0 on all directories.

I'm still a bit new to this, even after peeking at the source code. But it seems to me that there are...

My SSD is nowhere near big enough for /tmp, so that is assigned to a dedicated 2TB SATA drive. It would be large enough for these tests on a 14GB file, but my real workload has TB-sized inputs, so I...

I'm pretty sure that the buffer-size parameter controls only pass 1, which is where actual sorting occurs. It controls the size of the first set of temporary files. All subsequent passes are...

Thanks to all for your comments. I was asking for ways to tune UNIX sort, because while I know how, I'm unwilling to rewrite it for this project -- I'm likely to be mired in bugs for too long.

I...

Very interesting.

I had completely missed the --parallel option in reading the man page. I use this machine for everything, so I'm not sure I'll always want to swamp the cores, but I'll...

Indeed I did. In fact, I thought I had edited that typo, but clearly I didn't.

Being the impatient sort, I tried a few things.

It appears that, on my machine at least, the default --buffer-size is 4 GB, which yields 1.8GB temporary files. Don't ask me why the roughly...

All I've tried is a plain old call to sort(1). I'm unhappy with the time it takes.

It doesn't make sense to post the inputs or outputs, as they are over 300 GB in size, and are much like the...

I'm doing a hobby project that has me sorting huge files with sort of monotonous keys. It's very slow -- the current file is about 300 GB and has been sorting for a day. I know that sort has this...