Thank you for answering.

I should have post some code, that would have make things easier for sure.

I'm not trying to open 4096 files at the same time, I open the files, add a line, then close, and so on.

I decided to try with less directories, as in my previous example it would have make about 200 millions files. SO now I only have one directory with one hexadecimal letter. Then into each dir I have 4096 files, which is 65536 files at the end.

I also switched to perl as I thought it would be faster, but even with that the writing takes ages.

Here's some code :

Code:

#! /usr/bin/perl
use Digest::MD5 qw(md5_hex);
my $file = 'list';
open my $info, $file || die "Could not open $file: $!";
while( my $line = <$info>)  {   
    $line =~ s/\r|\n//g;
    my $md5 = md5_hex($line);
    my $add = substr $md5,0,1;
    my $add2 = substr $md5,1,3;
    my $add3 = substr $md5,4,3;
    $outfile = "md5hash__/".$add."/md5".$add2.".txt";
        open (FILE, ">> $outfile") || die "problem opening $outfile\n";
    print FILE $add3."\n".$line."\n";
    close(FILE);
}

Don't mind the code, it was just to try and see the speed.

Thank you for your help

EDIT : I should say, the file I'm opening for reading is about 25gB heavy. But that wasn't a problem when I only had 4096 files to write, so I thought it would be the same with 65536.