Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Performance Hit With Many Files
Special Forums Hardware Filesystems, Disks and Memory Performance Hit With Many Files Post 302293405 by cyner on Tuesday 3rd of March 2009 04:40:02 AM
Old 03-03-2009
Quote:
Originally Posted by otheus
You do know one way to store large amounts of files is to create a directory hierarchy that is keyed on the filenames themselves? So files named "ergo1802.txt" might be stored in:
Code:
    data/er/go/18/ergo1802.txt

Yes, that would probably be the smartest design if the file names are randomized or evenly distributed. I know the users directories at Sourceforge are organized like that (something like /home/u/us/username). Unfortunately, I've inherited a legacy system, and first need to determine if it's worth the trouble to change the design.

Maybe I should create a JFS partition on a virtual Linux machine and do some benchmarking. I don't have access to an idle AIX machine comparable to the one used in production. Of course, such a test would differ from "reality" in some ways: different OS, CPU architecture and storage solutions (local IDE drive compared to SAN).
 

9 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Anyone else see a performance hit from ext3

I reinstalled my Linux box with RedHat 7.2 and used the ext3 journaling file system. This thing is a pig now. There isn't much running on the box, and performance is sad. (1 Reply)
Discussion started by: 98_1LE
1 Replies

2. Shell Programming and Scripting

Shell Script to hit a url

Hi all, I have a php file that grabs xml, parses it and updates my db accordingly. I want to automate the execution of this process, rather than having to hit the url manually. I have been looking into using cron to execute a script to do this, however i'm not exactly sure what command i would... (1 Reply)
Discussion started by: restivz77
1 Replies

3. Programming

why multiple SIGINT raises when i hit C-c

hi, in my application, i have set up to capture SIGINT and execute a handler.the problem is whenever i hit C-c, multiple SIGINT are sent to the application.I have blocked the SIGINT right after catching the first one but it is unsuccessful.Here is what i do : jmp_buf main_loop; int... (1 Reply)
Discussion started by: Sedighzadeh
1 Replies

4. Shell Programming and Scripting

Hit count on a shell script

I have a unix shell script (ex.sh) written. How to find out how many users (incl. myself) have run this .sh ? I can insert code snipet at top of script if need be. - Ravi (2 Replies)
Discussion started by: ravi368
2 Replies

5. Shell Programming and Scripting

Getting Next Best Hit..

Hi.. I need to get the following output from the input file like this INPUT GRM1 GRM1 0 GRM1 ABC1 1 GRM1 FEQ1 2 GRM1 SED1 3 ABC2 GRM1 0 ABC2 ABC2 1 ABC2 FEQ1 2 ABC2 BED1 3 SED1 SED1 0 SED1 SED1 1 SED1 SED1 2 SED1 ABC1 3 OUTPUT: (7 Replies)
Discussion started by: empyrean
7 Replies

6. SuSE

Java hit

Hello, I'm having trouble looking for info for SUSIE on this CVE-2012-4681. This is basically the newest Java hit. It is mostly a web browser issue but I would like to see if the versions on our servers are vulnerable. I already found the pages/info for Solaris and RHEL. Any help would be... (4 Replies)
Discussion started by: bitlord
4 Replies

7. Cybersecurity

vnc password hit from Retina

Hello, I'm having an issue with VNC. Security at work says that they scanned my servers (Solaris, RHEL, SLES) and found that you don't need a password to access a VNC session. I have tested this and you can't login to the VNC session without a password. Can someone tell what the Retina scanner... (1 Reply)
Discussion started by: bitlord
1 Replies

8. Shell Programming and Scripting

Curl to hit the submit button

Hello, I am looking to hit a URL using curl and click on submit button so that I can get the results. The below is the code <input name="tos_accepted" id="tos_accepted" class="button" type="submit" value="Yes, I Agree"/> <input name="tos_discarded" id="tos_discarded"... (1 Reply)
Discussion started by: Kochappa
1 Replies

9. Shell Programming and Scripting

Best performance to merge two files

Hi Gurus, I need to merge two files. file1 (small file, only one line) this is first linefile2 (large file) abc def ghi ... I use below command to merge the file, since the file2 is really large file, the command read whole file2, the performance is not good. cat file1 > file3... (7 Replies)
Discussion started by: green_k
7 Replies

LEARN ABOUT DEBIAN

scrounge-ntfs

scrounge-ntfs(8)					    BSD System Manager's Manual 					  scrounge-ntfs(8)

NAME

     scrounge-ntfs -- helps retrieve data from corrupted NTFS partitions

SYNOPSIS

     scrounge-ntfs -l disk
     scrounge-ntfs -s disk
     scrounge-ntfs [-m mftoffset] [-c clustersize] [-o outdir] disk start end

DESCRIPTION

     scrounge-ntfs is a utility that can rescue data from corrupted NTFS partitions. It writes the files retrieved to another working file system.
     Certain information about the partition needs to be known in advance.

     The -l mode is meant to be run in advance of the data corruption, with the output stored away in a file. This allows scrounge-ntfs to recover
     data reliably. See the 'NOTES' section below for recover info when this isn't the case.

OPTIONS

     The options are as follows:

     -c 	 The cluster size (in sectors). When not specified a default of 8 is used.

     -l 	 List partition information for a drive. This will only work when the partition table for the given drive is intact.

     -m 	 When recovering data this specifies the location of the MFT from the beginning of the partition (in sectors). If not specified
		 then no directory information can be used, that is, all rescued files will be written to the same directory.

     -o 	 Directory to put rescued files in. If not specified then files will be placed in the current directory.

     -s 	 Search disk for partition information. (Not implemented yet).

     disk	 The raw device used to access the disk which contains the NTFS partition to rescue files from. eg: '/dev/hdc'

     start	 The beginning of the NTFS partition (in sectors).

     end	 The end of the NTFS partition (in sectors)

NOTES

     If you plan on using this program sucessfully you should prepare in advance by storing a copy of the partition information. Use the -l option
     to do this. Eventually searching for disk partition information will be implemented, which will solve this problem.

     When only one partition exists on a disk or you want to rescue the first partition there are ways to guess at the sector sizes and MFT loca-
     tion. See the scrounge-ntfs web page for more info:

     http://memberwebs.com/swalter/software/scrounge/

AUTHOR

     Stef Walter <stef@memberwebs.com>

scrounge-ntfs							   June 1, 2019 						     scrounge-ntfs
All times are GMT -4. The time now is 01:27 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy