Sponsored Content
Operating Systems AIX What is the limitation in AIX? Post 302804009 by bakunin on Tuesday 7th of May 2013 11:55:12 PM
Old 05-08-2013
What DGPickett means is the following:

A directory is quite similar to a file and the bigger a file gets the longer it takes the system to read it, which is to be expected. Run a "grep" against a file of 10GB and it will take longer than against a file of 1k size.

Let us consider the case where you issue a command

Code:
grep regexp /path/to/some/file

What happens? Before "grep" can start its work the operating system has to find out which file to open. So it looks in the directory "/path/to/some" and searches there for the inode of "file". A "directory" now is nothing else than a (quite unsorted) list of file names and inode-numbers. The longer this list is the longer it will take the take the OS to search it and find the inode it is interested in.

Usually you won't notice even this difference because the OS uses otherwise unused parts of the memory to buffer such information. This is part of the "file system cache": the system won't read the directory information from disk, but use the copy it has already stored in memory. As memory is much faster than disk this will speed up things considerably. But as the directory gets bigger and bigger and memory is a limited resource at some point the list might not fit in memory any more additionally hurting the speed with which this list is searched.

Bottom line: even if there are no theoretical limits there is some practical limit to directory sizes. This practical limit is pushed as hardware gets faster and memory keeps getting bigger, disks getting faster, etc.., but it still remains.

To split a large directory there is no "standard tool" like there is "split" for files. Just create new directories and use "mv" to move files from one to the other. A command like

Code:
mv /path/to/file /other/path

will physically move a file only of the directories "/path/to" and "/other/path" are not part of the same filesystem. If they are it is simply a matter of removing the directory information from the one list and putting it into the other. It will take the same time regardless of file size, because the file itself is not touched, just "file metadata" - information about files instead of files themselves.

I hope this clears things up.

bakunin
These 2 Users Gave Thanks to bakunin For This Post:
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

find limitation

Hi , i'm trying to use "find "command with "-size "option but i encounter 2gb file limitation. Can you confirm this limitation ? Is there a simple way to do the same thing ? My command is : <clazz01g-notes01>/base/base01 # find /base/base01 -name '*.nsf' -size +5242880000c -exec ls... (2 Replies)
Discussion started by: Nicol
2 Replies

2. HP-UX

HP-UX 11i - File Size Limitation And Number Of Folders Limitation

Hi All, Can anyone please clarify me the following questions: 1. Is there any file size limitation in HP-UX 11i, that I can able to create upto certain size of file (say 2 GB) and not more then that???? 2. At max. how many files we can able to keep inside a folder???? 3. How many... (2 Replies)
Discussion started by: sundeep_mohanty
2 Replies

3. Shell Programming and Scripting

Limitation of ls command

Hi, Iam using an alias to get the file count from one directory using normal ls command like ls file*|wc -l.If my file increases more than 35,000 ,my alias is not working.It shows that arg list too long. is that can be limitation of ls or problem in alias? I would appreciate if anyone can... (2 Replies)
Discussion started by: cskumar
2 Replies

4. Shell Programming and Scripting

Is this a bug or a limitation?

Hi, I'm having a problem with a while loop syntax that doesn't seem to loop correctly. TODAY=`date +%d%m%Y` while read hostname #for hostname in $(cat $CONFIG) do OUTFILE=/tmp/health_check.$hostname.$TODAY if then touch $OUTFILE func_header else rm $OUTFILE ... (2 Replies)
Discussion started by: gilberteu
2 Replies

5. AIX

AIX 5.3 : Limitation to 1 telnet session for some users

Hi, I search the way to limit, for a group on a AIX 5.3, one telnet session by user (Simultaneous). I search a lot in /etc/security but the only way found is with the pam authentication that i not use. No solution found also in smit menu... Thanks for your help. (2 Replies)
Discussion started by: feilong
2 Replies

6. Shell Programming and Scripting

SED on AIX Limitation

Hello, I have a problem running a script created in ksh for Linux (Tested on Debian 5.0, Ubuntu Server 10.04 and RHEL 5.1), it works properly. :b: I trying to pass it to a AIX 5.3. :wall: The problem is the character limit of 256 on a command system and SED. I need to cut the contents of... (8 Replies)
Discussion started by: nemesis.spa
8 Replies

7. AIX

Limitation for SFTP on AIX number of sessions

Hello. I am using AIX 6 and If wish to receive more than 500 files via SFTP, I get some time out errors. Could you please advise where is the limit for number of concurrent transfers setup in AIX Box or what is the limit and can that be changed? Many Thanks (3 Replies)
Discussion started by: panchpan
3 Replies

8. Shell Programming and Scripting

Limitation on rm command

Hi all, does any one know ,if there is any limitation on rm command limitation referes here as a size . Ex:when my script try to rum rm command which have size of nearly 20-22 GB ..CPU load gets high ? if anyone know the relation of CPU load and limitation of rm command . (8 Replies)
Discussion started by: niteshagrawal06
8 Replies

9. UNIX for Dummies Questions & Answers

Limitation in addition

whats wrong with this addition? Whats the maximum number of digits can be handled? pandeeswaran@ubuntu:~/Downloads$ const=201234454654768979799999 pandeeswaran@ubuntu:~/Downloads$ let new+=const pandeeswaran@ubuntu:~/Downloads$ echo $new -2152890657037557890 pandeeswaran@ubuntu:~/Downloads$ (4 Replies)
Discussion started by: pandeesh
4 Replies

10. AIX

AIX lpar bad disk I/O performance - 4k per IO limitation ?

Hi Guys, I have fresh new installed VIO 2.2.3.70 on a p710, 3 physical SAS disks, rootvg on hdisk0 and 3 VIO clients through vscsi, AIX7.1tl4 AIX6.1tl9 RHEL6.5ppc, each lpar has its rootvg installed on a LV on datavg (hdisk2) mapped to vhost0,1,2 There is no vg on hdisk1, I use it for my... (1 Reply)
Discussion started by: frenchy59
1 Replies
PERLAIX(1)						 Perl Programmers Reference Guide						PERLAIX(1)

NAME
README.aix - Perl version 5 on IBM Unix (AIX) systems DESCRIPTION
This document describes various features of IBM's Unix operating system (AIX) that will affect how Perl version 5 (hereafter just Perl) is compiled and/or runs. Compiling Perl 5 on AIX When compiling Perl, you must use an ANSI C compiler. AIX does not ship an ANSI compliant C-compiler with AIX by default, but binary builds of gcc for AIX are widely available. At the moment of writing, AIX supports two different native C compilers, for which you have to pay: xlC and vac. If you decide to use either of these two (which is quite a lot easier than using gcc), be sure to upgrade to the latest available patch level. Currently: xlC.C 3.1.4.10 or 3.6.6.0 or 4.0.2.2 or 5.0.2.9 or 6.0.0.3 vac.C 4.4.0.3 or 5.0.2.6 or 6.0.0.1 note that xlC has the OS version in the name as of version 4.0.2.0, so you will find xlC.C for AIX-5.0 as package xlC.aix50.rte 5.0.2.0 or 6.0.0.3 subversions are not the same `latest' on all OS versions. For example, the latest xlC-5 on aix41 is 5.0.2.9, while on aix43, it is 5.0.2.7. Perl can be compiled with either IBM's ANSI C compiler or with gcc. The former is recommended, as not only can it compile Perl with no difficulty, but also can take advantage of features listed later that require the use of IBM compiler-specific command-line flags. The IBM's compiler patch levels 5.0.0.0 and 5.0.1.0 have compiler optimization bugs that affect compiling perl.c and regcomp.c, respec- tively. If Perl's configuration detects those compiler patch levels, optimization is turned off for the said source code files. Upgrading to at least 5.0.2.0 is recommended. If you decide to use gcc, make sure your installation is recent and complete, and be sure to read the Perl README file for more gcc-spe- cific details. Please report any hoops you had to jump through to the development team. OS level Before installing the patches to the IBM C-compiler you need to know the level of patching for the Operating System. IBM's command 'oslevel' will show the base, but is not always complete (in this example oslevel shows 4.3.NULL, whereas the system might run most of 4.3.THREE): # oslevel 4.3.0.0 # lslpp -l | grep 'bos.rte ' bos.rte 4.3.3.75 COMMITTED Base Operating System Runtime bos.rte 4.3.2.0 COMMITTED Base Operating System Runtime # The same might happen to AIX 5.1 or other OS levels. As a side note, perl cannot be built without bos.adt.syscalls and bos.adt.libm installed # lslpp -l | egrep "syscalls|libm" bos.adt.libm 5.1.0.25 COMMITTED Base Application Development bos.adt.syscalls 5.1.0.36 COMMITTED System Calls Application # Building Dynamic Extensions on AIX AIX supports dynamically loadable objects as well as shared libraries. Shared libraries by convention end with the suffix .a, which is a bit misleading, as an archive can contain static as well as dynamic members. For perl dynamically loaded objects we use the .so suffix also used on many other platforms. Note that starting from Perl 5.7.2 (and consequently 5.8.0) and AIX 4.3 or newer Perl uses the AIX native dynamic loading interface in the so called runtime linking mode instead of the emulated interface that was used in Perl releases 5.6.1 and earlier or, for AIX releases 4.2 and earlier. This change does break backward compatibility with compiled modules from earlier perl releases. The change was made to make Perl more compliant with other applications like Apache/mod_perl which are using the AIX native interface. This change also enables the use of C++ code with static constructors and destructors in perl extensions, which was not possible using the emulated interface. The IBM ANSI C Compiler All defaults for Configure can be used. If you've chosen to use vac 4, be sure to run 4.4.0.3. Older versions will turn up nasty later on. For vac 5 be sure to run at least 5.0.1.0, but vac 5.0.2.6 or up is highly recommended. Note that since IBM has removed vac 5.0.2.1 through 5.0.2.5 from the software depot, these versions should be considered obsolete. Here's a brief lead of how to upgrade the compiler to the latest level. Of course this is subject to changes. You can only upgrade ver- sions from ftp-available updates if the first three digit groups are the same (in where you can skip intermediate unlike the patches in the developer snapshots of perl), or to one version up where the `base' is available. In other words, the AIX compiler patches are cumulative. vac.C.4.4.0.1 => vac.C.4.4.0.3 is OK (vac.C.4.4.0.2 not needed) xlC.C.3.1.3.3 => xlC.C.3.1.4.10 is NOT OK (xlC.C.3.1.4.0 is not available) # ftp ftp.software.ibm.com Connected to service.boulder.ibm.com. : welcome message ... Name (ftp.software.ibm.com:merijn): anonymous 331 Guest login ok, send your complete e-mail address as password. Password: ... accepted login stuff ftp> cd /aix/fixes/v4/ ftp> dir other other.ll output to local-file: other.ll? y 200 PORT command successful. 150 Opening ASCII mode data connection for /bin/ls. 226 Transfer complete. ftp> dir xlc xlc.ll output to local-file: xlc.ll? y 200 PORT command successful. 150 Opening ASCII mode data connection for /bin/ls. 226 Transfer complete. ftp> bye ... goodbye messages # ls -l *.ll -rw-rw-rw- 1 merijn system 1169432 Nov 2 17:29 other.ll -rw-rw-rw- 1 merijn system 29170 Nov 2 17:29 xlc.ll On AIX 4.2 using xlC, we continue: # lslpp -l | fgrep 'xlC.C ' xlC.C 3.1.4.9 COMMITTED C for AIX Compiler xlC.C 3.1.4.0 COMMITTED C for AIX Compiler # grep 'xlC.C.3.1.4.*.bff' xlc.ll -rw-r--r-- 1 45776101 1 6286336 Jul 22 1996 xlC.C.3.1.4.1.bff -rw-rw-r-- 1 45776101 1 6173696 Aug 24 1998 xlC.C.3.1.4.10.bff -rw-r--r-- 1 45776101 1 6319104 Aug 14 1996 xlC.C.3.1.4.2.bff -rw-r--r-- 1 45776101 1 6316032 Oct 21 1996 xlC.C.3.1.4.3.bff -rw-r--r-- 1 45776101 1 6315008 Dec 20 1996 xlC.C.3.1.4.4.bff -rw-rw-r-- 1 45776101 1 6178816 Mar 28 1997 xlC.C.3.1.4.5.bff -rw-rw-r-- 1 45776101 1 6188032 May 22 1997 xlC.C.3.1.4.6.bff -rw-rw-r-- 1 45776101 1 6191104 Sep 5 1997 xlC.C.3.1.4.7.bff -rw-rw-r-- 1 45776101 1 6185984 Jan 13 1998 xlC.C.3.1.4.8.bff -rw-rw-r-- 1 45776101 1 6169600 May 27 1998 xlC.C.3.1.4.9.bff # wget ftp://ftp.software.ibm.com/aix/fixes/v4/xlc/xlC.C.3.1.4.10.bff # On AIX 4.3 using vac, we continue: # lslpp -l | grep 'vac.C ' vac.C 5.0.2.2 COMMITTED C for AIX Compiler vac.C 5.0.2.0 COMMITTED C for AIX Compiler # grep 'vac.C.5.0.2.*.bff' other.ll -rw-rw-r-- 1 45776101 1 13592576 Apr 16 2001 vac.C.5.0.2.0.bff -rw-rw-r-- 1 45776101 1 14133248 Apr 9 2002 vac.C.5.0.2.3.bff -rw-rw-r-- 1 45776101 1 14173184 May 20 2002 vac.C.5.0.2.4.bff -rw-rw-r-- 1 45776101 1 14192640 Nov 22 2002 vac.C.5.0.2.6.bff # wget ftp://ftp.software.ibm.com/aix/fixes/v4/other/vac.C.5.0.2.6.bff # Likewise on all other OS levels. Then execute the following command, and fill in its choices # smit install_update -> Install and Update from LATEST Available Software * INPUT device / directory for software [ vac.C.5.0.2.6.bff ] [ OK ] [ OK ] Follow the messages ... and you're done. If you like a more web-like approach, a good start point can be http://www14.software.ibm.com/webapp/download/downloadaz.jsp and click "C for AIX", and follow the instructions. Using GNU's gcc for building perl Using gcc-3.x (tested with 3.0.4, 3.1, and 3.2) now works out of the box, as do recent gcc-2.9 builds available directly from IBM as part of their Linux compatibility packages, available here: http://www.ibm.com/servers/aix/products/aixos/linux/ Using Large Files with Perl Should yield no problems. Threaded Perl Threads seem to work OK, though at the moment not all tests pass when threads are used in combination with 64bit configurations. 64-bit Perl If your AIX is installed with 64-bit support, you can expect 64bit configurations to work. In combination with threads some tests might still fail. AIX 4.2 and extensions using C++ with statics In AIX 4.2 Perl extensions that use C++ functions that use statics may have problems in that the statics are not getting initialized. In newer AIX releases this has been solved by linking Perl with the libC_r library, but unfortunately in AIX 4.2 the said library has an obscure bug where the various functions related to time (such as time() and gettimeofday()) return broken values, and therefore in AIX 4.2 Perl is not linked against the libC_r. AUTHOR
H.Merijn Brand <h.m.brand@hccnet.nl> DATE
Version 0.0.6: 23 Dec 2002 perl v5.8.0 2003-02-18 PERLAIX(1)
All times are GMT -4. The time now is 10:57 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy