Sponsored Content
Top Forums Shell Programming and Scripting File size limitation of unix sort command. Post 85409 by cskumar on Wednesday 5th of October 2005 09:40:49 AM
Old 10-05-2005
Hi Pixelbeat,

Thanks for your quick response. We are trying this on HP UX 11. We are generating input file for sorting through c program and calling unix sort command through c only, as follows::

#include <stdio.h>
#include <string.h>
#include <stdlib.h>

FILE *fout;
int main(int argc,char *argv[])
{
int loopcnt=0;
unsigned long int TTRECS = 60000000;

if((fout = fopen("SORTINPUT","w")) == NULL)
{
printf("\nUnable to open the output_file file\n");
exit(1);
}
for (loopcnt=TTRECS;loopcnt>=1;loopcnt--)
{
fprintf(fout,"%d|%s%d|%s|\n",loopcnt,"Dummy",loopcnt,"Dummy Description");
}
fflush(fout);
printf("Executing::sort -t '|' -k 1,1 SORTINPUT> sorttestoutput\n");

system("echo Sort Start Time;date");
system("sort -t '|' -k 1,1 SORTINPUT > sorttestoutput");
system("echo status returned::$?");

system("echo Sort End Time;date");
printf("Total Records Configured:: %lu\n", TTRECS);
return 0;
}


We are trying this test just to make sure that unix sort command is not having any limitations on file size. But what we observed is when file is packed with more than 60 million records [ around 2 Gb size], we are getting an error stating "File size limit exceeded" on console. Can u kindly state,

1] whether we got any maximum file size limit in HP UX while creating a file through C program.

2] Can unix sort command can handle file size > 2 GB without any failure.

3] Is there any restriction for length of each record [each line] or each field[fields are separated by a delimeter].

Thanks,
 

10 More Discussions You Might Find Interesting

1. Solaris

unix group file limitation

Does anyone know how to get around the unix group file limitation whereby you have a limit of 1024 characters when adding users to a unix group? (3 Replies)
Discussion started by: asmillie
3 Replies

2. HP-UX

HP-UX 11i - File Size Limitation And Number Of Folders Limitation

Hi All, Can anyone please clarify me the following questions: 1. Is there any file size limitation in HP-UX 11i, that I can able to create upto certain size of file (say 2 GB) and not more then that???? 2. At max. how many files we can able to keep inside a folder???? 3. How many... (2 Replies)
Discussion started by: sundeep_mohanty
2 Replies

3. Linux

File size limitation for rcp

Hi I am trying to rcp a file from Solaris box to Linux. When the file size is 2,205,255,047, the rcp fails with the message Jan 10 01:11:53 hqsas167 rsh: pam_authenticate: error Authentication failed However when I rcp a file with smaller size - 9,434,477 - the rcp completes with... (2 Replies)
Discussion started by: schoubal
2 Replies

4. UNIX for Dummies Questions & Answers

Unix -- Space problem -- File number limitation?

Dear all Recently I cant touch file in one mount point (which is not full, 78% full only), it says can't write to device, obviously it means it's full, I deleted some files and I can write some files only. I wonder is there any file number limitation in a mount point and how can I check or how... (2 Replies)
Discussion started by: shanemcmahon
2 Replies

5. Shell Programming and Scripting

Size limitation in Tar command

Hi to every body there, I am new this forum and this is my first post. I am a new user of Unix, is there any size limitation of files while creating tar file. Thanks in advance (4 Replies)
Discussion started by: Manvar Khan
4 Replies

6. Shell Programming and Scripting

fetchmail - log file size limitation

Hi, I am using fetchmail in my application so as to download mails to the localhost where the application is hosted from the mailserver.Fetchmail is configured as as to run as a daemon polling mails during an interval of 1sec. So my concern here is, during each 2sec it is writing two... (10 Replies)
Discussion started by: DILEEP410
10 Replies

7. UNIX for Advanced & Expert Users

Find command -size option limitation ?

Hi All, I ran code in test environment to find the files more than 1TB given below is a snippet from code: FILE_SYSTEM=/home/arun MAX_FILE_LIMIT=1099511627776 find $FILE_SYSTEM -type f -size +"$MAX_FILE_LIMIT"c -ls -xdev 2>/dev/null | while read fname do echo "File larger than... (3 Replies)
Discussion started by: Arunprasad
3 Replies

8. Solaris

How to extend 2 GB file size limitation

Hello All, I am using a SunOS machine. My application creates output files for the downstream systems. However output files are restricted to 2GB of file size in SunOS due to which I am forced to create multiple files which is not supported by the downstream due to some limitations. Is... (5 Replies)
Discussion started by: pasupuleti81
5 Replies

9. Linux

File size limitation in Linux

Hi friends, I tried to take a backup of my PC using tar command. But it ended with an error tar: /home/backup/back.tar.gz: Cannot write: No space left on device tar: Error is not recoverable: exiting now But i checked the disk space and there is enough space is available. ]# df Filesystem... (11 Replies)
Discussion started by: siva3492
11 Replies

10. Linux

File size limitation in the EST 2012 x86_64 GNU/Linux

Hello Friends, I tried to take tar backup in my server, but it ended with an error. It said that: /home/back/pallava_backup/fbackup_backup/stape_config /home/back/romam_new.tar.gz tar: /home/backup/back.tar.gz: Cannot write: No space left on device tar: Error is not recoverable: exiting... (10 Replies)
Discussion started by: siva3492
10 Replies
look(1) 						      General Commands Manual							   look(1)

NAME
look - Finds lines in a sorted list SYNOPSIS
look [-df] [-tcharacter] string [file] The look command prints all lines in a sorted file that begin with string. OPTIONS
Uses dictionary order; only letters, digits, tabs, and spaces are used in comparisons. Searches without regard to case; treats uppercase and lowercase as equivalent. Ignores character and characters following it in the search string. If you specify look -tC ABCDE, the string ABCDE would become (in effect) AB, with CDE being ignored. This option is primarily for shell scripts, in which more than one string is being processed. DESCRIPTION
If no file is specified, look searches in the system word list /usr/share/dict/words, with the options -df assumed by default. The look command uses binary search. The -d and -f options affect comparisons as in sort. NOTES
In order to use the -f option, you must first sort file with the sort -f command; otherwise, look displays only lowercase items. If you do not specify -f, but specify a file (such as /usr/share/dict/words) that has been sorted with sort -f, look may not produce any output. EXAMPLES
To search a sorted file called sortfile for all lines that begin with the string as, enter: look as sortfile To search the system word list for all words beginning with smi, enter: look smi This might result in: smile smirk smith smithereens Smithfield Smithson smithy smitten FILES
System word list. SEE ALSO
Commands: grep(1), sort(1), spell(1) look(1)
All times are GMT -4. The time now is 03:26 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy