Sponsored Content
Top Forums Shell Programming and Scripting File size limitation of unix sort command. Post 85409 by cskumar on Wednesday 5th of October 2005 09:40:49 AM
Old 10-05-2005
Hi Pixelbeat,

Thanks for your quick response. We are trying this on HP UX 11. We are generating input file for sorting through c program and calling unix sort command through c only, as follows::

#include <stdio.h>
#include <string.h>
#include <stdlib.h>

FILE *fout;
int main(int argc,char *argv[])
{
int loopcnt=0;
unsigned long int TTRECS = 60000000;

if((fout = fopen("SORTINPUT","w")) == NULL)
{
printf("\nUnable to open the output_file file\n");
exit(1);
}
for (loopcnt=TTRECS;loopcnt>=1;loopcnt--)
{
fprintf(fout,"%d|%s%d|%s|\n",loopcnt,"Dummy",loopcnt,"Dummy Description");
}
fflush(fout);
printf("Executing::sort -t '|' -k 1,1 SORTINPUT> sorttestoutput\n");

system("echo Sort Start Time;date");
system("sort -t '|' -k 1,1 SORTINPUT > sorttestoutput");
system("echo status returned::$?");

system("echo Sort End Time;date");
printf("Total Records Configured:: %lu\n", TTRECS);
return 0;
}


We are trying this test just to make sure that unix sort command is not having any limitations on file size. But what we observed is when file is packed with more than 60 million records [ around 2 Gb size], we are getting an error stating "File size limit exceeded" on console. Can u kindly state,

1] whether we got any maximum file size limit in HP UX while creating a file through C program.

2] Can unix sort command can handle file size > 2 GB without any failure.

3] Is there any restriction for length of each record [each line] or each field[fields are separated by a delimeter].

Thanks,
 

10 More Discussions You Might Find Interesting

1. Solaris

unix group file limitation

Does anyone know how to get around the unix group file limitation whereby you have a limit of 1024 characters when adding users to a unix group? (3 Replies)
Discussion started by: asmillie
3 Replies

2. HP-UX

HP-UX 11i - File Size Limitation And Number Of Folders Limitation

Hi All, Can anyone please clarify me the following questions: 1. Is there any file size limitation in HP-UX 11i, that I can able to create upto certain size of file (say 2 GB) and not more then that???? 2. At max. how many files we can able to keep inside a folder???? 3. How many... (2 Replies)
Discussion started by: sundeep_mohanty
2 Replies

3. Linux

File size limitation for rcp

Hi I am trying to rcp a file from Solaris box to Linux. When the file size is 2,205,255,047, the rcp fails with the message Jan 10 01:11:53 hqsas167 rsh: pam_authenticate: error Authentication failed However when I rcp a file with smaller size - 9,434,477 - the rcp completes with... (2 Replies)
Discussion started by: schoubal
2 Replies

4. UNIX for Dummies Questions & Answers

Unix -- Space problem -- File number limitation?

Dear all Recently I cant touch file in one mount point (which is not full, 78% full only), it says can't write to device, obviously it means it's full, I deleted some files and I can write some files only. I wonder is there any file number limitation in a mount point and how can I check or how... (2 Replies)
Discussion started by: shanemcmahon
2 Replies

5. Shell Programming and Scripting

Size limitation in Tar command

Hi to every body there, I am new this forum and this is my first post. I am a new user of Unix, is there any size limitation of files while creating tar file. Thanks in advance (4 Replies)
Discussion started by: Manvar Khan
4 Replies

6. Shell Programming and Scripting

fetchmail - log file size limitation

Hi, I am using fetchmail in my application so as to download mails to the localhost where the application is hosted from the mailserver.Fetchmail is configured as as to run as a daemon polling mails during an interval of 1sec. So my concern here is, during each 2sec it is writing two... (10 Replies)
Discussion started by: DILEEP410
10 Replies

7. UNIX for Advanced & Expert Users

Find command -size option limitation ?

Hi All, I ran code in test environment to find the files more than 1TB given below is a snippet from code: FILE_SYSTEM=/home/arun MAX_FILE_LIMIT=1099511627776 find $FILE_SYSTEM -type f -size +"$MAX_FILE_LIMIT"c -ls -xdev 2>/dev/null | while read fname do echo "File larger than... (3 Replies)
Discussion started by: Arunprasad
3 Replies

8. Solaris

How to extend 2 GB file size limitation

Hello All, I am using a SunOS machine. My application creates output files for the downstream systems. However output files are restricted to 2GB of file size in SunOS due to which I am forced to create multiple files which is not supported by the downstream due to some limitations. Is... (5 Replies)
Discussion started by: pasupuleti81
5 Replies

9. Linux

File size limitation in Linux

Hi friends, I tried to take a backup of my PC using tar command. But it ended with an error tar: /home/backup/back.tar.gz: Cannot write: No space left on device tar: Error is not recoverable: exiting now But i checked the disk space and there is enough space is available. ]# df Filesystem... (11 Replies)
Discussion started by: siva3492
11 Replies

10. Linux

File size limitation in the EST 2012 x86_64 GNU/Linux

Hello Friends, I tried to take tar backup in my server, but it ended with an error. It said that: /home/back/pallava_backup/fbackup_backup/stape_config /home/back/romam_new.tar.gz tar: /home/backup/back.tar.gz: Cannot write: No space left on device tar: Error is not recoverable: exiting... (10 Replies)
Discussion started by: siva3492
10 Replies
sortbib(1)							   User Commands							sortbib(1)

NAME
sortbib - sort a bibliographic database SYNOPSIS
sortbib [-s KEYS] database... DESCRIPTION
sortbib sorts files of records containing refer key-letters by user-specified keys. Records may be separated by blank lines, or by `.[' and `.]' delimiters, but the two styles may not be mixed together. This program reads through each database and pulls out key fields, which are sorted separately. The sorted key fields contain the file pointer, byte offset, and length of corresponding records. These records are delivered using disk seeks and reads, so sortbib may not be used in a pipeline to read standard input. The most common key-letters and their meanings are given below. %A Author's name %B Book containing article referenced %C City (place of publication) %D Date of publication %E Editor of book containing article referenced %F Footnote number or label (supplied by refer) %G Government order number %H Header commentary, printed before reference %I Issuer (publisher) %J Journal containing article %K Keywords to use in locating reference %L Label field used by -k option of refer %M Bell Labs Memorandum (undefined) %N Number within volume %O Other commentary, printed at end of reference %P Page number(s) %Q Corporate or Foreign Author (unreversed) %R Report, paper, or thesis (unpublished) %S Series title %T Title of article or book %V Volume number %X Abstract -- used by roffbib, not by refer %Y,Z Ignored by refer By default, sortbib alphabetizes by the first %A and the %D fields, which contain the senior author and date. sortbib sorts on the last word on the %A line, which is assumed to be the author's last name. A word in the final position, such as `jr.' or `ed.', will be ignored if the name beforehand ends with a comma. Authors with two-word last names or unusual constructions can be sorted correctly by using the nroff convention `' in place of a blank. A %Q field is considered to be the same as %A, except sorting begins with the first, not the last, word. sortbib sorts on the last word of the %D line, usually the year. It also ignores leading articles (like `A' or `The') when sorting by titles in the %T or %J fields; it will ignore articles of any modern European language. If a sort-significant field is absent from a record, sortbib places that record before other records containing that field. No more than 16 databases may be sorted together at one time. Records longer than 4096 characters will be truncated. OPTIONS
-sKEYS Specify new KEYS. For instance, -sATD will sort by author, title, and date, while -sA+D will sort by all authors, and date. Sort keys past the fourth are not meaningful. ATTRIBUTES
See attributes(5) for descriptions of the following attributes: +-----------------------------+-----------------------------+ | ATTRIBUTE TYPE | ATTRIBUTE VALUE | +-----------------------------+-----------------------------+ |Availability |SUNWdoc | +-----------------------------+-----------------------------+ SEE ALSO
addbib(1), indxbib(1), lookbib(1), refer(1), roffbib(1), attributes(5) BUGS
Records with missing author fields should probably be sorted by title. SunOS 5.11 14 Sep 1992 sortbib(1)
All times are GMT -4. The time now is 09:26 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy