Sponsored Content
Full Discussion: Sort a huge file in UNIX
Top Forums UNIX for Beginners Questions & Answers Sort a huge file in UNIX Post 303045681 by sure_sreekar23 on Wednesday 8th of April 2020 08:38:31 AM
Old 04-08-2020
Sort a huge file in UNIX

Hi all,

We are using Unix sort command to sort a 10 GB file. And the command is sometimes failing and some time it's running successfully.

We observed when it fails it's giving us below error:
/Tmp/somedirname - No space left on Device

So I saw in man pages that using sort -T would solve error. But I am unsure of the syntax. Can someone help we with the syntax
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

File size limitation of unix sort command.

hi , iam trying to sort millions of records which is delimited and i cant able to use sort command more than 60 million..if i try to do so i got an message stating that "File size limit exceeded",Is there any file size limit for using sort command.. How can i solve this problem. thanks ... (7 Replies)
Discussion started by: cskumar
7 Replies

2. Shell Programming and Scripting

UNix one-liner to sort a file

Hi everyone, I have a txt file as: >001.b1 GCTAGTGCTAGCTAGCTAGCATCGATCGAT >002.b1 CAGTCAGTCGTAGTGCTAGCTGATGCTCGT >003.b1 CGATCGTAGTCGTATCGATGCTGACGTAGG >002.g1 ATGCTGATCGACTAGCTAGTCGT >015.b1 CGATCTAGTAGTGCTAGTCGTTT >001.g1 ATGCTGATCGACTAGCTAGTCGT >003.g1 CGATGCTAGTCGATGCTGACGGG (3 Replies)
Discussion started by: ad23
3 Replies

3. UNIX for Dummies Questions & Answers

Mainframe to Unix. Huge file >5GB

Hi, The files are being zipped from mainframe to the UNIX server. I have to unzip huge files on AIX server which is too old. The UNZIP as well as gzip utility version is not able to handle it and if we have to install the latest version there is a lot of process which takes lot of process. Is... (4 Replies)
Discussion started by: Bijesh
4 Replies

4. Shell Programming and Scripting

Comparision of two huge unix files - Reconcilation

Hi, I have two huge file; each one has approximately 150000 lines. I need to compare both of them and store the unmatched lines into a different file. I have searched for everything in google but did not get solution. Files are: File1 NRALBAMINDB20003726 NRALBAMINDB20003727... (16 Replies)
Discussion started by: Suman Singh
16 Replies

5. UNIX for Advanced & Expert Users

Script to sort the files and append the extension .sort to the sorted version of the file

Hello all - I am to this forum and fairly new in learning unix and finding some difficulty in preparing a small shell script. I am trying to make script to sort all the files given by user as input (either the exact full name of the file or say the files matching the criteria like all files... (3 Replies)
Discussion started by: pankaj80
3 Replies

6. UNIX for Dummies Questions & Answers

Unable to sort file name in Unix

Hi, I'm unable to sort the files with the required names on Unix. Details have been enclosed:- Below are the fle name inside a folder, now i need to sort them in a way that they are group with the seconds occurance in the file name (E.g 01954,07947,06794) 01.01954.MXRS1.rcv... (3 Replies)
Discussion started by: kkkk_ssss
3 Replies

7. Shell Programming and Scripting

Unix Scripting : Sort a Portion of a File and not the complete file

Need to sort a portion of a file in a Alphabetical Order. Example : The user adam is not sorted and the user should get sorted. I don't want the complete file to get sorted. Currently All_users.txt contains the following lines. ############## # ARS USERS ############## mike, Mike... (6 Replies)
Discussion started by: evrurs
6 Replies

8. UNIX for Dummies Questions & Answers

sort a unix file by 3rd column

Hi, Can anybody tell me how to sort a unix file by 3rd column and not by ltr? Please help Thanks in advance (2 Replies)
Discussion started by: vinnyvirk
2 Replies

9. Shell Programming and Scripting

How to print huge values in sqlplus variable in UNIX?

Hi, I ahve a unix code as below. sqlTxt=$* sqlReturn=`echo "set feedback off; set heading off; set term off; set verify off; set serveroutput on size unlimited set lin 1000; VARIABLE GV_return_val NUMBER; VARIABLE GV_script_error varchar2(4000); EXEC :GV_return_code := 0; EXEC... (6 Replies)
Discussion started by: bhaski2012
6 Replies

10. Shell Programming and Scripting

How to Modify a file content in UNIX and sort for only required fields ?

I have the below contents in a file after making the below curl call curl ... | grep -E "state|Rno" | paste -sd',\n' | grep "Disconnected" > test "state" : "Disconnected",, "Rno" : "5554f1d2" "state" : "Disconnected",, "Rno" : "10587563" "state" : "Disconnected",, "Rno" :... (2 Replies)
Discussion started by: Vaibhav H
2 Replies
sort(3pm)						 Perl Programmers Reference Guide						 sort(3pm)

NAME
sort - perl pragma to control sort() behaviour SYNOPSIS
use sort 'stable'; # guarantee stability use sort '_quicksort'; # use a quicksort algorithm use sort '_mergesort'; # use a mergesort algorithm use sort 'defaults'; # revert to default behavior no sort 'stable'; # stability not important use sort '_qsort'; # alias for quicksort my $current; BEGIN { $current = sort::current(); # identify prevailing algorithm } DESCRIPTION
With the "sort" pragma you can control the behaviour of the builtin "sort()" function. In Perl versions 5.6 and earlier the quicksort algorithm was used to implement "sort()", but in Perl 5.8 a mergesort algorithm was also made available, mainly to guarantee worst case O(N log N) behaviour: the worst case of quicksort is O(N**2). In Perl 5.8 and later, quicksort defends against quadratic behaviour by shuffling large arrays before sorting. A stable sort means that for records that compare equal, the original input ordering is preserved. Mergesort is stable, quicksort is not. Stability will matter only if elements that compare equal can be distinguished in some other way. That means that simple numerical and lexical sorts do not profit from stability, since equal elements are indistinguishable. However, with a comparison such as { substr($a, 0, 3) cmp substr($b, 0, 3) } stability might matter because elements that compare equal on the first 3 characters may be distinguished based on subsequent characters. In Perl 5.8 and later, quicksort can be stabilized, but doing so will add overhead, so it should only be done if it matters. The best algorithm depends on many things. On average, mergesort does fewer comparisons than quicksort, so it may be better when complicated comparison routines are used. Mergesort also takes advantage of pre-existing order, so it would be favored for using "sort()" to merge several sorted arrays. On the other hand, quicksort is often faster for small arrays, and on arrays of a few distinct values, repeated many times. You can force the choice of algorithm with this pragma, but this feels heavy-handed, so the subpragmas beginning with a "_" may not persist beyond Perl 5.8. The default algorithm is mergesort, which will be stable even if you do not explicitly demand it. But the stability of the default sort is a side-effect that could change in later versions. If stability is important, be sure to say so with a use sort 'stable'; The "no sort" pragma doesn't forbid what follows, it just leaves the choice open. Thus, after no sort qw(_mergesort stable); a mergesort, which happens to be stable, will be employed anyway. Note that no sort "_quicksort"; no sort "_mergesort"; have exactly the same effect, leaving the choice of sort algorithm open. CAVEATS
As of Perl 5.10, this pragma is lexically scoped and takes effect at compile time. In earlier versions its effect was global and took effect at run-time; the documentation suggested using "eval()" to change the behaviour: { eval 'use sort qw(defaults _quicksort)'; # force quicksort eval 'no sort "stable"'; # stability not wanted print sort::current . " "; @a = sort @b; eval 'use sort "defaults"'; # clean up, for others } { eval 'use sort qw(defaults stable)'; # force stability print sort::current . " "; @c = sort @d; eval 'use sort "defaults"'; # clean up, for others } Such code no longer has the desired effect, for two reasons. Firstly, the use of "eval()" means that the sorting algorithm is not changed until runtime, by which time it's too late to have any effect. Secondly, "sort::current" is also called at run-time, when in fact the compile-time value of "sort::current" is the one that matters. So now this code would be written: { use sort qw(defaults _quicksort); # force quicksort no sort "stable"; # stability not wanted my $current; BEGIN { $current = sort::current; } print "$current "; @a = sort @b; # Pragmas go out of scope at the end of the block } { use sort qw(defaults stable); # force stability my $current; BEGIN { $current = sort::current; } print "$current "; @c = sort @d; } perl v5.18.2 2013-11-04 sort(3pm)
All times are GMT -4. The time now is 12:03 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy