Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Explanation of the sort command
Top Forums UNIX for Dummies Questions & Answers Explanation of the sort command Post 302758875 by Don Cragun on Monday 21st of January 2013 02:31:47 AM
Old 01-21-2013
Quote:
Originally Posted by jimbojames
Hi Don,

The output from running
Code:
ls ${DEST_LOCATION}/${FILES} | tee /tmp/pipe1 | sort -r -k 4,4n | tee /tmp/pipe2 | head -1

is
Code:
/data/projects/PROD_IPU/inbound/SOVEREIGN/SOVEREIGN_PreviousOld.CSV]

.

The contents of pipe1 are:
Code:
/data/projects/PROD_IPU/inbound/SOVEREIGN/SOVEREIGN_PROD_20130112235921.CSV
/data/projects/PROD_IPU/inbound/SOVEREIGN/SOVEREIGN_PROD_20130113001600.CSV
/data/projects/PROD_IPU/inbound/SOVEREIGN/SOVEREIGN_PROD_20130114002828.CSV
/data/projects/PROD_IPU/inbound/SOVEREIGN/SOVEREIGN_PROD_20130115144300.CSV
/data/projects/PROD_IPU/inbound/SOVEREIGN/SOVEREIGN_PROD_20130116105000.CSV
/data/projects/PROD_IPU/inbound/SOVEREIGN/SOVEREIGN_PROD_20130116122400.CSV
/data/projects/PROD_IPU/inbound/SOVEREIGN/SOVEREIGN_PROD_20130116150101.CSV
/data/projects/PROD_IPU/inbound/SOVEREIGN/SOVEREIGN_PROD_20130116235654.CSV
/data/projects/PROD_IPU/inbound/SOVEREIGN/SOVEREIGN_PROD_20130117113000.CSV
/data/projects/PROD_IPU/inbound/SOVEREIGN/SOVEREIGN_PROD_20130117133000.CSV
/data/projects/PROD_IPU/inbound/SOVEREIGN/SOVEREIGN_PROD_20130117155000.CSV
/data/projects/PROD_IPU/inbound/SOVEREIGN/SOVEREIGN_PROD_20130117235938.CSV
/data/projects/PROD_IPU/inbound/SOVEREIGN/SOVEREIGN_PROD_201302091000.CSV

And the contents of pipe2 are:
Code:
/data/projects/PROD_IPU/inbound/SOVEREIGN/SOVEREIGN_PROD_201302091000.CSV
/data/projects/PROD_IPU/inbound/SOVEREIGN/SOVEREIGN_PROD_20130117235938.CSV
/data/projects/PROD_IPU/inbound/SOVEREIGN/SOVEREIGN_PROD_20130117155000.CSV
/data/projects/PROD_IPU/inbound/SOVEREIGN/SOVEREIGN_PROD_20130117133000.CSV
/data/projects/PROD_IPU/inbound/SOVEREIGN/SOVEREIGN_PROD_20130117113000.CSV
/data/projects/PROD_IPU/inbound/SOVEREIGN/SOVEREIGN_PROD_20130116235654.CSV
/data/projects/PROD_IPU/inbound/SOVEREIGN/SOVEREIGN_PROD_20130116150101.CSV
/data/projects/PROD_IPU/inbound/SOVEREIGN/SOVEREIGN_PROD_20130116122400.CSV
/data/projects/PROD_IPU/inbound/SOVEREIGN/SOVEREIGN_PROD_20130116105000.CSV
/data/projects/PROD_IPU/inbound/SOVEREIGN/SOVEREIGN_PROD_20130115144300.CSV
/data/projects/PROD_IPU/inbound/SOVEREIGN/SOVEREIGN_PROD_20130114002828.CSV
/data/projects/PROD_IPU/inbound/SOVEREIGN/SOVEREIGN_PROD_20130113001600.CSV
/data/projects/PROD_IPU/inbound/SOVEREIGN/SOVEREIGN_PROD_20130112235921.CSV

What I am attempting to do is identify the newest file and delete any other than this, which I think is working using the combination of
Code:
FILENAME=$(ssh ${DEST_SERV} "ls ${DEST_LOCATION}/${FILES} | sort -r -k 4n | head -1")

and
Code:
ssh ${DEST_SERV} "ls ${DEST_LOCATION}/${FILES} | grep -v ${FILENAME} | xargs rm -rf"

.
I'm sorry, but this just is not possible. The head -1 should print the 1st line in pipe2. But:
Code:
/data/projects/PROD_IPU/inbound/SOVEREIGN/SOVEREIGN_PreviousOld.CSV]

does not appear in pipe2 nor in pipe1!

But, whether it is there or not, the -k option and its option argument (whether it is 4n or 4,4n have absolutely no effect in this pipeline since sort is given input that only contains one field.

The files in pipe1 and pipe2 are in alphabetic and (assuming the last part of the name before the .CSV is the date in YYYYMMDDhhmm[ss] format) in date order with file1 in increasing order and file2 in reverse order.

Note that if the commands:
Code:
FILENAME=$(ls ${DEST_LOCATION}/${FILES} | sort -r -k 4n | head -1)
ls ${DEST_LOCATION}/${FILES} | grep -v ${FILENAME} | xargs rm -rf"

do what you want, the commands:
Code:
last=""
for i in ${DEST_LOCATION}/${FILES}
do      if [ "$last" != "" ]
        then   rm "$last"
        fi
        last="$i"
done

or equivalently:
Code:
last="";for i in ${DEST_LOCATION}/${FILES};do if [ "$last" != "" ];then rm "$last";fi;last="$i";done

will do the same thing in the shell just using rm without needing to call ls twice, sort, head, grep, or xargs.

Note also that if
Code:
/data/projects/PROD_IPU/inbound/SOVEREIGN/SOVEREIGN_PreviousOld.CSV]

is present with the files listed in file1 and file2, all of the files shown in file1 and file2 will be removed just leaving
Code:
/data/projects/PROD_IPU/inbound/SOVEREIGN/SOVEREIGN_PreviousOld.CSV]

.
Last edited by Don Cragun; 01-21-2013 at 11:01 AM.. Reason: Change *.CSV* to the list actually used in the original scripts
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

sed command explanation needed

Hi, Could you please explain me the below statement -- phrase wise. sed -e :a -e '$q;N;'$cnt',$D;ba' abc.txt > xyz.txt if suppose $cnt contains value: 10 it copies last 9 lines of abc.txt to xyz.txt why it is copying last 9 rather than 10. and also what is ba and $D over there in... (4 Replies)
Discussion started by: subbukns
4 Replies

2. Shell Programming and Scripting

command line explanation

Hello everyone, I found this command line in a website: perl -pi.bak -we's/\z/Your new line\n/ if $. == 2;' your_text_file.txt With this command line you can insert a new line anywhere you want in a text without overwriting what's in it. -p causes perl to assume a loop around your... (4 Replies)
Discussion started by: goude
4 Replies

3. Shell Programming and Scripting

How to Sort Floating Numbers Using the Sort Command?

Hi to all. I'm trying to sort this with the Unix command sort. user1:12345678:3.5:2.5:8:1:2:3 user2:12345679:4.5:3.5:8:1:3:2 user3:12345687:5.5:2.5:6:1:3:2 user4:12345670:5.5:2.5:5:3:2:1 user5:12345671:2.5:5.5:7:2:3:1 I need to get this: user3:12345687:5.5:2.5:6:1:3:2... (7 Replies)
Discussion started by: daniel.gbaena
7 Replies

4. UNIX for Dummies Questions & Answers

SED command explanation

can someone please explain the below sed command.. sed 's/\(*|\)\(.*\)/\2\1/' (6 Replies)
Discussion started by: raghu_shekar
6 Replies

5. Shell Programming and Scripting

sed sorting command explanation

sed '$!N; /^\(.*\)\n\1$/!P; D' i found this file which removes duplicates irrespective for sorted or unsorted file. keep first occurance and remove the further occurances. can any1 explain how this is working.. i need to remove duplicates following file. duplicate criteria is not the... (3 Replies)
Discussion started by: mukeshguliao
3 Replies

6. UNIX for Advanced & Expert Users

command explanation

can anyone please tell me what does this expression means , i am under probation and need some explanation :) $AUDIT_DIR -type f -mtime +$AUDIT_EXPIRE \ -exec rm {} > /dev/null 2>&1 \; AUDIT_DIR="/var/log/" AUDIT_EXPIRE='30' Please use code tags! (4 Replies)
Discussion started by: semaan
4 Replies

7. Shell Programming and Scripting

Need explanation a of command in linux

Hi All I ran a script in Linux. In the script i have lines like && echo "Failed: Missing ${CM_ENV_FILE} \n" && return 1 . ${CM_ENV_FILE} Where CM_ENV_FILE = /data/ds/dpr_ebicm_uat//etl/cm3_0/entities/BBME/parameters/cm.env But its taking this path... (1 Reply)
Discussion started by: vee_789
1 Replies

8. Shell Programming and Scripting

sed command explanation

Will someone give me an explanation on how the sed command below works. sed 's/.*//' Thanks! (3 Replies)
Discussion started by: scj2012
3 Replies

9. UNIX for Beginners Questions & Answers

Explanation for sort utility and its detail

I tried to use sort utility and typed sort --help, read one of lines; its -k option, and part of it: I am really not getting it Anyone do me a useful favor to save me out of my ignorance ? Please use correct tags as required by forum rules! (1 Reply)
Discussion started by: abdulbadii
1 Replies

10. UNIX for Beginners Questions & Answers

Explanation of Nawk command

Hi Folks, I am struggling to understand nawk command which was used by another developer. Can you please explain what each character or string is doing here below: if ; then (3 Replies)
Discussion started by: kirans.229
3 Replies
All times are GMT -4. The time now is 01:03 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy