Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Sort data by date and then search by column

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Sort data by date and then search by column
# 1  
Old 05-16-2013
Sort data by date and then search by column
Hi,
I have a file where data is pipe separated.First i want to sort the file content by date . Then i want to pick up the records based on the first column which should be unique and not have duplicates.

NYSE|yyyrrrddd|toronto|isin|ticker|2013-05-15
UKS|yyyrrrddd|toronto|isin|ticker|2013-05-14
BSE|yyyrrrddd|toronto|isin|ticker|2013-05-13
HKS|yyyrrrddd|toronto|isin|ticker|2013-05-14
TSX|yyyrrrddd|toronto|isin|ticker|2013-05-16
NYSE|yyyrrrddd|toronto|isin|ticker|2013-05-15
BSE|yyyrrrddd|toronto|isin|ticker|2013-05-15

First i would like to sort by date and get the latest ones. Secondly there should be no duplicates and hence my output should be like this :

TSX|yyyrrrddd|toronto|isin|ticker|2013-05-16
NYSE|yyyrrrddd|toronto|isin|ticker|2013-05-15
BSE|yyyrrrddd|toronto|isin|ticker|2013-05-15
UKS|yyyrrrddd|toronto|isin|ticker|2013-05-14
UKS|yyyrrrddd|toronto|isin|ticker|2013-05-14

Any help with the relevant script is requested.
# 2  
Old 05-17-2013
You can sort the file to get the order you want and remove duplicates:

Code:
$ cat t
NYSE|yyyrrrddd|toronto|isin|ticker|2013-05-15
UKS|yyyrrrddd|toronto|isin|ticker|2013-05-14
BSE|yyyrrrddd|toronto|isin|ticker|2013-05-13
HKS|yyyrrrddd|toronto|isin|ticker|2013-05-14
TSX|yyyrrrddd|toronto|isin|ticker|2013-05-16
NYSE|yyyrrrddd|toronto|isin|ticker|2013-05-15
BSE|yyyrrrddd|toronto|isin|ticker|2013-05-15

$ sort -u -t'|' -k6,6r -k1,1 t
TSX|yyyrrrddd|toronto|isin|ticker|2013-05-16
BSE|yyyrrrddd|toronto|isin|ticker|2013-05-15
NYSE|yyyrrrddd|toronto|isin|ticker|2013-05-15
HKS|yyyrrrddd|toronto|isin|ticker|2013-05-14
UKS|yyyrrrddd|toronto|isin|ticker|2013-05-14
BSE|yyyrrrddd|toronto|isin|ticker|2013-05-13


What else do you want to do?
# 3  
Old 05-17-2013
I have kept the command you mentioned inside a .sh file and upon execution i got the below 2 message:
Code:
cat: cannot open t
sort: can't stat t: No such file or directory

1) below is my .sh file content where abc.psv is file which has the | data
Code:


Code:



cat t /common/PDS/abc.psv
sort -u -t'|' -k6,6r -k1,1 t /common/PDS/abc.psv

2) the abc.psv file is kept in the path /common/PDS/
Last edited by Franklin52; 05-18-2013 at 10:09 AM.. Reason: Please use code tags
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

How do I sort data in column properly?

Can i use 'column' command to get the required 3rd column output? Input example: 1 2 345678 90 2 2 356 42 3 3 8265 55 Output required: 1 2 345678 90 2 2 356 42 3 3 8265 55 Basically i want the 3rd column to be justified to the right, instead of left.... (3 Replies)
Discussion started by: nurul_nadzirah
3 Replies

2. Shell Programming and Scripting

Help with sort only column 2 data separately

Input File Contig_1_294435nt 242231 242751 Contig_1_294435nt 242390 242782 Contig_1_294435nt 242390 242782 Contig_1_294435nt 291578 291668 Contig_2_242278nt 75910 76271 Contig_2_242278nt 76036 76316 Contig_2_242278nt 76036 76316... (2 Replies)
Discussion started by: perl_beginner
2 Replies

3. Shell Programming and Scripting

sed --> sort data by date

Hi, i "tried" to sort data by date. So far, i used sed to take the data from the last and the actual month. Now, after changing the year it is not working properly. i use: GNU bash, version 4.2.45(1)-release (x86_64-suse-linux-gnu) sed -n '/\//p' $Home/../scripte/pd_0.txt y is a... (6 Replies)
Discussion started by: IMPe
6 Replies

4. Shell Programming and Scripting

Sort, sed, and zero padding date column csv bash scripting

Hello people, I am having problem to sort, sed and zero padding of column in csv file. 7th column only. Input of csv file: 1,2,3,4,5,6,4/1/2010 12:00 AM,8 1,2,3,4,5,6,3/11/2010 9:39 AM,8 1,2,3,4,5,6,5/12/2011 3:43 PM,8 1,2,3,4,5,6,12/20/2009 7:23 PM,8 Output:... (5 Replies)
Discussion started by: sean1357
5 Replies

5. Shell Programming and Scripting

Sort data by date first and then remove duplicates

Hi , I have below data inside a file named ref.psv . I want to create a shell script which will do the below 2 points : (1) sort the file content first based on the latest date which is the last column in the file (actual file its the 175th column) (2)after sorting the file based on latest date... (3 Replies)
Discussion started by: samrat dutta
3 Replies

6. Shell Programming and Scripting

Advanced: Sort, count data in column, append file name

Hi. I am not sure the title gives an optimal description of what I want to do. Also, I tried to post this in the "UNIX for Dummies Questions & Answers", but it seems no-one was able to help out. I have several text files that contain data in many columns. All the files are organized the same... (14 Replies)
Discussion started by: JamesT
14 Replies

7. Shell Programming and Scripting

Sort data As per first Column

hI I have file A NSU30504 5 6 G 6 NSU3050B T 7 9 J NSU30506 T I 8 9 NSU3050C H J K L Output: NSU3050B T 7 9 J NSU3050C H J K L NSU30504 5 6 G 6 NSU30506 T I 8 9Video tutorial on how to use code tags in The UNIX and Linux Forums. (13 Replies)
Discussion started by: pareshkp
13 Replies

8. Shell Programming and Scripting

Sort data from column to row

Hi, I need somebody's help with sorting data with awk. I've got a file: 10 aaa 4584 12 bbb 6138 20 ccc 4417 21 ddd 7796 10 eee 7484 12 fff ... (5 Replies)
Discussion started by: killerbee
5 Replies

9. Programming

sort data by date.. pls help

hi all, could anyone help me? I need to query output by compare dates from 2 table and i'm using a UNION query..and wanted to sort the output by date.. My query like this: SELECT TO_CHAR(DATE) DATE1, INVOICE FROM ACCOUNT1 WHERE DATE < (to_date('122003','MMYYYY')) UNION ... (2 Replies)
Discussion started by: kate katherine
2 Replies

10. Shell Programming and Scripting

Search max value in a column in a file instead of sort

Hi Everyone, 1.txt 00:00:00 0 0 0 0 0 0 0 00:00:01 0 0 0 2 1 33 2 00:00:02 5 0 0 0 0 0 0 00:00:03 0 4 0 0 0 0 0... (5 Replies)
Discussion started by: jimmy_y
5 Replies
Login or Register to Ask a Question