Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Help in writing a KSH script to filter the latest record?

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Help in writing a KSH script to filter the latest record?
# 1  
Old 02-05-2010
Help in writing a KSH script to filter the latest record?
Hi All,

I have a text file with the folowing content.

Code:
BANGALORE|1417|2010-02-04 08:41:04.174|dob|xxx
BANGALORE|1416|2010-02-04 08:23:19.566|dob|yyy
BANGALORE|1415|2010-02-04 08:20:14.497|dob|aaa
BANGALORE|1414|2010-02-04 08:19:40.065|dob|vvv
BANGALORE|1413|2010-02-04 08:16:29.817|dob|zzz
BANGALORE|1413|2010-02-04 08:16:29.817|dob|www
BANGALORE|1413|2010-02-04 08:16:29.817|dob|rrr
BANGALORE|1413|2010-02-04 08:16:29.817|dob|JJJ
BANGALORE|1413|2010-02-04 08:16:29.817|dob|kkk
BANGALORE|1413|2010-02-04 08:16:29.817|dob|mmm
BANGALORE|1419|2010-02-04 08:17:29.817|dob|zzz
BANGALORE|1413|2010-02-04 08:16:29.817|dob|iii
BANGALORE|1413|2010-02-04 08:16:29.817|dob|nnn
BANGALORE|1418|2010-02-05 08:19:40.065|dob|vvv
BANGALORE|1413|2010-02-04 08:16:29.817|dob|ppp
BANGALORE|1413|2010-02-04 08:16:29.817|dob|ooo
BANGALORE|1429|2010-02-04 08:11:29.817|dob|zzz

I want to extract the latest (The third column can be used to identify the latest record) records in the above file for each name (the last field).
Finaly the output should be like this.

Code:
BANGALORE|1415|2010-02-04 08:20:14.497|dob|aaa
BANGALORE|1413|2010-02-04 08:16:29.817|dob|iii
BANGALORE|1413|2010-02-04 08:16:29.817|dob|jjj
BANGALORE|1413|2010-02-04 08:16:29.817|dob|kkk
BANGALORE|1413|2010-02-04 08:16:29.817|dob|mmm
BANGALORE|1413|2010-02-04 08:16:29.817|dob|nnn
BANGALORE|1413|2010-02-04 08:16:29.817|dob|ooo
BANGALORE|1413|2010-02-04 08:16:29.817|dob|ppp
BANGALORE|1413|2010-02-04 08:16:29.817|dob|rrr
BANGALORE|1418|2010-02-05 08:19:40.065|dob|vvv
BANGALORE|1413|2010-02-04 08:16:29.817|dob|www
BANGALORE|1417|2010-02-04 08:41:04.174|dob|xxx
BANGALORE|1416|2010-02-04 08:23:19.566|dob|yyy
BANGALORE|1419|2010-02-04 08:17:29.817|dob|zzz

Can any one help me to write the unix shell script to get this output?

Thanks in advance.
Karpak
Last edited by Scott; 02-05-2010 at 04:44 AM.. Reason: Fixed code tags
# 2  
Old 02-05-2010
Since the records appear to be time sorted, you could use something like this:
Code:
awk -F '|' 'END{for(i in A)print A[i]}{A[$NF]=$0}' infile | sort -t'|' -k5,5

# 3  
Old 02-05-2010
hello my friend Scrutinizer,
i test there may be sth wrong with your script because the third field is not sorted.

Quote:
sort -t"|" +2 yourfile | awk -F"|" 'BEGIN{arr[""]="";}{arr[$5]=$0;}END{for(i in arr){if(arr[i]!="")print arr[i];}}' | sort -t"|" -f +4
# 4  
Old 02-05-2010
If the dates are not sorted and if you have GNU date than you could this I suppose:
Code:
#!/bin/ksh
typeset -A A
typeset -A D
while IFS="|" read city field2 tdate field4 key ; do
  newdate=$(date -d "$tdate" '+%s')
  if [ $newdate -gt "${D[$key]}" ]; then
    A[$key]="$city|$field2|$tdate|$field4|$key"
    D[$key]=$newdate
  fi
done < infile126
for i in "${A[@]}"; do
  echo "$i"
done|sort -t'|' -k5,5

# 5  
Old 02-05-2010
Thank you very much
Dear Scrutinizer,

Based on data with slight modification. Your logic works fine.

Thank you very much

Karpak
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Display latest record from file based on multiple columns combination

I have requirement to print latest record from file based on multiple columns combination. EWAPE EW1SLE0000 EW1SOMU01 ABORTED 03/16/2015 100004 03/16/2015 100005 001 EWAPE EW1SLE0000 EW1SOMU01 ABORTED 03/18/2015 140003 03/18/2015 140004 001 EWAPE EW1SLE0000 EW1SOMU01 ABORTED 03/18/2015 220006... (1 Reply)
Discussion started by: tmalik79
1 Replies

2. Shell Programming and Scripting

Shell Script (ksh) - SQLPlus query filter using a string variable

Using ksh, I am using SQLPlus to execute a query with a filter using a string variable. REPO_DB=DEV1 FOLDER_NM='U_nmalencia' FOLDER_CHECK=$(sqlplus -s /nolog <<EOF CONNECT user/pswd_select@${REPO_DB} set echo off heading off feedback off select subj_name from subject where... (5 Replies)
Discussion started by: nkm0brm
5 Replies

3. Shell Programming and Scripting

Finding the Latest record

Dear All, I have getting data as follows, the second field signifies table name and last one is time stamp. I have return always latest record based on time stamp. Could you please help me ? I/P ==== ... (1 Reply)
Discussion started by: srikanth38
1 Replies

4. Shell Programming and Scripting

Delete record filter by column

Dear friend, I have a file 2 files with column wise FILE_A ------------------------------ x,1,@ y,3,$ x,5,% FILE_B -------------------- x,1,@ i like to delete the all lines in FILE_A ,if first column available in FILE_B. output (in FILE_A) y,3,$ x,5,% (10 Replies)
Discussion started by: Jewel
10 Replies

5. Shell Programming and Scripting

awk-filter record by another file

I have file1 3049 3138 4672 22631 45324 112382 121240 125470 130289 186128 193996 194002 202776 228002 253221 273523 284601 284605 641858 (8 Replies)
Discussion started by: biomed
8 Replies

6. Shell Programming and Scripting

Need a ksh script for adding the space at the end of record in a flat file

Hi, I need a ksh script for the below requirement: i have a Delimited flat file with 200 records delimiter is '|~|' i need a script to insert space at the end if the record is ending with delimiter '|~|' if it didnt end with delimiter it should not append space. Example: ram|~|2|~| ... (16 Replies)
Discussion started by: srikanth_sagi
16 Replies

7. UNIX for Dummies Questions & Answers

Select the latest dated file-ksh script

Hello all!! I am new here and new in scripting! I want to write a ksh script to select the most recent file from a dir and use it in a variable. I have a directory with files named like: YYYMMDD A basic idea of the script I want to write is #!/usr/bin/ksh latest= latest_dated_file at... (2 Replies)
Discussion started by: chris_euop
2 Replies

8. Shell Programming and Scripting

Filter record from a file

Reposting since I didnt not get any reply. I have a problem while filtering records from a file. Can somebody help please? For eg: Consider the below files Record file: 0003@00000000000190@20100401@201004012010040120100401@003@... (1 Reply)
Discussion started by: gpaulose
1 Replies

9. Shell Programming and Scripting

filter the uniq record problem

Anyone can help for filter the uniq record for below example? Thank you very much Input file 20090503011111|test|abc 20090503011112|tet1|abc|def 20090503011112|test1|bcd|def 20090503011131|abc|abc 20090503011131|bbc|bcd 20090503011152|bcd|abc 20090503011151|abc|abc... (8 Replies)
Discussion started by: bleach8578
8 Replies

10. Shell Programming and Scripting

filter and get the latest order number

hello, how can I filter and get the latest order number (last five digits) below: input file: johnmm00001 maryyy00121 johnm100222 johnmm00003 maryyy00122 output file: johnmm00003 maryyy00122 johnm100222 (6 Replies)
Discussion started by: happyv
6 Replies
Login or Register to Ask a Question