Sponsored Content
Top Forums Shell Programming and Scripting Remove lines of the same time stamp leaving the highest Post 302394522 by borderblaster on Thursday 11th of February 2010 05:29:31 PM
Old 02-11-2010
Remove lines of the same time stamp leaving the highest

Hi guys,

I have a log that looks like that below. Columns 2 3 4 5 6 7 is the date/time stamp separated by comma.
Code:
UUU,02,06,2010,10,00,00,00,0000000000000000,0000000000000000,0000000000001224
UUU,02,06,2010,10,05,00,00,0000000000000000,0000000000000000,0000000000001502
UUU,02,06,2010,10,10,00,00,0000000000000000,0000000000000000,0000000000001514
UUU,02,06,2010,10,10,00,00,0000000000000000,0000000000000000,0000000000001305
UUU,02,06,2010,10,10,00,00,0000000000000000,0000000000000000,0000000000001456
UUU,02,06,2010,10,15,00,00,0000000000000000,0000000000000000,0000000000001324

The problem is it can often contain entries for the same time stamp, I want to remove entries with the same time and just leave one entry that has the highest number in column 11.

So the above log would be outputed as:

Code:
UUU,02,06,2010,10,00,00,00,0000000000000000,0000000000000000,0000000000001224
UUU,02,06,2010,10,05,00,00,0000000000000000,0000000000000000,0000000000001502
UUU,02,06,2010,10,10,00,00,0000000000000000,0000000000000000,0000000000001514
UUU,02,06,2010,10,15,00,00,0000000000000000,0000000000000000,0000000000001324

Anyone know how to do it, it would be nice to be done in perl.
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Copy all the files with time stamp and remove header,trailer from file

All, I am new to unix and i have the following requirement. I have file(s) landing into input directory with timestamp, first i want to copy all these files into seperate directory then i want to rename these files without timestamp and also remove header,trailer from that file.. Could... (35 Replies)
Discussion started by: ksrams
35 Replies

2. Shell Programming and Scripting

remove a colon and number and leaving the rest

just have a file 1:2333 2:-09393 ]3:45453 4:-09999 5:-09933 6:93939 question is to get output by removing colons as well as number before each colon (in bold) 2333 -09393 45453 -09999 09933 93939 (5 Replies)
Discussion started by: cdfd123
5 Replies

3. Shell Programming and Scripting

shell script to remove all lines which exceeds a particular date & time

I have a text file which got 6th coloumn as date and 7th coloumn as time. The text contains data for last one week. I need to remove all the data whose date & time is after 03/08/2011 06:00:00 and save it on another file TEXT FILE ======== 6 dbclstr-b IXT_Web Memphis_Prod_SQL_Diff... (4 Replies)
Discussion started by: ajiwww
4 Replies

4. Shell Programming and Scripting

How to get time duration between two human readable time stamp in Unix?

Here is two time I have: Jul 12 16:02:01 Jul 13 01:02:01 and how can I do a simple match to get difference between two time which is 09:00:00 Thanks in advance. (3 Replies)
Discussion started by: ford99
3 Replies

5. UNIX for Advanced & Expert Users

In a huge file, Delete duplicate lines leaving unique lines

Hi All, I have a very huge file (4GB) which has duplicate lines. I want to delete duplicate lines leaving unique lines. Sort, uniq, awk '!x++' are not working as its running out of buffer space. I dont know if this works : I want to read each line of the File in a For Loop, and want to... (16 Replies)
Discussion started by: krishnix
16 Replies

6. Solaris

System time and Cron time stamp not matching

On Solaris 10 server the system date won't match with the timestamp on files created by a cron jobs, Please help here is what i get when i check for system date infodba-ie10ux014:/tcpdv1_ie10/tcadmin/bin\n\r-> date Tue Apr 24 15:27:43 GMT 2012at same time i executed a cron job, and checked... (4 Replies)
Discussion started by: karghum
4 Replies

7. Shell Programming and Scripting

Delete lines containing and remove the blank line at the same time

Is there a way to delete a line containing something and the blank line at the same time? If you do this it leaves a blank line behind. sed '/yum/d' .bash_historyI know this works but I would think there would be a way to do it with one command sed '/yum/d' .bash_history | sed '/^$/d'In... (2 Replies)
Discussion started by: cokedude
2 Replies

8. Shell Programming and Scripting

Leaving last 30 lines

If we want to display lines from file leaving last 30 lines. i dont know the count of lines in file (14 Replies)
Discussion started by: mirwasim
14 Replies

9. Shell Programming and Scripting

Collecting all lines between two time stamp from the log

Can you help me to collect the entire logs between two time stamp. The below awk command collecting the logs only if the line has time stamp. awk '$0>=from && $0<=to' from="150318 23:19:04" to="150318 23:55:04" log file 150318 23:19:04 logentries 150318 23:29:04 logentries 150318... (11 Replies)
Discussion started by: zenkarthi
11 Replies

10. Shell Programming and Scripting

Remove lines having older end time

Hi, In my bash script I want to add a code which remove all entries older than x days. To simplify this problem, I have divided into 3 parts. (2 parts are done looking answer for 3rd part.) To find the latest log date - Done Latest_Time=`find . -name '*.tps' -exec sed '/endTime/!d; s/{//;... (3 Replies)
Discussion started by: Agoyals1986
3 Replies
PCAP_LIST_TSTAMP_TYPES(3)				     Library Functions Manual					 PCAP_LIST_TSTAMP_TYPES(3)

NAME
pcap_list_tstamp_types, pcap_free_tstamp_types - get a list of time stamp types supported by a capture device, and free that list SYNOPSIS
#include <pcap/pcap.h> int pcap_list_tstamp_types(pcap_t *p, int **tstamp_typesp); void pcap_free_tstamp_types(int *tstamp_types); DESCRIPTION
pcap_list_tstamp_types() is used to get a list of the supported time stamp types of the interface associated with the pcap descriptor. pcap_list_tstamp_types() allocates an array to hold the list and sets *tstamp_typesp to point to the array. See pcap-tstamp(7) for a list of all the time stamp types. The caller is responsible for freeing the array with pcap_free_tstamp_types(), which frees the list pointed to by tstamp_types. RETURN VALUE
pcap_list_tstamp_types() returns the number of time stamp types in the array on success and PCAP_ERROR on failure. A return value of zero means that you cannot specify a time stamp type; you are limited to the capture device's default time stamp type. If PCAP_ERROR is returned, pcap_geterr() or pcap_perror() may be called with p as an argument to fetch or display the error text. SEE ALSO
pcap(3), pcap_geterr(3), pcap_tstamp_type_val_to_name(3), pcap-tstamp(7) 21 August 2010 PCAP_LIST_TSTAMP_TYPES(3)
All times are GMT -4. The time now is 11:50 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy