Sponsored Content
Top Forums Shell Programming and Scripting Remove lines of the same time stamp leaving the highest Post 302394522 by borderblaster on Thursday 11th of February 2010 05:29:31 PM
Old 02-11-2010
Remove lines of the same time stamp leaving the highest

Hi guys,

I have a log that looks like that below. Columns 2 3 4 5 6 7 is the date/time stamp separated by comma.
Code:
UUU,02,06,2010,10,00,00,00,0000000000000000,0000000000000000,0000000000001224
UUU,02,06,2010,10,05,00,00,0000000000000000,0000000000000000,0000000000001502
UUU,02,06,2010,10,10,00,00,0000000000000000,0000000000000000,0000000000001514
UUU,02,06,2010,10,10,00,00,0000000000000000,0000000000000000,0000000000001305
UUU,02,06,2010,10,10,00,00,0000000000000000,0000000000000000,0000000000001456
UUU,02,06,2010,10,15,00,00,0000000000000000,0000000000000000,0000000000001324

The problem is it can often contain entries for the same time stamp, I want to remove entries with the same time and just leave one entry that has the highest number in column 11.

So the above log would be outputed as:

Code:
UUU,02,06,2010,10,00,00,00,0000000000000000,0000000000000000,0000000000001224
UUU,02,06,2010,10,05,00,00,0000000000000000,0000000000000000,0000000000001502
UUU,02,06,2010,10,10,00,00,0000000000000000,0000000000000000,0000000000001514
UUU,02,06,2010,10,15,00,00,0000000000000000,0000000000000000,0000000000001324

Anyone know how to do it, it would be nice to be done in perl.
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Copy all the files with time stamp and remove header,trailer from file

All, I am new to unix and i have the following requirement. I have file(s) landing into input directory with timestamp, first i want to copy all these files into seperate directory then i want to rename these files without timestamp and also remove header,trailer from that file.. Could... (35 Replies)
Discussion started by: ksrams
35 Replies

2. Shell Programming and Scripting

remove a colon and number and leaving the rest

just have a file 1:2333 2:-09393 ]3:45453 4:-09999 5:-09933 6:93939 question is to get output by removing colons as well as number before each colon (in bold) 2333 -09393 45453 -09999 09933 93939 (5 Replies)
Discussion started by: cdfd123
5 Replies

3. Shell Programming and Scripting

shell script to remove all lines which exceeds a particular date & time

I have a text file which got 6th coloumn as date and 7th coloumn as time. The text contains data for last one week. I need to remove all the data whose date & time is after 03/08/2011 06:00:00 and save it on another file TEXT FILE ======== 6 dbclstr-b IXT_Web Memphis_Prod_SQL_Diff... (4 Replies)
Discussion started by: ajiwww
4 Replies

4. Shell Programming and Scripting

How to get time duration between two human readable time stamp in Unix?

Here is two time I have: Jul 12 16:02:01 Jul 13 01:02:01 and how can I do a simple match to get difference between two time which is 09:00:00 Thanks in advance. (3 Replies)
Discussion started by: ford99
3 Replies

5. UNIX for Advanced & Expert Users

In a huge file, Delete duplicate lines leaving unique lines

Hi All, I have a very huge file (4GB) which has duplicate lines. I want to delete duplicate lines leaving unique lines. Sort, uniq, awk '!x++' are not working as its running out of buffer space. I dont know if this works : I want to read each line of the File in a For Loop, and want to... (16 Replies)
Discussion started by: krishnix
16 Replies

6. Solaris

System time and Cron time stamp not matching

On Solaris 10 server the system date won't match with the timestamp on files created by a cron jobs, Please help here is what i get when i check for system date infodba-ie10ux014:/tcpdv1_ie10/tcadmin/bin\n\r-> date Tue Apr 24 15:27:43 GMT 2012at same time i executed a cron job, and checked... (4 Replies)
Discussion started by: karghum
4 Replies

7. Shell Programming and Scripting

Delete lines containing and remove the blank line at the same time

Is there a way to delete a line containing something and the blank line at the same time? If you do this it leaves a blank line behind. sed '/yum/d' .bash_historyI know this works but I would think there would be a way to do it with one command sed '/yum/d' .bash_history | sed '/^$/d'In... (2 Replies)
Discussion started by: cokedude
2 Replies

8. Shell Programming and Scripting

Leaving last 30 lines

If we want to display lines from file leaving last 30 lines. i dont know the count of lines in file (14 Replies)
Discussion started by: mirwasim
14 Replies

9. Shell Programming and Scripting

Collecting all lines between two time stamp from the log

Can you help me to collect the entire logs between two time stamp. The below awk command collecting the logs only if the line has time stamp. awk '$0>=from && $0<=to' from="150318 23:19:04" to="150318 23:55:04" log file 150318 23:19:04 logentries 150318 23:29:04 logentries 150318... (11 Replies)
Discussion started by: zenkarthi
11 Replies

10. Shell Programming and Scripting

Remove lines having older end time

Hi, In my bash script I want to add a code which remove all entries older than x days. To simplify this problem, I have divided into 3 parts. (2 parts are done looking answer for 3rd part.) To find the latest log date - Done Latest_Time=`find . -name '*.tps' -exec sed '/endTime/!d; s/{//;... (3 Replies)
Discussion started by: Agoyals1986
3 Replies
CAPSH(1)							   User Commands							  CAPSH(1)

NAME
capsh - capability shell wrapper SYNOPSIS
capsh [OPTION]... DESCRIPTION
Linux capability support and use can be explored and constrained with this tool. This tool provides a handy wrapper for certain types of capability testing and environment creation. It also provides some debugging features useful for summarizing capability state. OPTIONS
The tool takes a number of optional arguments, acting on them in the order they are provided. They are as follows: --print Display prevailing capability and related state. -- [args] Execute /bin/bash with trailing arguments. Note, you can use -c 'command to execute' for specific commands. == Execute capsh again with remaining arguments. Useful for testing exec() behavior. --caps=cap-set Set the prevailing process capabilities to those specified by cap-set. Where cap-set is a text-representation of capability state as per cap_from_text(3). --drop=cap-list Remove the listed capabilities from the prevailing bounding set. The capabilities are a comma separated list of capa- bilities as recognized by the cap_from_name(3) function. Use of this feature requires that the capsh program is oper- ating with CAP_SETPCAP in its effective set. --inh=cap-list Set the inheritable set of capabilities for the current process to equal those provided in the comma separated list. For this action to succeed, the prevailing process should already have each of these capabilities in the union of the current inheritable and permitted capability sets, or the capsh program is operating with CAP_SETPCAP in its effec- tive set. --user=username Assume the identity of the named user. That is, look up the user's uid and gid with getpwuid(3) and their group mem- berships with getgrouplist(3) and set them all. --uid=id Force all uid values to equal id using the setuid(2) system call. --gid=<id> Force all gid values to equal id using the setgid(2) system call. --groups=<id-list> Set the supplementary groups to the numerical list provided. The groups are set with the setgroups(2) system call. --keep=<0|1> In a non-pure capability mode, the kernel provides liberal privilege to the super-user. However, it is normally the case that when the super-user changes uid to some lesser user, then capabilities are dropped. For these situations, the kernel can permit the process to retain its capabilities after a setuid(2) system call. This feature is known as keep-caps support. The way to activate it using this script is with this argument. Setting the value to 1 will cause keep-caps to be active. Setting it to 0 will cause keep-caps to deactivate for the current process. In all cases, keep-caps is deactivated when an exec() is performed. See --secbits for ways to disable this feature. --secbits=N XXX - need to document this feature. --chroot=path Execute the chroot(2) system call with the new root-directory (/) equal to path. This operation requires CAP_SYS_CHROOT to be in effect. --forkfor=sec --killit=sig --decode=N This is a convenience feature. If you look at /proc/1/status there are some capability related fields of the follow- ing form: CapInh: 0000000000000000 CapPrm: ffffffffffffffff CapEff: fffffffffffffeff CapBnd: ffffffffffffffff This option provides a quick way to decode a capability vector represented in this form. For example, the missing capability from this effective set is 0x0100. By running: capsh --decode=0x0100 we observe that the missing capability is: cap_setpcap. --supports=xxx As the kernel evolves, more capabilities are added. This option can be used to verify the existence of a capability on the system. For example, --supports=cap_syslog will cause capsh to promptly exit with a status of 1 when run on kernel 2.6.27. However, when run on kernel 2.6.38 it will silently succeed. EXIT STATUS Following successful execution the tool exits with status 0. Following an error, the tool immediately exits with status 1. AUTHOR
Written by Andrew G. Morgan <morgan@kernel.org>. REPORTING BUGS
Please report bugs to the author. SEE ALSO
libcap(3), getcap(8),setcap(8) and capabilities(7). libcap 2 2011-04-24 CAPSH(1)
All times are GMT -4. The time now is 04:15 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy