Sponsored Content
Top Forums Shell Programming and Scripting how can I delete duplicates in the log? Post 302350031 by taekwondo on Wednesday 2nd of September 2009 03:09:48 PM
Old 09-02-2009
sort -u worked great. Thanks so much!

---------- Post updated at 03:09 PM ---------- Previous update was at 03:02 PM ----------

why do I need to cat the file? I am trying to format a log so that the problems come out such as errors, invalid users, illegal, authentication failures etc. are in the report. I would like to clean it up as well, so that you don't have 100 entries if a single user had issues. Ideally it would do a count first, then may be state the user tried to login 10 times for e.g. Thanks V.
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

An interactive way to delete duplicates

1)I am trying to write a script that works interactively lists duplicated records on certain field/column and asks user to delete one or more. And finally it deletes all the records the used has asked for. I have an idea to store those line numbers in an array, not sure how to do this in... (3 Replies)
Discussion started by: chvs2000
3 Replies

2. Shell Programming and Scripting

How can i delete the duplicates based on one column of a line

I have my data something like this (08/03/2009 22:57:42.414)(:) king aaaaaaaaaaaaaaaa bbbbbbbbbbbbbbbbbbbbbb (08/03/2009 22:57:42.416)(:) John cccccccccccc cccccvssssssssss baaaaa (08/03/2009 22:57:42.417)(:) Michael ddddddd tststststtststts (08/03/2009 22:57:42.425)(:) Ravi... (11 Replies)
Discussion started by: rdhanek
11 Replies

3. Shell Programming and Scripting

Delete duplicates via script?

Hello, i have the following problem: there are two folders with a lot of files. Example: FolderA contains AAA, BBB, CCC FolderB contains DDD, EEE, AAA How can i via script identify AAA as duplicate in Folder B and delete it there? So that only DDD and EEE remain, in Folder B? Thank you... (16 Replies)
Discussion started by: Y-T
16 Replies

4. Shell Programming and Scripting

Delete Duplicates on the basis of two column values.

Hi All, i need ti delete two duplicate processss which are running on the same device type (column 1) and port ID (column 2). here is the sample data p1sc1m1 15517 11325 0 01:00:24 ? 0:00 scagntclsx25octtcp 2967 in3v mvmp01 0 8000 N S 969 750@751@752@ p1sc1m1 15519 11325 0 01:00:24 ? ... (5 Replies)
Discussion started by: neeraj617
5 Replies

5. Shell Programming and Scripting

Fastest way to delete duplicates from a large filelist.....

OK I have two filelists...... The first is formatted like this.... /path/to/the/actual/file/location/filename.jpg and has up to a million records The second list shows filename.jpg where there is more then on instance. and has maybe up to 65,000 records I want to copy files... (4 Replies)
Discussion started by: Bashingaway
4 Replies

6. Shell Programming and Scripting

delete from line and remove duplicates

My Input.....file1 ABCDE4435 Connected to 107.71.136.122 (SubNetwork=ONRM_RootMo_R SubNetwork=XYVLTN29CRBR99 MeContext=ABCDE4435 ManagedElement=1) ABCDE4478 Connected to 166.208.30.57 (SubNetwork=ONRM_RootMo_R SubNetwork=KLFMTN29CR0R04 MeContext=ABCDE4478 ManagedElement=1) ABCDE4478... (5 Replies)
Discussion started by: pareshkp
5 Replies

7. Shell Programming and Scripting

Search for duplicates and delete but remain the first one based on a specific pattern

Hi all, I have been trying to delete duplicates based on a certain pattern but failed to make it works. There are more than 1 pattern which are duplicated but i just want to remove 1 pattern only and remain the rest. I cannot use awk '!x++' inputfile.txt or sed '/pattern/d' or use uniq and sort... (7 Replies)
Discussion started by: redse171
7 Replies

8. Shell Programming and Scripting

Delete duplicates in CA bundle

I do have a big CA bundle certificate file and each time if i get request to add new certificate to the existing bundle i need to make sure it is not present already. How i can validate the duplicates. The alignment of the certificate within the bundle seems to be different. Example: Cert 1... (7 Replies)
Discussion started by: diva_thilak
7 Replies

9. Shell Programming and Scripting

Delete only if duplicates found in each record

Hi, i have another problem. I have been trying to solve it by myself but failed. inputfile ;; ID T08578 NAME T08578 SBASE 30696 EBASE 32083 TYPE P func just test func chronology func cholesterol func null INT 30765-37333 INT 37154-37318 Link 5546 Link 8142 (4 Replies)
Discussion started by: redse171
4 Replies

10. Shell Programming and Scripting

To Delete the duplicates using Part of File Name

I am using the below script to delete duplicate files but it is not working for directories with more than 10k files "Argument is too long" is getting for ls -t. Tried to replace ls -t with find . -type f \( -iname "*.xml" \) -printf '%T@ %p\n' | sort -rg | sed -r 's/* //' | awk... (8 Replies)
Discussion started by: gold2k8
8 Replies
cmannotifyd(8)						     CMAN Notification Daemon						    cmannotifyd(8)

NAME
cmannotifyd - CMAN Notification Daemon SYNOPSIS
cmannotifyd [-f] [-d] DESCRIPTION
The cmannotifyd daemon talks to CMAN and provides a mechanism to notify external entities about cluster changes. CMAN dispatches 3 kind of notifications: - CMAN_REASON_TRY_SHUTDOWN when cman requests to all clients if it is allowed to shutdown. - CMAN_REASON_STATECHANGE when cman detects a node joining or leaving the cluster. - CMAN_REASON_CONFIG_UPDATE when a configuration change event has been detected/requested. These notifications are then dispatched to the shell script cman_notify in the environment variable CMAN_NOTIFICATION. cman_notify will then execute all the scripts in the configured notification directory (default: /etc/cluster/cman-notify.d) passing a very minimal set of envvars including, of course, the CMAN_NOTIFICATION= type. The execution order is set by the filename as shown by "LC_ALL=C ls -las". cmannotifyd logs are stored in the default log file (/var/log/cluster/cmannotifyd.log). cman_notify logs are stored in the default log file (/var/log/cluster/cman_notify.log). By default the output from the scripts executed by cman_notify is redirected to /dev/null. Users can either set CMAN_NOTIFICATION_DEBUG=1 in their environment or set proper debug configura- tion in cluster.conf to redirect scripts output to the cman_notify log file. NOTES
cmannotifyd does not block on cman_notify nor check the exit status of the script. Notifications are dispatched in the same order as they arrive, one by one. CMAN_REASON_TRY_SHUTDOWN is passed to scripts for information only, they can not influence cman's decsion about whether or not to shut down. CMAN_REASON_STATECHANGE also implies CMAN_NOTIFICATION_QUORUM exported in the environment. CMAN_NOTIFICATION_QUORUM will be set to 1 (when the node is part of a quorate cluster) or 0 (otherwise). A template for cman_notify scripts can be found in the doc/ directory. OPTIONS
-f Run in the foreground (do not fork / daemonize). -d Enable debug output. November 2008 cmannotifyd(8)
All times are GMT -4. The time now is 08:03 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy