Sponsored Content
Top Forums Shell Programming and Scripting how can I delete duplicates in the log? Post 302350027 by taekwondo on Wednesday 2nd of September 2009 02:45:13 PM
Old 09-02-2009
how can I delete duplicates in the log?

I have a log file and I am trying to run a script against it to search for key issues such as invalid users, errors etc. In one part, I grep for session closed and get a lot of the same thing,, ie. root username etc. I want to remove the multiple root and just have it do a count, like wc -l

These are the unidentified users
auth could not identify password for [henrys]: 1 Time(s)
auth could not identify password for [henrys]: 1 Time(s)
auth could not identify password for [henrys]: 2 Time(s)
auth could not identify password for [henrys]: 1 Time(s)
Maybe a simple variable tip?
or a for i loop?
Thanks, just learning scripting.
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

An interactive way to delete duplicates

1)I am trying to write a script that works interactively lists duplicated records on certain field/column and asks user to delete one or more. And finally it deletes all the records the used has asked for. I have an idea to store those line numbers in an array, not sure how to do this in... (3 Replies)
Discussion started by: chvs2000
3 Replies

2. Shell Programming and Scripting

How can i delete the duplicates based on one column of a line

I have my data something like this (08/03/2009 22:57:42.414)(:) king aaaaaaaaaaaaaaaa bbbbbbbbbbbbbbbbbbbbbb (08/03/2009 22:57:42.416)(:) John cccccccccccc cccccvssssssssss baaaaa (08/03/2009 22:57:42.417)(:) Michael ddddddd tststststtststts (08/03/2009 22:57:42.425)(:) Ravi... (11 Replies)
Discussion started by: rdhanek
11 Replies

3. Shell Programming and Scripting

Delete duplicates via script?

Hello, i have the following problem: there are two folders with a lot of files. Example: FolderA contains AAA, BBB, CCC FolderB contains DDD, EEE, AAA How can i via script identify AAA as duplicate in Folder B and delete it there? So that only DDD and EEE remain, in Folder B? Thank you... (16 Replies)
Discussion started by: Y-T
16 Replies

4. Shell Programming and Scripting

Delete Duplicates on the basis of two column values.

Hi All, i need ti delete two duplicate processss which are running on the same device type (column 1) and port ID (column 2). here is the sample data p1sc1m1 15517 11325 0 01:00:24 ? 0:00 scagntclsx25octtcp 2967 in3v mvmp01 0 8000 N S 969 750@751@752@ p1sc1m1 15519 11325 0 01:00:24 ? ... (5 Replies)
Discussion started by: neeraj617
5 Replies

5. Shell Programming and Scripting

Fastest way to delete duplicates from a large filelist.....

OK I have two filelists...... The first is formatted like this.... /path/to/the/actual/file/location/filename.jpg and has up to a million records The second list shows filename.jpg where there is more then on instance. and has maybe up to 65,000 records I want to copy files... (4 Replies)
Discussion started by: Bashingaway
4 Replies

6. Shell Programming and Scripting

delete from line and remove duplicates

My Input.....file1 ABCDE4435 Connected to 107.71.136.122 (SubNetwork=ONRM_RootMo_R SubNetwork=XYVLTN29CRBR99 MeContext=ABCDE4435 ManagedElement=1) ABCDE4478 Connected to 166.208.30.57 (SubNetwork=ONRM_RootMo_R SubNetwork=KLFMTN29CR0R04 MeContext=ABCDE4478 ManagedElement=1) ABCDE4478... (5 Replies)
Discussion started by: pareshkp
5 Replies

7. Shell Programming and Scripting

Search for duplicates and delete but remain the first one based on a specific pattern

Hi all, I have been trying to delete duplicates based on a certain pattern but failed to make it works. There are more than 1 pattern which are duplicated but i just want to remove 1 pattern only and remain the rest. I cannot use awk '!x++' inputfile.txt or sed '/pattern/d' or use uniq and sort... (7 Replies)
Discussion started by: redse171
7 Replies

8. Shell Programming and Scripting

Delete duplicates in CA bundle

I do have a big CA bundle certificate file and each time if i get request to add new certificate to the existing bundle i need to make sure it is not present already. How i can validate the duplicates. The alignment of the certificate within the bundle seems to be different. Example: Cert 1... (7 Replies)
Discussion started by: diva_thilak
7 Replies

9. Shell Programming and Scripting

Delete only if duplicates found in each record

Hi, i have another problem. I have been trying to solve it by myself but failed. inputfile ;; ID T08578 NAME T08578 SBASE 30696 EBASE 32083 TYPE P func just test func chronology func cholesterol func null INT 30765-37333 INT 37154-37318 Link 5546 Link 8142 (4 Replies)
Discussion started by: redse171
4 Replies

10. Shell Programming and Scripting

To Delete the duplicates using Part of File Name

I am using the below script to delete duplicate files but it is not working for directories with more than 10k files "Argument is too long" is getting for ls -t. Tried to replace ls -t with find . -type f \( -iname "*.xml" \) -printf '%T@ %p\n' | sort -rg | sed -r 's/* //' | awk... (8 Replies)
Discussion started by: gold2k8
8 Replies
asadmin-delete-auth-realm(1AS)					   User Commands				    asadmin-delete-auth-realm(1AS)

NAME
asadmin-delete-auth-realm, delete-auth-realm - removes the named authorized realm SYNOPSIS
delete-auth-realm --user admin_user [--password admin_password] [--host localhost] [--port 4848] [--secure|-s] [--passwordfile filename] [--terse=false] [--echo=false] [--interactive=true] auth_realm_name Removes the named authorized realm. This command is supported in remote mode only. OPTIONS
--user authorized domain application server administrative username. --password password to administer the domain application server. --host machine name where the domain application server is running. --port port number of the domain application server listening for administration requests. --secure if true, uses SSL/TLS to communicate with the domain application server. --passwordfile file containing the domain application server password. --terse indicates that any output data must be very concise, typically avoiding human-friendly sentences and favoring well- formatted data for consumption by a script. Default is false. --echo setting to true will echo the command line statement on the standard output. Default is false. --interactive if set to true (default), only the required password options are prompted. OPERANDS
auth_realm_name name of this realm. Example 1: Using delete-auth-realm asadmin> delete-auth-realm --user admin1 --passwordfile password --host pigeon --port 5001 db Command delete-auth-realm executed successfully Where db is the auth realm deleted. EXIT STATUS
0 command executed successfully 1 error in executing the command asadmin-create-auth-realm(1AS), asadmin-list-auth-realms(1AS) J2EE 1.4 SDK March 2004 asadmin-delete-auth-realm(1AS)
All times are GMT -4. The time now is 02:22 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy