Sponsored Content
Special Forums UNIX and Linux Applications Infrastructure Monitoring Remove Similar Lines from a File Post 302134507 by Nysif Steve on Friday 31st of August 2007 08:03:06 AM
Old 08-31-2007
Remove Similar Lines from a File

I have a log file "logreport" that contains several lines as seen below:


04:20:00 /usr/lib/snmp/snmpdx: [ID 702911 daemon.error] Agent snmpd appeared dead but responded to ping
06:38:08 /usr/lib/snmp/snmpdx: [ID 702911 daemon.error] Agent snmpd appeared dead but responded to ping
07:11:05 /usr/lib/snmp/snmpdx: [ID 702911 daemon.error] Agent snmpd appeared dead but responded to ping


I would like to edit the report to remove entries that report duplicate events (event being the portion highlighted in red). I have no knowledge of what the events will be or how long they are. I am trying to produce an output close to what is seen below:


04:20:00 /usr/lib/snmp/snmpdx: [ID 702911 daemon.error] Agent snmpd appeared dead but responded to ping
This Error was reproduced 2 times
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How to sort a file and then print similar lines once

Hi! I have a trouble with the sort and the uniq. I know I have to use them, I just have trouble with putting them in the right order. I have a text file with unsorted lines (each line has a few words, the first word in the line is a number.). I need to sort this file in order to be... (6 Replies)
Discussion started by: shira
6 Replies

2. Infrastructure Monitoring

Remove Similar entries in a File

-------------------------------------------------------------------------------- I have a log file "logreport" that contains several lines as seen below: 04:20:00 /usr/lib/snmp/snmpdx: Agent snmpd appeared dead but responded to ping 06:38:08 /usr/lib/snmp/snmpdx: Agent snmpd appeared... (2 Replies)
Discussion started by: Nysif Steve
2 Replies

3. Shell Programming and Scripting

Counting similar lines from file UNIX

I have a file which contains data as below: nbk1j7o pageName=/jsp/RMBS/RMBSHome.jsf nbk1j7o pageName=/jsp/RMBS/RMBSHome.jsf nbk1j7o pageName=/jsp/RMBS/RMBSHome.jsf nbk1j7o pageName=/jsp/RMBS/RMBSHome.jsf nbk1j7o pageName=/jsp/common/index.jsf nbk1j7o pageName=/jsp/common/index.jsf nbk1wqe... (6 Replies)
Discussion started by: mohsin.quazi
6 Replies

4. Shell Programming and Scripting

remove one of each similar lines in a file

Hello folks I have a question for you gurus of sed or grep (maybe awk, but I would prefer the first two) I have a file (f1) that says: (actually, these are not numbers but md5sum, but for simplicity, let's assume these numbers.) 1 2 3 4 5And I have a file (f2) that says 1|a 1|b 1|c 2|d... (3 Replies)
Discussion started by: tukuyomi
3 Replies

5. UNIX for Dummies Questions & Answers

merge lines within a file that start with a similar pattern

Hello! i have a text file.. which contains the data as follows i want to merge the declarations lines pertaining to one datatype in to a single line as follows i've searched the forum for help.. but couldn't find much help.. how can i do this?? (1 Reply)
Discussion started by: a_ba
1 Replies

6. UNIX for Dummies Questions & Answers

Matching and reporting near-similar lines in a file

Hi, I have a file with the lines as below: C_10_A05_T7 C_10_A06_SP6 C_10_B05_SP6 C_10_B05_T7 C_10_B01_SP6 C_10_B01_T7 C_12_G07_SP6 C_12_G11_SP6 C_12_G11_T7 C_2_H18_T7 C_2_I02_SP6 C_2_I02_T7 C_2_I13_SP6 C_2_I17_SP6 The four segments of each line are connected by '_' symbols. I... (7 Replies)
Discussion started by: Fahmida
7 Replies

7. Shell Programming and Scripting

extracting lines from a file with similar first name

consider i have two files cat onlyviews1.sql CREATE VIEW V11 AS SELECT id, name, FROM etc etc WHERE etc etc; CREATE VIEW V22 AS SELECT id, name, FROM etc etc WHERE etc etc; CREATE VIEW V33 AS (10 Replies)
Discussion started by: vivek d r
10 Replies

8. Shell Programming and Scripting

removing lines with similar values from file

Hello, got a file with this structure: 33274 171030 02/29/2012 37897 P_GEH 2012-02-29 10:31:26 33275 171049 02/29/2012 38132 P_GEH 2012-02-29 10:35:27 33276 171058 02/29/2012 38515 P_GEH 2012-02-29 10:43:26 33277 170748 02/29/2012 40685 P_KOM ... (3 Replies)
Discussion started by: krecik28
3 Replies

9. Shell Programming and Scripting

Reducing text file using similar lines

Hello, I am a java programmer but want to try unix for a purpose where I need to reduce a file using its first field.. Here is the sample data: admin;2;0;; admission;8;0;; aman;1;0;; caroline;0;4;; cook;0;4;; cook;2;0;; far;0;3;; far;1;5;; I am explaining the dataset first. There... (5 Replies)
Discussion started by: shekhar2010us
5 Replies

10. UNIX for Dummies Questions & Answers

Want to remove all lines but not latest 50 lines from a file

Hi, I have a huge file which has Lacs of lines. File system got full. I want your guys help to suggest me a solution so that I can remove all lines from that file but not last 50,000 lines. I want solution which can remove lines from existing file so that I can have some space left with. (28 Replies)
Discussion started by: prashant2507198
28 Replies
SNMP_CONFIG(5)							     Net-SNMP							    SNMP_CONFIG(5)

NAME
snmp_config - handling of Net-SNMP configuration files DESCRIPTION
The Net-SNMP package uses various configuration files to configure its applications. This manual page merely describes the overall nature of them, so that the other manual pages don't have to. DIRECTORIES SEARCHED
First off, there are numerous places that configuration files can be found and read from. By default, the applications look for configura- tion files in the following 4 directories, in order: /etc/snmp, /usr/share/snmp, /usr/lib64/snmp, and $HOME/.snmp. In each of these direc- tories, it looks for files with the extension of both conf and local.conf (reading the second ones last). In this manner, there are 8 default places a configuration file can exist for any given configuration file type. Additionally, the above default search path can be overridden by setting the environment variable SNMPCONFPATH to a colon-separated list of directories to search for. The path for the persistent data should be included when running applications that use persistent storage, such as snmpd. Applications will read persistent configuration files in the following order of preference: file in SNMP_PERSISTENT_FILE environment variable directories in SNMPCONFPATH environment variable directory defined by persistentDir snmp.conf variable directory in SNMP_PERSISTENT_DIR environment variable default /var/lib/net-snmp directory Finally, applications will write persistent configuration files in the following order of preference: file in SNMP_PERSISTENT_FILE environment variable directory defined by persistentDir snmp.conf variable directory in SNMP_PERSISTENT_DIR environment variable default /var/lib/net-snmp directory Note: When using SNMP_PERSISTENT_FILE, the filename should match the application name. For example, /var/net-snmp/snmpd.conf. CONFIGURATION FILE TYPES
Each application may use multiple configuration files, which will configure various different aspects of the application. For instance, the SNMP agent (snmpd) knows how to understand configuration directives in both the snmpd.conf and the snmp.conf files. In fact, most applications understand how to read the contents of the snmp.conf files. Note, however, that configuration directives understood in one file may not be understood in another file. For further information, read the associated manual page with each configuration file type. Also, most of the applications support a -H switch on the command line that will list the configuration files it will look for and the directives in each one that it understands. The snmp.conf configuration file is intended to be a application suite wide configuration file that supports directives that are useful for controlling the fundamental nature of all of the SNMP applications, such as how they all manipulate and parse the textual SNMP MIB files. SWITCHING CONFIGURATION TYPES IN MID-FILE It's possible to switch in mid-file the configuration type that the parser is supposed to be reading. Since that sentence doesn't make much sense, lets give you an example: say that you wanted to turn on packet dumping output for the agent by default, but you didn't want to do that for the rest of the applications (ie, snmpget, snmpwalk, ...). Normally to enable packet dumping in the configuration file you'd need to put a line like: dumpPacket true into the snmp.conf file. But, this would turn it on for all of the applications. So, instead, you can put the same line in the snmpd.conf file so that it only applies to the snmpd daemon. However, you need to tell the parser to expect this line. You do this by putting a spe- cial type specification token inside a [] set. In other words, inside your snmpd.conf file you could put the above snmp.conf directive by adding a line like so: [snmp] dumpPacket true This tells the parser to parse the above line as if it were inside a snmp.conf file instead of an snmpd.conf file. If you want to parse a bunch of lines rather than just one then you can make the context switch apply to the remainder of the file or until the next context switch directive by putting the special token on a line by itself: # make this file handle snmp.conf tokens: [snmp] dumpPacket true logTimestamp true # return to our original snmpd.conf tokens: [snmpd] rocommunity mypublic COMMENTS
Any lines beginning with the character '#' in the configuration files are treated as a comment and are not parsed. API INTERFACE
Information about writing C code that makes use of this system in either the agent's MIB modules or in applications can be found in the read_config(3) manual page. SEE ALSO
snmpconf(1), read_config(3), snmp.conf(5), snmpd.conf(5) 4th Berkeley Distribution 5 May 2005 SNMP_CONFIG(5)
All times are GMT -4. The time now is 11:19 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy