Top Forums UNIX for Dummies Questions & Answers Remove Duplicate lines from File Post 302134388 by radoulov on Thursday 30th of August 2007 03:22:37 PM
awk '{x[substr($0,9)]++;y[substr($0,9)]=$1}END{
	for(i in x)
printf "%s\nThis Error was reproduced %d times\n",y[i]i,x[i]
}' logfile

As always, use nawk or /usr/xpg4/bin/awk on Solaris.
Test Your Knowledge in Computers #505
Difficulty: Medium
An executable statement may not invoke (or call or execute) another procedure (also called subroutine, function, method, etc.),
True or False?

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Remove Duplicate Lines in File

I am doing KSH script to remove duplicate lines in a file. Let say the file has format below. FileA 1253-6856 3101-4011 1827-1356 1822-1157 1822-1157 1000-1410 1000-1410 1822-1231 1822-1231 3101-4011 1822-1157 1822-1231 and I want to simply it with no duplicate line as file... (5 Replies)
Discussion started by: Teh Tiack Ein
5 Replies

2. Shell Programming and Scripting

Command/Script to remove duplicate lines from the file?

Hello, Can anyone tell Command/Script to remove duplicate lines from the file? (2 Replies)
Discussion started by: Rahulpict
2 Replies

3. UNIX for Dummies Questions & Answers

How to delete or remove duplicate lines in a file

Hi please help me how to remove duplicate lines in any file. I have a file having huge number of lines. i want to remove selected lines in it. And also if there exists duplicate lines, I want to delete the rest & just keep one of them. Please help me with any unix commands or even fortran... (7 Replies)
Discussion started by: reva
7 Replies

4. Shell Programming and Scripting

remove duplicate lines from file linux/sh

greetings, i'm hoping there is a way to cat a file, remove duplicate lines and send that output to a new file. the file will always vary but be something similar to this: please keep in mind that the above could be eight occurrences of each hostname or it might simply have another four of an... (2 Replies)
Discussion started by: crimso
2 Replies

5. Shell Programming and Scripting

Remove duplicate lines from first file comparing second file

Hi, I have two files with below data:: file1:- 123|aaa|ppp 445|fff|yyy 999|ttt|jjj 555|hhh|hhh file2:- 445|fff|yyy 555|hhh|hhh The records present in file1, not present in file 2 should be writtent to the out put file. output:- 123|aaa|ppp 999|ttt|jjj Is there any one line... (3 Replies)
Discussion started by: gani_85
3 Replies

6. Shell Programming and Scripting

How do I remove the duplicate lines in this file?

Hey guys, need some help to fix this script. I am trying to remove all the duplicate lines in this file. I wrote the following script, but does not work. What is the problem? The output file should only contain five lines: Later! (5 Replies)
Discussion started by: Ernst
5 Replies

7. Shell Programming and Scripting

Remove duplicate lines from a 50 MB file size

hi, Please help me to write a command to delete duplicate lines from a file. And the size of file is 50 MB. How to remove duplicate lins from such a big file. (6 Replies)
Discussion started by: vsachan
6 Replies

8. Shell Programming and Scripting

Remove duplicate lines from a file

Hi, I have a csv file which contains some millions of lines in it. The first line(Header) repeats at every 50000th line. I want to remove all the duplicate headers from the second occurance(should not remove the first line). I don't want to use any pattern from the Header as I have some... (7 Replies)
Discussion started by: sudhakar T
7 Replies

9. Shell Programming and Scripting

Remove duplicate lines from file based on fields

Dear community, I have to remove duplicate lines from a file contains a very big ammount of rows (milions?) based on 1st and 3rd columns The data are like this: Region 23/11/2014 09:11:36 41752 Medio 23/11/2014 03:11:38 4132 Info 23/11/2014 05:11:09 4323... (2 Replies)
Discussion started by: Lord Spectre
2 Replies

10. Shell Programming and Scripting

Remove duplicate lines, sort it and save it as file itself

Hi, all I have a csv file that I would like to remove duplicate lines based on 1st field and sort them by the 1st field. If there are more than 1 line which is same on the 1st field, I want to keep the first line of them and remove the rest. I think I have to use uniq or something, but I still... (8 Replies)
Discussion started by: refrain
8 Replies
snmp_config(4)							   File Formats 						    snmp_config(4)

snmp_config - overview of Net-SNMP configuration files SYNOPSIS
snmp_config DESCRIPTION
This page gives an overview of the various configuration files used by the Net-SNMP software that is shipped with the Solaris operating system. In a configuration file, lines beginning with a hash character (#) are treated as a comment and are not parsed. Search Order By default, the Net-SNMP applications look for configuration files in the following directories in the order listed: 1. /etc/sma/snmp 2. /usr/sfw/lib 3. $HOME/.snmp In each of these directories, the Net-SNMP applications look for files with the extensions .conf and local.conf, in that order. The default search path described above can be overridden by setting the environment variable SNMPCONFPATH to a colon-separated list of directories. SNMPCONFPATH is used to allow users to place configuration files in specific directories for their application needs. Cur- rently defaulted to /etc/sma/snmp and /usr/local/share/snmp. Applications that store persistent data will also look in the /var/net-snmp directory for configuration files. Switching Configuration Types in Mid-File You can switch in mid-file the configuration type that the parser is supposed to be reading. For example, assume you want to turn on packet dumping output for the agent by default, but you do not want to turn on packet dumping for the rest of the applications (such as snmpget and snmpwalk). Normally, to enable packet dumping, you would enter a line such as the one below in the snmp.conf file: dumpPacket true Such a line turns on packet dumping for all of the applications. Instead, you can put the same line in the snmpd.conf file so that it applies only to the snmpd daemon. However, you need to tell the parser to expect this line. You do this by putting a special, type-specifi- cation token inside square brackets. For example, inside your snmpd.conf file you can enter: [snmp] dumpPacket true This tells the parser to parse the line as if it were inside a snmp.conf file instead of an snmpd.conf file. If you want to parse multiple lines rather than just one, you can make the context switch apply to the remainder of the file or until the next context switch directive by putting the special token on a line by itself: # make this file handle snmp.conf tokens: [snmp] dumpPacket true logTimestamp true # return to our original snmpd.conf tokens: [snmpd] rocommunity mypublic ATTRIBUTES
See attributes(5) for descriptions of the following attributes: +-----------------------------+-----------------------------+ | ATTRIBUTE TYPE | ATTRIBUTE VALUE | +-----------------------------+-----------------------------+ |Availability |SUNWsmcmd | +-----------------------------+-----------------------------+ |Interface Stability |External | +-----------------------------+-----------------------------+ SEE ALSO
snmpd.conf(4), attributes(5), sma_snmp(5) SunOS 5.10 16 Jan 2004 snmp_config(4)

Featured Tech Videos

All times are GMT -4. The time now is 02:51 AM.
Unix & Linux Forums Content Copyright 1993-2020. All Rights Reserved.
Privacy Policy