Sponsored Content
Top Forums Shell Programming and Scripting Printing out duplicate lines in a file with Csh Post 302606195 by jim mcnamara on Friday 9th of March 2012 08:21:33 PM
Old 03-09-2012
Your example doesn't seem to match your requirement. I'll match the example:
if test appears in the line, look for duplicates (2 only, not 3)

Code:
awk ' /test/ {arr[$0]++} 
       END{ for(i in arr) {if(arr[i]==2){print i}}}' infile > outfile

This User Gave Thanks to jim mcnamara For This Post:
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Remove Duplicate Lines in File

I am doing KSH script to remove duplicate lines in a file. Let say the file has format below. FileA 1253-6856 3101-4011 1827-1356 1822-1157 1822-1157 1000-1410 1000-1410 1822-1231 1822-1231 3101-4011 1822-1157 1822-1231 and I want to simply it with no duplicate line as file... (5 Replies)
Discussion started by: Teh Tiack Ein
5 Replies

2. UNIX for Advanced & Expert Users

Duplicate lines in the file

Hi, I have a file with duplicate lines in it. I want to keep only the duplicate lines and delete the non duplicates. Can some one please help me? Regards Narayana Gupta (3 Replies)
Discussion started by: guptan
3 Replies

3. UNIX for Dummies Questions & Answers

removing duplicate lines from a file

Hi, I am trying to remove duplicate lines from a file. For example the contents of example.txt is: this is a test 2342 this is a test 34343 this is a test 43434 and i want to remove the "this is a test" lines only and end up with the numbers in the file, that is, end up with: 2342... (4 Replies)
Discussion started by: ocelot
4 Replies

4. UNIX for Dummies Questions & Answers

Remove Duplicate lines from File

I have a log file "logreport" that contains several lines as seen below: 04:20:00 /usr/lib/snmp/snmpdx: Agent snmpd appeared dead but responded to ping 06:38:08 /usr/lib/snmp/snmpdx: Agent snmpd appeared dead but responded to ping 07:11:05 /usr/lib/snmp/snmpdx: Agent snmpd appeared dead but... (18 Replies)
Discussion started by: Nysif Steve
18 Replies

5. Shell Programming and Scripting

CSH: step on the lines in a file

Hi, does somebody know how can I step on the lines in a file? I would like to read a file line by line. And after every line I'd like to do other commands. Lets help me! ;) (4 Replies)
Discussion started by: mig8
4 Replies

6. Shell Programming and Scripting

Duplicate lines in a file

Hi All, I am trying to remove the duplicate entries in a file and print them just once. For example, if my input file has: 00:44,37,67,56,15,12 00:44,34,67,56,15,12 00:44,58,67,56,15,12 00:44,35,67,56,15,12 00:59,37,67,56,15,12 00:59,34,67,56,15,12 00:59,35,67,56,15,12... (7 Replies)
Discussion started by: faiz1985
7 Replies

7. UNIX for Advanced & Expert Users

In a huge file, Delete duplicate lines leaving unique lines

Hi All, I have a very huge file (4GB) which has duplicate lines. I want to delete duplicate lines leaving unique lines. Sort, uniq, awk '!x++' are not working as its running out of buffer space. I dont know if this works : I want to read each line of the File in a For Loop, and want to... (16 Replies)
Discussion started by: krishnix
16 Replies

8. UNIX for Advanced & Expert Users

Inserting duplicate lines in a file

Hi, I copied the contents of a binary file into a .text file using hd (hexdump) command. The data in binary file is such that I get in many places like following 00000250 00 00 00 00 3f 2d 91 68 3f 69 fb e7 00 00 00 00 |....?-.h?i......| 00000260 00 00 00 00 00 00 00 00 00 00 00 00 00... (2 Replies)
Discussion started by: KidD312
2 Replies

9. Shell Programming and Scripting

bash keep only duplicate lines in file

hello all in my bash script I have a file and I only want to keep the lines that appear twice in the file.Is there a way to do this? thanks in advance! (4 Replies)
Discussion started by: vlm
4 Replies

10. UNIX for Dummies Questions & Answers

Duplicate lines in a file

I have a file with following data A B C I would like to print like this n times(For eg:5 times) A B C A B C A B C A B C A (7 Replies)
Discussion started by: nsuresh316
7 Replies
fmlgrep(1F)							   FMLI Commands						       fmlgrep(1F)

NAME
fmlgrep - search a file for a pattern SYNOPSIS
fmlgrep [-b] [-c] [-i] [-l] [-n] [-s] [-v] limited_regular_expression [filename]... DESCRIPTION
fmlgrep searches filename for a pattern and prints all lines that contain that pattern. fmlgrep uses limited regular expressions (expres- sions that have string values that use a subset of the possible alphanumeric and special characters) like those described on the regexp(5) manual page to match the patterns. It uses a compact non-deterministic algorithm. Be careful when using FMLI special characters (for instance, $, `, ', ") in limited_regular_expression. It is safest to enclose the entire limited_regular_expression in single quotes ' ... '. If filename is not specified, fmlgrep assumes standard input. Normally, each line matched is copied to standard output. The file name is printed before each line matched if there is more than one input file. OPTIONS
The following options are supported: -b Precede each line by the block number on which it was found. This can be useful in locating block numbers by context (first block is 0). -c Print only a count of the lines that contain the pattern. -i Ignore upper/lower case distinction during comparisons. -l Print only the names of files with matching lines, separated by new-lines. Does not repeat the names of files when the pattern is found more than once. -n Precede each line by its line number in the file (first line is 1). -s Suppress error messages about nonexistent or unreadable files. -v Print all lines except those that contain the pattern. EXIT STATUS
The following exit values are returned: 0 if the pattern is found (that is, TRUE) 1 if the pattern is not found (that is, FALSE) 2 if an invalid expression was used or filename is inaccessible ATTRIBUTES
See attributes(5) for descriptions of the following attributes: +-----------------------------+-----------------------------+ | ATTRIBUTE TYPE | ATTRIBUTE VALUE | +-----------------------------+-----------------------------+ |Availability |SUNWcsu | +-----------------------------+-----------------------------+ SEE ALSO
egrep(1), fgrep(1), fmlcut(1F), grep(1), attributes(5), regexp(5) NOTES
Lines are limited to BUFSIZ characters; longer lines are truncated. BUFSIZ is defined in /usr/include/stdio.h. If there is a line with embedded nulls, fmlgrep will only match up to the first null; if it matches, it will print the entire line. SunOS 5.11 28 Mar 1995 fmlgrep(1F)
All times are GMT -4. The time now is 12:44 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy