Sponsored Content
Full Discussion: match similar rows. uniq?
Top Forums UNIX for Dummies Questions & Answers match similar rows. uniq? Post 302176448 by drl on Tuesday 18th of March 2008 09:34:05 AM
Old 03-18-2008
Hi.

The GNU/Linux version of uniq -- uniq (coreutils) 5.2.1 -- has these options (among others):
Code:
       -f, --skip-fields=N
              avoid comparing the first N fields

       -i, --ignore-case
              ignore differences in case when comparing

       -t, --separator=SEP
              use SEParator to delimit fields

       -W, --check-fields=N
              compare no more than N fields in lines

However, if you have a fewer-featured uniq, then the solution from radoulov would be useful. I'm sure he'd be willing to explain it if you asked politely ... cheers, drl
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

merge similar rows

I have a large file (10M lines) that contains two columns: a frequency and a string, ex: 3 aaaaa 4 bbbbb 2 ccccc 5 aaaaa 1 ddddd 4 ccccc I need to merge the lines whose string part is the same, while updating the frequency. The output should look like this: 8 aaaaa 4 bbbbb 5 ccccc... (2 Replies)
Discussion started by: tootles564
2 Replies

2. Shell Programming and Scripting

printing 3 files side by side based on similar values in rows

Hi I'm trying to compare 3 or more files based on similar values and outputting them into 3 columns. For example: file1 ABC DEF GHI file2 DEF DER file3 ABC DER The output should come out like this file1 file2 file3 ABC ABC (4 Replies)
Discussion started by: zerofire123
4 Replies

3. Shell Programming and Scripting

Awk match rows

Hi, I am pretty new to awk. I have a text file of the following style a b c d e f g h i 1 a b c d e f g h i 2 a b c d e f g h i 3 j k l m n o p q r 4 s t u v w x y z # 5 s t u v w x y z #7 I want the minimum of 10th column if the first 9 columns match with its before and after... (6 Replies)
Discussion started by: jacobs.smith
6 Replies

4. UNIX for Dummies Questions & Answers

[diff] hide missing rows, show similar

Hi all! Having the following two csv files: file1 AAA;0000;RED CCC;9900;GREEN file2 AAA;0000;BLACK BBB;0099;BLU What's the correct syntax to hide only the missing rows (BBB,CCC) and show the rows that differ only with last field? I expect something like this: diff <options> file1... (2 Replies)
Discussion started by: Evan
2 Replies

5. Shell Programming and Scripting

match sentence and word adn fetch similar words in alist

Hi all, I have ot match sentence list and word list anf fetch similar words in a separate file second file with 2 columns So I want the output shuld be 2 columns like this (3 Replies)
Discussion started by: manigrover
3 Replies

6. Shell Programming and Scripting

Perl match multiple numbers from a variable similar to egrep

I want to match the number exactly from the variable which has multiple numbers seperated by pipe symbol similar to search in egrep.below is the code which i tried #!/usr/bin/perl my $searchnum = $ARGV; my $num = "148|1|0|256"; print $num; if ($searchnum =~ /$num/) { print "found"; }... (2 Replies)
Discussion started by: kar_333
2 Replies

7. Shell Programming and Scripting

Rows to Columns with match criteria

Hello Friends, I have a input file having hundreds of rows. I want them to translate in to columns if column 1 is same. Input data: zp06 xxx zp06 rrr zp06 hhh zp06 aaa zp06 ggg zp06 qwer zp06 ser zl11 old3 zl11 old4 zl11 old5 zl11 old6 zl11 old7 zm14 luri zm14 body zm14 ucp (9 Replies)
Discussion started by: suresh3566
9 Replies

8. Shell Programming and Scripting

Uniq or sort -u or similar only between { }

Hi ! I am trying to remove doubbled entrys in a textfile only between delimiters. Like that example but i dont know how to do that with sort or similar. input: { aaa aaa } { aaa aaa } output: { aaa } { (8 Replies)
Discussion started by: fugitivus
8 Replies

9. Shell Programming and Scripting

Transposing rows to columns with multiple similar lines

Hi, I am trying to transpose rows to columns for thousands of records. The problem is there are records that have the same lines that need to be separated. the input file as below:- ID 1A02_HUMAN AC P01892; O19619; P06338; P10313; P30444; P30445; P30446; P30514; AC Q29680; Q29837;... (2 Replies)
Discussion started by: redse171
2 Replies

10. Shell Programming and Scripting

To group the text (rows) by similar columns-names in a file

As part of some report generation, I've written a script to fetch the values from DB. But, unluckily, for certain Time ranges(1-9.99,10-19.99 etc), I don't have data in DB. In such cases, I would like to write zero (0) instead of empty. The desired output will be exported to csv file. ... (1 Reply)
Discussion started by: kumar_karpuram
1 Replies
UNIQ(1) 							   User Commands							   UNIQ(1)

NAME
uniq - report or omit repeated lines SYNOPSIS
uniq [OPTION]... [INPUT [OUTPUT]] DESCRIPTION
Filter adjacent matching lines from INPUT (or standard input), writing to OUTPUT (or standard output). With no options, matching lines are merged to the first occurrence. Mandatory arguments to long options are mandatory for short options too. -c, --count prefix lines by the number of occurrences -d, --repeated only print duplicate lines, one for each group -D, --all-repeated[=METHOD] print all duplicate lines groups can be delimited with an empty line METHOD={none(default),prepend,separate} -f, --skip-fields=N avoid comparing the first N fields --group[=METHOD] show all items, separating groups with an empty line METHOD={separate(default),prepend,append,both} -i, --ignore-case ignore differences in case when comparing -s, --skip-chars=N avoid comparing the first N characters -u, --unique only print unique lines -z, --zero-terminated end lines with 0 byte, not newline -w, --check-chars=N compare no more than N characters in lines --help display this help and exit --version output version information and exit A field is a run of blanks (usually spaces and/or TABs), then non-blank characters. Fields are skipped before chars. Note: 'uniq' does not detect repeated lines unless they are adjacent. You may want to sort the input first, or use 'sort -u' without 'uniq'. Also, comparisons honor the rules specified by 'LC_COLLATE'. GNU coreutils online help: <http://www.gnu.org/software/coreutils/> Report uniq translation bugs to <http://translationproject.org/team/> AUTHOR
Written by Richard M. Stallman and David MacKenzie. COPYRIGHT
Copyright (C) 2013 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>. This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. SEE ALSO
comm(1), join(1), sort(1) The full documentation for uniq is maintained as a Texinfo manual. If the info and uniq programs are properly installed at your site, the command info coreutils 'uniq invocation' should give you access to the complete manual. GNU coreutils 8.22 June 2014 UNIQ(1)
All times are GMT -4. The time now is 08:44 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy