Sponsored Content
Top Forums UNIX for Dummies Questions & Answers deleteing duplicate lines sing uniq while ignoring a column Post 302429831 by Scott on Tuesday 15th of June 2010 05:45:24 PM
Old 06-15-2010
A long shot...

Code:
awk '!A[substr($0, index($0, " "))]++' file1 
10 chr1 ASF 30
15 chr1 ASF 20
6 chr2 EBC 15
4 chr2 EBC 30

 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

oneliner:sing SED on a specific column

is this possible a one liner sed command. I have a text file ex. from : xxx yyy ZZZ /test/devl/aasdasd.log1 xxx yyy ZZZ /test/devl/aasdasd.log2 to : xxx yyy ZZZ /test/prod/aasdasd.log1 xxx yyy ZZZ /test/prod/aasdasd.log2 and I want to sed only the fourth column sed 's/devl/prod/g' ... (8 Replies)
Discussion started by: chaseeem
8 Replies

2. Shell Programming and Scripting

[uniq + awk?] How to remove duplicate blocks of lines in files?

Hello again, I am wanting to remove all duplicate blocks of XML code in a file. This is an example: input: <string-array name="threeItems"> <item>item1</item> <item>item2</item> <item>item3</item> </string-array> <string-array name="twoItems"> <item>item1</item> <item>item2</item>... (19 Replies)
Discussion started by: raidzero
19 Replies

3. UNIX for Dummies Questions & Answers

[SOLVED] remove lines that have duplicate values in column two

Hi, I've got a file that I'd like to uniquely sort based on column 2 (values in column 2 begin with "comp"). I tried sort -t -nuk2,3 file.txtBut got: sort: multi-character tab `-nuk2,3' "man sort" did not help me out Any pointers? Input: Output: (5 Replies)
Discussion started by: pathunkathunk
5 Replies

4. Shell Programming and Scripting

Replace a column with a value by ignoring the header lines

i have a file in the gz format , the content of the file is as follow. gzcat f1.gz # 1.name # 2.location # 3.age # 4.dob . . . . . . . . . # 43.hobbies < Aravind,33,chennai,09091980, , , , , , , surfing> (5 Replies)
Discussion started by: aravindj80
5 Replies

5. UNIX for Dummies Questions & Answers

awk to sum column field from duplicate row/lines

Hello, I am new to Linux environment , I working on Linux script which should send auto email based on the specific condition from log file. Below is the sample log file Name m/c usage abc xxx 10 abc xxx 20 abc xxx 5 xyz ... (6 Replies)
Discussion started by: asjaiswal
6 Replies

6. UNIX for Dummies Questions & Answers

awk solution to duplicate lines based on column

Hi experts, I have a tab-delimited file with one column containing values separated by a comma. I wish to duplicate the entire line for every value in that comma-delimited field. For example: $cat file 4444 4444 4444 4444 9990 2222,7777 6666 2222 ... (3 Replies)
Discussion started by: torchij
3 Replies

7. Shell Programming and Scripting

Count duplicate lines ignoring certain columns

I have this structure: col1 col2 col3 col4 col5 27 xxx 38 aaa ttt 2 xxx 38 aaa yyy 1 xxx 38 aaa yyy I need to collapse duplicate lines ignoring column 1 and add values of duplicate lines (col1) so it will look like this: col1 col2 col3 col4 col5 27 xxx 38 aaa ttt ... (3 Replies)
Discussion started by: coppuca
3 Replies

8. Shell Programming and Scripting

Remove duplicate lines after ignoring case and spaces between

Oracle Linux 6.5 $ cat someStrings.txt GRANT select on MANHPRD.S_PROD_INT TO OR_PHIL; GRANT select on MANHPRD.S_PROD_INT TO OR_PHIL; GRANT select on SCOTT.emp to JOHN; grant select on scott.emp to john; grant select on scott.dept to hr;If you ignore the case and the empty space between the... (6 Replies)
Discussion started by: kraljic
6 Replies

9. Shell Programming and Scripting

Removing duplicate lines on first column based with pipe delimiter

Hi, I have tried to remove dublicate lines based on first column with pipe delimiter . but i ma not able to get some uniqu lines Command : sort -t'|' -nuk1 file.txt Input : 38376KZ|09/25/15|1.057 38376KZ|09/25/15|1.057 02006YB|09/25/15|0.859 12593PS|09/25/15|2.803... (2 Replies)
Discussion started by: parithi06
2 Replies

10. UNIX for Beginners Questions & Answers

Find lines with duplicate values in a particular column

I have a file with 5 columns. I want to pull out all records where the value in column 4 is not unique. For example in the sample below, I would want it to print out all lines except for the last two. 40991764 2419 724 47182 Cand A 40992936 3591 724 47182 Cand B 40993016 3671 724 47182 Cand C... (5 Replies)
Discussion started by: kaktus
5 Replies
SUBSTR(3)								 1								 SUBSTR(3)

substr - Return part of a string

SYNOPSIS
string substr (string $string, int $start, [int $length]) DESCRIPTION
Returns the portion of $string specified by the $start and $length parameters. PARAMETERS
o $string - The input string. Must be one character or longer. o $start - If $start is non-negative, the returned string will start at the $start'th position in $string, counting from zero. For instance, in the string ' abcdef', the character at position 0 is ' a', the character at position 2 is ' c', and so forth. If $start is negative, the returned string will start at the $start'th character from the end of $string. If $string is less than or equal to $start characters long, FALSE will be returned. Example #1 Using a negative $start <?php $rest = substr("abcdef", -1); // returns "f" $rest = substr("abcdef", -2); // returns "ef" $rest = substr("abcdef", -3, 1); // returns "d" ?> o $length - If $length is given and is positive, the string returned will contain at most $length characters beginning from $start (depend- ing on the length of $string). If $length is given and is negative, then that many characters will be omitted from the end of $string (after the start position has been calculated when a $start is negative). If $start denotes the position of this trunca- tion or beyond, false will be returned. If $length is given and is 0, FALSE or NULL, an empty string will be returned. If $length is omitted, the substring starting from $start until the end of the string will be returned. Example #2 Using a negative $length <?php $rest = substr("abcdef", 0, -1); // returns "abcde" $rest = substr("abcdef", 2, -1); // returns "cde" $rest = substr("abcdef", 4, -4); // returns false $rest = substr("abcdef", -3, -1); // returns "de" ?> RETURN VALUES
Returns the extracted part of $string; or FALSE on failure, or an empty string. CHANGELOG
+--------------+---------------------------------------------------+ | Version | | | | | | | Description | | | | +--------------+---------------------------------------------------+ |5.2.2 - 5.2.6 | | | | | | | If the $start parameter indicates the position | | | of a negative truncation or beyond, false is | | | returned. Other versions get the string from | | | start. | | | | +--------------+---------------------------------------------------+ EXAMPLES
Example #3 Basic substr(3) usage <?php echo substr('abcdef', 1); // bcdef echo substr('abcdef', 1, 3); // bcd echo substr('abcdef', 0, 4); // abcd echo substr('abcdef', 0, 8); // abcdef echo substr('abcdef', -1, 1); // f // Accessing single characters in a string // can also be achieved using "square brackets" $string = 'abcdef'; echo $string[0]; // a echo $string[3]; // d echo $string[strlen($string)-1]; // f ?> Example #4 substr(3) casting behaviour <?php class apple { public function __toString() { return "green"; } } echo "1) ".var_export(substr("pear", 0, 2), true).PHP_EOL; echo "2) ".var_export(substr(54321, 0, 2), true).PHP_EOL; echo "3) ".var_export(substr(new apple(), 0, 2), true).PHP_EOL; echo "4) ".var_export(substr(true, 0, 1), true).PHP_EOL; echo "5) ".var_export(substr(false, 0, 1), true).PHP_EOL; echo "6) ".var_export(substr("", 0, 1), true).PHP_EOL; echo "7) ".var_export(substr(1.2e3, 0, 4), true).PHP_EOL; ?> The above example will output: 1) 'pe' 2) '54' 3) 'gr' 4) '1' 5) false 6) false 7) '1200' ERRORS
/EXCEPTIONS Returns FALSE on error. Example #5 <?php var_dump(substr('a', 1)); // bool(false) ?> SEE ALSO
strrchr(3), substr_replace(3), preg_match(3), trim(3), mb_substr(3), wordwrap(3), String access and modification by character. PHP Documentation Group SUBSTR(3)
All times are GMT -4. The time now is 11:33 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy