|
|
Sponsored Content
Top Forums
Shell Programming and Scripting
[uniq + awk?] How to remove duplicate blocks of lines in files?
Post 302557570 by vgersh99 on Wednesday 21st of September 2011 02:35:40 PM
09-21-2011
Moderator
Quote:
Code:
<string-array name="stringArray1"> <item>Element1|Element2<item> <item>@android:color/black<item> </string-array> <style name="style1" parent="@android:style/mainStyle"></style> <style name="style1" parent="@android:style/mainStyle"> <item name="android:textColor">@drawable/black</item> <item name="android:typeface">sans</item> <item name="android:textStyle">bold</item> </style> <string-array name="stringArray1"> <item>Element1|Element2<item> <item>@android:color/black<item> </string-array> <string-array name="stringArray2"></string-array>
Code:
nawk -F'"' '$1 ~ "=$" && NF>1 {f=($(NF-1) in a)?0:1;if(f)a[$(NF-1)]} f' myFileCode:
<string-array name="stringArray1"> <item>Element1|Element2<item> <item>@android:color/black<item> </string-array> <style name="style1" parent="@android:style/mainStyle"></style> <item name="android:textColor">@drawable/black</item> <item name="android:typeface">sans</item> <item name="android:textStyle">bold</item> </style> <string-array name="stringArray2"></string-array>
|
|
10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
Hi,
is it possible to remove all duplicate lines from all txt files in a specific folder?
This is too hard for me maybe someone could help.
lets say we have an amount of textfiles 1 or 2 or 3 or... maximum 50
each textfile has lines with text.
I want all lines of all textfiles... (8 Replies)
Discussion started by: lowmaster
8 Replies
2. UNIX for Dummies Questions & Answers
I have a data set that has 4 columns, I want to know if I can delete duplicate lines while ignoring one of the columns, for example
10 chr1 ASF 30
15 chr1 ASF 20
5 chr1 ASF 30
6 chr2 EBC 15
4 chr2 EBC 30
...
I want to know if I can delete duplicate lines while ignoring column 1, so the... (5 Replies)
Discussion started by: japaneseguitars
5 Replies
3. Shell Programming and Scripting
Input:
hello hello
hello hello
monkey
donkey
hello hello
drink
dance
drink
Output should be:
hello hello
monkey
donkey
drink
dance (9 Replies)
Discussion started by: cola
9 Replies
4. Shell Programming and Scripting
Dear All,
I have multiple files having number of records, consist of more than 10 columns some column values are duplicate and i want to remove these duplicate values from these files.
Duplicate values may come in different files.... all files laying in single directory..
Need help to... (3 Replies)
Discussion started by: arvindng
3 Replies
5. Shell Programming and Scripting
Hi,
I came to know that using awk '!x++' removes the duplicate lines. Can anyone please explain the above syntax. I want to understand how the above awk syntax removes the duplicates.
Thanks in advance,
sudvishw :confused: (7 Replies)
Discussion started by: sudvishw
7 Replies
6. Shell Programming and Scripting
Cant get awk 1liner to remove duplicate lines from Delimited file, get "event not found" error..help
Hi,
I am on a Solaris8 machine
If someone can help me with adjusting this awk 1 liner (turning it into a real awkscript) to get by this "event not found error"
...or
Present Perl solution code that works for Perl5.8 in the csh shell ...that would be great.
******************
... (3 Replies)
Discussion started by: andy b
3 Replies
7. Windows & DOS: Issues & Discussions
So, I have text files,
one "fail.txt"
And one
"color.txt"
I now want to use a command line (DOS) to remove ANY line that is PRESENT IN BOTH from each text file.
Afterwards there shall be no duplicate lines. (1 Reply)
Discussion started by: pasc
1 Replies
8. Shell Programming and Scripting
Hi All
I have a list of files which will have duplicate list of blocks of text. Following is a sample of the file, I have removed the sensitive information from the file.
All the code samples starts from <TR BGCOLOR="white"> and Ends with IP address and two html tags like this.
10.14.22.22... (3 Replies)
Discussion started by: mahasona
3 Replies
9. Shell Programming and Scripting
I have two large files (~250GB) that I am trying to remove the where GT: 0/0 or 1/1 or 2/2 for both files. I was going to use a bash with the below awk, which I think will find each line but how do I remove that line is that condition is found? Thank you :).
Input
20 60055 . A ... (4 Replies)
Discussion started by: cmccabe
4 Replies
10. Shell Programming and Scripting
I create a CGI in bash/html.
My awk script looks like :
echo "<table>"
for fn in /var/www/cgi-bin/LPAR_MAP/*;
do
echo "<td>"
echo "<PRE>"
awk -F',|;' -v test="$test" '
NR==1 {
split(FILENAME ,a,"");
}
$0 ~ test {
if(!header++){
... (12 Replies)
Discussion started by: Tim2424
12 Replies
LEARN ABOUT OSF1
ppmtopgm
ppmtopgm(1) General Commands Manual ppmtopgm(1) NAME
ppmtopgm - convert a portable pixmap into a portable graymap SYNOPSIS
ppmtopgm [ppmfile] DESCRIPTION
Reads a portable pixmap as input. Produces a portable graymap as output. The output is a "black and white" rendering of the original image, as in a black and white photograph. The quantization formula used is .299 r + .587 g + .114 b. Note that although there is a pgmtoppm program, it is not necessary for simple conversions from pgm to ppm , because any ppm program can read pgm (and pbm ) files automatically. pgmtoppm is for colorizing a pgm file. Also, see ppmtorgb3 for a different way of converting color to gray. And ppmdist generates a grayscale image from a color image, but in a way that makes it easy to differentiate the original colors, not necessarily a way that looks like a black and white photograph. QUOTE
Cold-hearted orb that rules the night Removes the colors from our sight Red is gray, and yellow white But we decide which is right And which is a quantization error. SEE ALSO
pgmtoppm(1),ppmtorgb3(1),rgb3toppm(1),ppmdist(1),ppm(5),pgm(5) AUTHOR
Copyright (C) 1989 by Jef Poskanzer. 10 April 2000 ppmtopgm(1)