|
|
Sponsored Content
Top Forums
Shell Programming and Scripting
remove duplicate lines using awk
Post 302492341 by sudvishw on Monday 31st of January 2011 04:17:36 AM
01-31-2011
|
|
10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
I am doing KSH script to remove duplicate lines in a file. Let say the file has format below.
FileA
1253-6856
3101-4011
1827-1356
1822-1157
1822-1157
1000-1410
1000-1410
1822-1231
1822-1231
3101-4011
1822-1157
1822-1231
and I want to simply it with no duplicate line as file... (5 Replies)
Discussion started by: Teh Tiack Ein
5 Replies
2. Shell Programming and Scripting
I have following file content (3 fields each line):
23 888 10.0.0.1
dfh 787 10.0.0.2
dssf dgfas 10.0.0.3
dsgas dg 10.0.0.4
df dasa 10.0.0.5
df dag 10.0.0.5
dfd dfdas 10.0.0.5
dfd dfd 10.0.0.6
daf nfd 10.0.0.6
...
as can be seen, that the third field is ip address and sorted. but... (3 Replies)
Discussion started by: fredao
3 Replies
3. UNIX for Dummies Questions & Answers
I have a log file "logreport" that contains several lines as seen below:
04:20:00 /usr/lib/snmp/snmpdx: Agent snmpd appeared dead but responded to ping
06:38:08 /usr/lib/snmp/snmpdx: Agent snmpd appeared dead but responded to ping
07:11:05 /usr/lib/snmp/snmpdx: Agent snmpd appeared dead but... (18 Replies)
Discussion started by: Nysif Steve
18 Replies
4. Shell Programming and Scripting
Input:
hello hello
hello hello
monkey
donkey
hello hello
drink
dance
drink
Output should be:
hello hello
monkey
donkey
drink
dance (9 Replies)
Discussion started by: cola
9 Replies
5. Shell Programming and Scripting
Hi, I have a huge file which is about 50GB. There are many lines. The file format likes
21 rs885550 0 9887804 C C T C C C C C C C
21 rs210498 0 9928860 0 0 C C 0 0 0 0 0 0
21 rs303304 0 9941889 A A A A A A A A A A
22 rs303304 0 9941890 0 A A A A A A A A A
The question is that there are a few... (4 Replies)
Discussion started by: zhshqzyc
4 Replies
6. Shell Programming and Scripting
Hello again, I am wanting to remove all duplicate blocks of XML code in a file. This is an example:
input:
<string-array name="threeItems">
<item>item1</item>
<item>item2</item>
<item>item3</item>
</string-array>
<string-array name="twoItems">
<item>item1</item>
<item>item2</item>... (19 Replies)
Discussion started by: raidzero
19 Replies
7. Shell Programming and Scripting
Cant get awk 1liner to remove duplicate lines from Delimited file, get "event not found" error..help
Hi,
I am on a Solaris8 machine
If someone can help me with adjusting this awk 1 liner (turning it into a real awkscript) to get by this "event not found error"
...or
Present Perl solution code that works for Perl5.8 in the csh shell ...that would be great.
******************
... (3 Replies)
Discussion started by: andy b
3 Replies
8. UNIX for Dummies Questions & Answers
Hi
I need this output. Thanks.
Input:
TAZ
YET
FOO
FOO
VAK
TAZ
BAR
Output:
YET
VAK
BAR (10 Replies)
Discussion started by: tara123
10 Replies
9. Shell Programming and Scripting
Hi All,
I am storing the result in the variable result_text using the below code.
result_text=$(printf "$result_text\t\n$name") The result_text is having the below text. Which is having duplicate lines.
file and time for the interval 03:30 - 03:45
file and time for the interval 03:30 - 03:45 ... (4 Replies)
Discussion started by: nalu
4 Replies
10. Shell Programming and Scripting
I create a CGI in bash/html.
My awk script looks like :
echo "<table>"
for fn in /var/www/cgi-bin/LPAR_MAP/*;
do
echo "<td>"
echo "<PRE>"
awk -F',|;' -v test="$test" '
NR==1 {
split(FILENAME ,a,"");
}
$0 ~ test {
if(!header++){
... (12 Replies)
Discussion started by: Tim2424
12 Replies
LEARN ABOUT CENTOS
paps
PAPS(1) General Commands Manual PAPS(1) NAME
paps - UTF-8 to PostScript converter using Pango SYNOPSIS
paps [options] files... DESCRIPTION
paps reads a UTF-8 encoded file and generates a PostScript language rendering of the file. The rendering is done by creating outline curves through the pango ft2 backend. OPTIONS
These programs follow the usual GNU command line syntax, with long options starting with two dashes (`-'). A summary of options is included below. --landscape Landscape output. Default is portrait. --columns=cl Number of columns output. Default is 1. --font=desc Set the font description. Default is Monospace 12. --rtl Do rtl layout. --paper ps Choose paper size. Known paper sizes are legal, letter, a4. Default is A4. --bottom-margin=bm Set bottom margin in postscript points (1/72 inch). Default is 36. --top-margin=tm Set top margin. Default is 36. --left-margin=lm Set left margin. Default is 36. --right-margin=rm Set right margin. Default is 36. --help Show summary of options. --header Draw page header for each page. --markup Interpret the text as pango markup. --encoding=ENCODING Assume the documentation encoding is ENCODING. --lpi Set the lines per inch. This determines the line spacing. --cpi Set the characters per inch. This is an alternative method of specifying the font size. --stretch-chars Indicates that characters should be stretched in the y-direction to fill up their vertical space. This is similar to the texttops behaviour. AUTHOR
paps was written by Dov Grobgeld <dov.grobgeld@gmail.com>. This manual page was written by Lior Kaplan <kaplan@debian.org>, for the Debian project (but may be used by others). April 17, 2006 PAPS(1)