Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: How to put the command to remove duplicate lines in my awk script?
Top Forums Shell Programming and Scripting How to put the command to remove duplicate lines in my awk script? Post 303037652 by Neo on Thursday 8th of August 2019 11:36:56 PM
Old 08-09-2019
I always do these types of tasks in PHP; but that's just me. We have a lot of AWK lovers here who will help with the AWK code.

My comment is only on the HTML:

Regarding the code above; my only comment is that, for the most part web developers most agree that <table> tags should be avoided and <div> tags should be used instead.

I have mostly eliminated all <table> tags here at unix.com, but there are still a few <table> tags here at unix.com from decades of legacy code I need to obsolete someday......
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

how to remove duplicate lines

I have following file content (3 fields each line): 23 888 10.0.0.1 dfh 787 10.0.0.2 dssf dgfas 10.0.0.3 dsgas dg 10.0.0.4 df dasa 10.0.0.5 df dag 10.0.0.5 dfd dfdas 10.0.0.5 dfd dfd 10.0.0.6 daf nfd 10.0.0.6 ... as can be seen, that the third field is ip address and sorted. but... (3 Replies)
Discussion started by: fredao
3 Replies

2. Shell Programming and Scripting

Command/Script to remove duplicate lines from the file?

Hello, Can anyone tell Command/Script to remove duplicate lines from the file? (2 Replies)
Discussion started by: Rahulpict
2 Replies

3. Shell Programming and Scripting

awk script to remove duplicate rows in line

i have the long file more than one ns and www and mx in the line like . i need the first ns record and first www and first mx from line . the records are seperated with tthe ; i am try ing in awk scripting not getiing the solution. ... (4 Replies)
Discussion started by: kiranmosarla
4 Replies

4. Shell Programming and Scripting

Command to remove duplicate lines with perl,sed,awk

Input: hello hello hello hello monkey donkey hello hello drink dance drink Output should be: hello hello monkey donkey drink dance (9 Replies)
Discussion started by: cola
9 Replies

5. Shell Programming and Scripting

remove duplicate lines using awk

Hi, I came to know that using awk '!x++' removes the duplicate lines. Can anyone please explain the above syntax. I want to understand how the above awk syntax removes the duplicates. Thanks in advance, sudvishw :confused: (7 Replies)
Discussion started by: sudvishw
7 Replies

6. Shell Programming and Scripting

Remove duplicate lines

Hi, I have a huge file which is about 50GB. There are many lines. The file format likes 21 rs885550 0 9887804 C C T C C C C C C C 21 rs210498 0 9928860 0 0 C C 0 0 0 0 0 0 21 rs303304 0 9941889 A A A A A A A A A A 22 rs303304 0 9941890 0 A A A A A A A A A The question is that there are a few... (4 Replies)
Discussion started by: zhshqzyc
4 Replies

7. Shell Programming and Scripting

[uniq + awk?] How to remove duplicate blocks of lines in files?

Hello again, I am wanting to remove all duplicate blocks of XML code in a file. This is an example: input: <string-array name="threeItems"> <item>item1</item> <item>item2</item> <item>item3</item> </string-array> <string-array name="twoItems"> <item>item1</item> <item>item2</item>... (19 Replies)
Discussion started by: raidzero
19 Replies

8. Shell Programming and Scripting

AWK Command to duplicate lines in a file?

Hi, I have a file with date in it like: UserString1 UserString2 UserString3 UserString4 UserString5 I need two entries for each line so it reads like UserString1 UserString1 UserString2 UserString2 etc. Can someone help me with the awk command please? Thanks (4 Replies)
Discussion started by: Grueben
4 Replies

9. Shell Programming and Scripting

Cant get awk 1liner to remove duplicate lines from Delimited file, get "event not found" error..help

Hi, I am on a Solaris8 machine If someone can help me with adjusting this awk 1 liner (turning it into a real awkscript) to get by this "event not found error" ...or Present Perl solution code that works for Perl5.8 in the csh shell ...that would be great. ****************** ... (3 Replies)
Discussion started by: andy b
3 Replies

10. Shell Programming and Scripting

How to remove duplicate lines?

Hi All, I am storing the result in the variable result_text using the below code. result_text=$(printf "$result_text\t\n$name") The result_text is having the below text. Which is having duplicate lines. file and time for the interval 03:30 - 03:45 file and time for the interval 03:30 - 03:45 ... (4 Replies)
Discussion started by: nalu
4 Replies

LEARN ABOUT DEBIAN

git-ls-remote

GIT-LS-REMOTE(1)						    Git Manual							  GIT-LS-REMOTE(1)

NAME

       git-ls-remote - List references in a remote repository

SYNOPSIS

       git ls-remote [--heads] [--tags]  [-u <exec> | --upload-pack <exec>]
		     [--exit-code] <repository> [<refs>...]

DESCRIPTION

       Displays references available in a remote repository along with the associated commit IDs.

OPTIONS

       -h, --heads, -t, --tags
	   Limit to only refs/heads and refs/tags, respectively. These options are not mutually exclusive; when given both, references stored in
	   refs/heads and refs/tags are displayed.

       -u <exec>, --upload-pack=<exec>
	   Specify the full path of git-upload-pack on the remote host. This allows listing references from repositories accessed via SSH and
	   where the SSH daemon does not use the PATH configured by the user.

       --exit-code
	   Exit with status "2" when no matching refs are found in the remote repository. Usually the command exits with status "0" to indicate it
	   successfully talked with the remote repository, whether it found any matching refs.

       <repository>
	   Location of the repository. The shorthand defined in $GIT_DIR/branches/ can be used. Use "." (dot) to list references in the local
	   repository.

       <refs>...
	   When unspecified, all references, after filtering done with --heads and --tags, are shown. When <refs>... are specified, only
	   references matching the given patterns are displayed.

EXAMPLES

	   $ git ls-remote --tags ./.
	   d6602ec5194c87b0fc87103ca4d67251c76f233a	   refs/tags/v0.99
	   f25a265a342aed6041ab0cc484224d9ca54b6f41	   refs/tags/v0.99.1
	   7ceca275d047c90c0c7d5afb13ab97efdf51bd6e	   refs/tags/v0.99.3
	   c5db5456ae3b0873fc659c19fafdde22313cc441	   refs/tags/v0.99.2
	   0918385dbd9656cab0d1d81ba7453d49bbc16250	   refs/tags/junio-gpg-pub
	   $ git ls-remote http://www.kernel.org/pub/scm/git/git.git master pu rc
	   5fe978a5381f1fbad26a80e682ddd2a401966740	   refs/heads/master
	   c781a84b5204fb294c9ccc79f8b3baceeb32c061	   refs/heads/pu
	   b1d096f2926c4e37c9c0b6a7bf2119bedaa277cb	   refs/heads/rc
	   $ echo http://www.kernel.org/pub/scm/git/git.git >.git/branches/public
	   $ git ls-remote --tags public v*
	   d6602ec5194c87b0fc87103ca4d67251c76f233a	   refs/tags/v0.99
	   f25a265a342aed6041ab0cc484224d9ca54b6f41	   refs/tags/v0.99.1
	   c5db5456ae3b0873fc659c19fafdde22313cc441	   refs/tags/v0.99.2
	   7ceca275d047c90c0c7d5afb13ab97efdf51bd6e	   refs/tags/v0.99.3

GIT

       Part of the git(1) suite

Git 1.7.10.4							    11/24/2012							  GIT-LS-REMOTE(1)
All times are GMT -4. The time now is 09:36 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy