Command to remove duplicate lines with perl,sed,awk Post: 302463149

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Command/Script to remove duplicate lines from the file?

Hello, Can anyone tell Command/Script to remove duplicate lines from the file?

2. Shell Programming and Scripting

Sed or Awk to remove specific lines

I have searched the forum for this - forgive me if I missed a previous post. I have the following file: blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah alter table "informix".esc_acct add constraint (foreign key (fi_id) references "informix".fi ...

3. Shell Programming and Scripting

How to remove lines before and after with awk / sed ?

Hi guys, I need to remove the pattern (ID=180), one line before and four lines after. Thanks.

4. Shell Programming and Scripting

perl/shell need help to remove duplicate lines from files

Dear All, I have multiple files having number of records, consist of more than 10 columns some column values are duplicate and i want to remove these duplicate values from these files. Duplicate values may come in different files.... all files laying in single directory.. Need help to...

5. Shell Programming and Scripting

remove duplicate lines using awk

Hi, I came to know that using awk '!x++' removes the duplicate lines. Can anyone please explain the above syntax. I want to understand how the above awk syntax removes the duplicates. Thanks in advance, sudvishw :confused:

6. Shell Programming and Scripting

[uniq + awk?] How to remove duplicate blocks of lines in files?

Hello again, I am wanting to remove all duplicate blocks of XML code in a file. This is an example: input: <string-array name="threeItems"> <item>item1</item> <item>item2</item> <item>item3</item> </string-array> <string-array name="twoItems"> <item>item1</item> <item>item2</item>...

7. Shell Programming and Scripting

AWK Command to duplicate lines in a file?

Hi, I have a file with date in it like: UserString1 UserString2 UserString3 UserString4 UserString5 I need two entries for each line so it reads like UserString1 UserString1 UserString2 UserString2 etc. Can someone help me with the awk command please? Thanks

8. Shell Programming and Scripting

Sed/awk/perl command to replace pattern in multiple lines

Hi I know sed and awk has options to give range of line numbers, but I need to replace pattern in specific lines Something like sed -e '1s,14s,26s/pattern/new pattern/' file name Can somebody help me in this.... I am fine with see/awk/perl Thank you in advance

9. Shell Programming and Scripting

Using sed, awk or perl to remove substring of all lines except the first

Greetings All, I would like to find all occurences of a pattern and delete a substring from the all matching lines EXCEPT the first. For example: 1234::group:user1,user2,user3,blah1,blah2,blah3 2222::othergroup:user9,user8 4444::othergroup2:user3,blah,blah,user1 1234::group3:user5,user1 ...

10. Shell Programming and Scripting

How to put the command to remove duplicate lines in my awk script?

I create a CGI in bash/html. My awk script looks like : echo "<table>" for fn in /var/www/cgi-bin/LPAR_MAP/*; do echo "<td>" echo "<PRE>" awk -F',|;' -v test="$test" ' NR==1 { split(FILENAME ,a,""); } $0 ~ test { if(!header++){ ...

LEARN ABOUT REDHAT

nwbpset

NWBPSET(1)							      nwbpset								NWBPSET(1)

NAME

       nwbpset - Create a bindery property or set its value

SYNOPSIS

       nwbpset [ -h ] [ -S server ] [ -U user name ] [ -P password | -n ] [ -C ]

DESCRIPTION

       nwbpset Reads a property specification from the standard input and creates and sets the corresponding property. The format is determined by
       the output of 'nwbpvalues -c'. nwbpset will hopefully become an important part of the bindery management suite of ncpfs, together with

       As another example, look at the following command line:

	 nwbpvalues -t 1 -o supervisor -p user_defaults -c |
	 sed '2s/.*/ME/'|
	 sed '3s/.*/LOGIN_CONTROL/'|
	 nwbpset

       With this command, the property user_defaults of the user object 'supervisor' is copied into the property login_control of the user  object
       'me'.

	 nwbpvalues -t 1 -o me -p login_control -c |
	 sed '9s/.*/ff/'|
	 nwbpset

       This command disables the user object me.

       Feel free to contribute other examples!

       nwbpset looks up the file $HOME/.nwclient to find a file server, a user name and possibly a password. See nwclient(5) for more information.
       Please note that the access permissions of $HOME/.nwclient MUST be 600 for security reasons.

OPTIONS

       -h
	  -h is used to print out a short help text.

       -S server
	  server is the name of the server you want to use.

       -U user
	  user is the user name to use for login.

       -P password
	  password is the password to use for login. If neither -n nor -P are given, and the user has no open connection to  the  server,  nwbpset
	  prompts for a password.

       -n
	  -n should be given if no password is required for the login.

       -C
	  By default, passwords are converted to uppercase before they are sent to the server, because most servers require this. You can turn off
	  this conversion by -C.

AUTHORS

       nwbpset was written by Volker Lendecke. See the Changes file of ncpfs for other contributors.

nwbpset 							     8/7/1996								NWBPSET(1)

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Command/Script to remove duplicate lines from the file?

Discussion started by: Rahulpict

2. Shell Programming and Scripting

Sed or Awk to remove specific lines

Discussion started by: Shoeless_Mike

3. Shell Programming and Scripting

How to remove lines before and after with awk / sed ?

Discussion started by: ashimada

4. Shell Programming and Scripting

perl/shell need help to remove duplicate lines from files

Discussion started by: arvindng