Remove lines Post: 302958777

Sponsored Content

Top Forums UNIX for Dummies Questions & Answers Remove lines Post 302958777 by RavinderSingh13 on Monday 26th of October 2015 09:55:09 AM

10-26-2015

Moderator

Hello krsnadasa,

Following is the explanation for above code, hope it may help you in same.

Code:

awk -F"|" '                     ######## Making field seprator as |
!arr[$1]++                      ######## creating an array named arr whose index is $1's value so ! arr[$1] means if any occurrence of $1 which is NOT present in 
                                        array arr then do it will make entry of that particular item into array an make it's value as one so next time it can't be taken 
                                        as we have put condition as ! arr[$1] 
                                        so if this condition is TRUE means when an entry is being added for $1 then no action is mentioned so it will print that line.
' OFS="|" file                  ######## mentioning Output field separator as | , reading Input_file as named file too

Thanks,
R. Singh

This User Gave Thanks to RavinderSingh13 For This Post:

RavinderSingh13

View Public Profile for RavinderSingh13

Find all posts by RavinderSingh13

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Need to remove lines

Hi All, I have a xml file . And I need to remove the header inofrmation and the tralier information from the xml file. I have pasted the sample xml below: <?xml version="1.0" encoding="UTF-8" ?> - <!-- Generated by Ascential Software Corporation, DataStage - XMLOutput stage - Mon Nov...

2. Shell Programming and Scripting

Remove All Lines Between Two Unique Lines

Hi all! Im wondering if its possible to remove all lines between two lines. Im working with a document like this: data1 data2 <Remove> data3 data4 </Remove> data5 data6 I need it to end up like this if that possible: data1 data2 data5 data6 There are multiple instances of...

3. Shell Programming and Scripting

Remove lines

i want to remove the records starting with abc or xyz in the file without redirecting the original file. sample : file1.txt abc1234 124234 2020202 3242342 xyz2342 2afafa Expecting output 124234 2020202 3242342 2afafa

4. Shell Programming and Scripting

Remove all lines except lines starting with [

Hello, I am trying to remove all the lines in file except lines starting with [ How can i accomplish this? Thank you very much in advance.

5. Shell Programming and Scripting

remove blank lines and merge lines in shell

Hi, I'm not a expert in shell programming, so i've come here to take help from u gurus. I'm trying to tailor a csv file that i got to make it work for the LOAD FROM command. I've a datatable csv of the below format - --in file format xx,xx,xx ,xx , , , , ,,xx, xxxx,, ,, xxx,...

6. Shell Programming and Scripting

Remove x lines form top and y lines form bottom using AWK?

How to remove x lines form top and y lines form bottom. This works, but like awk only cat file | head -n-y | awk 'NR>(x-1)' so remove last 3 lines and 5 firstcat file | head -n-3 | awk 'NR>4'

7. UNIX for Dummies Questions & Answers

Want to remove all lines but not latest 50 lines from a file

Hi, I have a huge file which has Lacs of lines. File system got full. I want your guys help to suggest me a solution so that I can remove all lines from that file but not last 50,000 lines. I want solution which can remove lines from existing file so that I can have some space left with.

8. Shell Programming and Scripting

Two files, remove lines from second based on lines in first

I have two files, a keepout.txt and a database.csv. They're unsorted, but could be sorted. keepout: user1 buser3 anuser19 notheruser27 database: user1,2343,"information about",field,blah,34 user2,4231,"mo info",etc,stuff,43 notheruser27,4344,"hiya",thing,more thing,423...

9. Shell Programming and Scripting

Remove lines that are subsets of other lines in File

Hello everyone, Although it seems easy, I've been stuck with this problem for a moment now and I can't figure out a way to get it done. My problem is the following: I have a file where each line is a sequence of IP addresses, example : 10.0.0.1 10.0.0.2 10.0.0.5 10.0.0.1 10.0.0.2...

10. Shell Programming and Scripting

awk to remove lines that do not start with digit and combine line or lines

I have been searching and trying to come up with an awk that will perform the following on a converted text file (original is a pdf). 1. Since the first two lines are (begin with) text they are removed 2. if $1 is a number then all text is merged (combined) into one line until the next...

LEARN ABOUT OSX

colrm

COLRM(1)						    BSD General Commands Manual 						  COLRM(1)

NAME

     colrm -- remove columns from a file

SYNOPSIS

     colrm [start [stop]]

DESCRIPTION

     The colrm utility removes selected columns from the lines of a file.  A column is defined as a single character in a line.  Input is read
     from the standard input.  Output is written to the standard output.

     If only the start column is specified, columns numbered less than the start column will be written.  If both start and stop columns are spec-
     ified, columns numbered less than the start column or greater than the stop column will be written.  Column numbering starts with one, not
     zero.

     Tab characters increment the column count to the next multiple of eight.  Backspace characters decrement the column count by one.

ENVIRONMENT

     The LANG, LC_ALL and LC_CTYPE environment variables affect the execution of colrm as described in environ(7).

EXIT STATUS

     The colrm utility exits 0 on success, and >0 if an error occurs.

SEE ALSO

     awk(1), column(1), cut(1), paste(1)

HISTORY

     The colrm command appeared in 3.0BSD.

BSD
								  August 4, 2004							       BSD

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Need to remove lines

Discussion started by: kumar66

2. Shell Programming and Scripting

Remove All Lines Between Two Unique Lines

Discussion started by: Grizzly

3. Shell Programming and Scripting

Remove lines

Discussion started by: Jairaj

4. Shell Programming and Scripting

Remove all lines except lines starting with [

Discussion started by: maxo