Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Delete lines inside a file.

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Delete lines inside a file.
# 1  
Old 12-02-2008
Delete lines inside a file.
hi....

I have a file having more then thousand lines.
i want to remove selected lines in it.
And also if there exists two duplicate lines, I want to delete one of them.
Please help me with awk and shell.

Smilie
# 2  
Old 12-02-2008
sed -n 'G; s/\n/&&/; /^\([ -~]*\n\).*\n\1/d; s/\n//; h; P'
should remove duplicate lines.
what are selected lines ? by Number ?
# 3  
Old 12-03-2008
Didn`t worked
Hi again,

1) It didn`t workSmilie, my objective is to remove duplicate lines inside a file.
first echo duplicate lines and then remove them.

2) Also how can i delete a particular line from a file? when i don`t know the line number. What I know is the content to be deleted.



Thanks in advance......Smilie
# 4  
Old 12-04-2008
No good question = No response.

You should think out things before asking.
# 5  
Old 12-04-2008
Code:
uniq -d filename

will display the repeated lines
Code:
uniq filename outfilename

this will remove all duplicate lines from your file
Code:
sed '/your_string_to_be_deleted/d' filename>outfilename

will remove the required line from your file
# 6  
Old 12-04-2008
Hi vidhyadhar,

--> uniq infile outfile

the above command removes the repeated lines from the file 'infile' only if the lines are repeated consequtively..
for example:
if infile contains the following contents

one
two
three
five
four
five
one

the uniq cmd wont remove the repeated lines one and five!!
# 7  
Old 12-04-2008
try this to display duplicate lines

Code:
echo `awk 'a[$0]++' try-file.txt`

to remove duplicate lines:
Code:
 awk '! a[$0]++' try-file.txt

to print lines which have a pattern:
Code:
awk '/pattern/'
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Delete 40 lines after every 24 lines from a file

Hello, I have file of more than 10000 lines. I want to delete 40 lines after every 20 lines. e.g from a huge file, i want to delete line no from 34 - 74, then 94 - 134 etc and so on. Please let me know how i can do it. Best regards, (11 Replies)
Discussion started by: nehashine
11 Replies

2. Shell Programming and Scripting

Delete the lines from file

I have a text file say g1.txt and content of this file is ...as below 1|HideCDrive, | : REG ADD "HKCU\Software\Microsoft\Windows\CurrentVersion\Policies\Explorer" /v NoDrives /t REG_DWORD /d 4 /f, 2|HideRunButton, | : REG ADD "HKCU\Software\Microsoft\Windows\CurrentVersion\Policies\Explorer"... (7 Replies)
Discussion started by: jalpasoni
7 Replies

3. UNIX for Advanced & Expert Users

In a huge file, Delete duplicate lines leaving unique lines

Hi All, I have a very huge file (4GB) which has duplicate lines. I want to delete duplicate lines leaving unique lines. Sort, uniq, awk '!x++' are not working as its running out of buffer space. I dont know if this works : I want to read each line of the File in a For Loop, and want to... (16 Replies)
Discussion started by: krishnix
16 Replies

4. Shell Programming and Scripting

KSH SHELL: problem calculation number of lines inside compressed file

Hi Gurus, I am working with a korn shell script to simplify some operations of calculation number of lines inside compressed file. The called function (inside a cycle) is the following: ######################################### # F.ne: CheckCount #########################################... (3 Replies)
Discussion started by: GERMANICO
3 Replies

5. UNIX for Dummies Questions & Answers

How get only required lines & delete the rest of the lines in file

Hiiii I have a file which contains huge data as a.dat: PDE 1990 1 9 18 51 28.90 24.7500 95.2800 118.0 6.1 0.0 BURMA event name: 010990D time shift: 7.3000 half duration: 5.0000 latitude: 24.4200 longitude: 94.9500 depth: 129.6000 Mrr: ... (7 Replies)
Discussion started by: reva
7 Replies

6. Shell Programming and Scripting

How to delete lines in a file that have duplicates or derive the lines that aper once

Input: a b b c d d I need: a c I know how to get this (the lines that have duplicates) : b d sort file | uniq -d But i need opossite of this. I have searched the forum and other places as well, but have found solution for everything except this variant of the problem. (3 Replies)
Discussion started by: necroman08
3 Replies

7. Shell Programming and Scripting

Count lines between two patterns inside a file

Hi, Im doing a script to find the number of lines included inside a file newly. These lines are in between #ifdef FLAG1 and #else or #endif or #else and #endif. I tried like this, awk '/#ifdef Flag1/,/#e/{print}' aa.c | wc -l awk '/#ifndef Flag1/,/#endif/{print}' aa.c | awk... (6 Replies)
Discussion started by: priyadarshini
6 Replies

8. Shell Programming and Scripting

Deleting lines inside a file without opening the file

Hi, Just consider there are around 10 lines in a file. Now is it possible to delete the first 2 lines in the file without opening the file. No matter whatever the content of the file is, I just wanna delete the first 2 lines without opening the file. Is that possible? If so, please help me out.... (3 Replies)
Discussion started by: toms
3 Replies

9. UNIX for Dummies Questions & Answers

delete lines in a file

I've got a file like this: Grid-ref= 443, 229 167 169 204 233 290 309 308 326 300 251 194 161 148 189 228 251 296 329 331 338 308 263 219 179 178 203 215 252 277 319 327 335 312 264 196 149 120 172 226 253 297 329 323 322 305 242 203 136 ... (20 Replies)
Discussion started by: su_in99
20 Replies

10. Shell Programming and Scripting

Delete lines in a file

I wish to delete lines from a log file. Should take a date string as a variable. Find the date in the file, get the line number for the string. Delete lines from 1 to that (line number - 1), all from within a shell script. Is that possible ?. Thnks, :confused: (6 Replies)
Discussion started by: umal
6 Replies
Login or Register to Ask a Question