Login or Register to Ask a Question and Join Our Community


UNIX for Dummies Questions & Answers

find and delete reccuring entries in a file

 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers find and delete reccuring entries in a file
# 1  
Old 06-18-2007
find and delete reccuring entries in a file
I have a file with multiple lines, lets say for example

magazine1
magazine2
magazine3
magazine3
magazine4
magazine5

So i am trying to find a way to find and delete the duplicate entries.

Any ideas?

Thanks
# 2  
Old 06-18-2007
Did you try uniq command ?
Code:
  uniq < file > file.out

Thanks,
Nagarajan Ganesan
# 3  
Old 06-18-2007
thanks man you delivered!

At first a i was disappointed because it didn't work but then i thought that maybe a sort before should do the work, and it did!

Many thanks.

It's amazing how many unknown commands could possibly exist and save you so much trrouble.
Do you by any chance have a favorite site that displays with examples the majority of them?
# 4  
Old 06-18-2007
Greekozz,
There are several sites that have all Unix commands.
The problem is that the amount of information is overwhelming.
Only practical years of experience will help you remember what command
to use.
What I recommend you to do, it to keep watching the solutions in this site.
Good luck!
 
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Find and delete a certain HEX and its following value in a file

Hello there, I've been trying to do this half of the day and it's like I haven't come a single step further, so I hope you guys can help me with my problem: I have a text file that contains strings that should not be there and which I want to delete automatically from the command line. The... (4 Replies)
Discussion started by: surfi
4 Replies

2. Shell Programming and Scripting

Find if file is 45 min old if yes delete

Hello, we have process2(my_pro.sh) which check for the file Pr_rn_Chk (generated by other process1). The process runs every 15 min. if exist Pr_rn_Chk the process will exit. Else it will continue. I want to check if the file Pr_rn_Chk is older than 45 min delete the file. so that 4th run... (3 Replies)
Discussion started by: kumar30213
3 Replies

3. Shell Programming and Scripting

Find and delete file previous to some date

Hello All, I have a directory containing of many .dat file, but with different naming conventions. I want to delete files which are created before and on 10th September 2013. I tried this command to see files which are are the files created before 10th September find path/to/file -type... (6 Replies)
Discussion started by: nnani
6 Replies

4. Shell Programming and Scripting

Script to delete old entries from a file

I need to remove the file entries that are 30 days old. I have a file with the below data. 111.XX.XX.99 - - "GET /folder1/wp-content" 304 111.XX.XX.99 - - "GET /folder1/wp-content" 304 111.XX.XX.99 - - "GET /folder1/wp-content" 304 111.XX.XX.99 - - "GET /folder1/wp-content" 304... (5 Replies)
Discussion started by: anil510
5 Replies

5. Shell Programming and Scripting

Find the text in the file and delete the rest of the line.

Hi, I have one requiremnet like this. I need to find some particular string (eg.IncludeDateTime = ) in a file. And wherever it finds the string the unix script has to delete the remaining text coming after the string (ie., 'IncludeDateTime = ' ) in the same line. I tried to write this script in... (5 Replies)
Discussion started by: jannusuresh
5 Replies

6. Shell Programming and Scripting

Delete old date entries from a file

Hello, I have following log file. alog.log Oct 8 xxxxx Oct 9 xxxxx Oct 10 xxxxx Nov 8 xxxxx Nov 8 xxxxx . . . Jan 8 xxxxx I need to delete all the lines which are having date entries older than 60 days. So If the current date is Jan 9 2011 then it should delete all the lines upto... (3 Replies)
Discussion started by: Ravin83
3 Replies

7. Shell Programming and Scripting

To find a word in a file and delete the file

Hi, I need to find a word in a file, and I need to delete the file where the word resides on the file. I did try with the grep -H "word" using the find command, but no luck. (6 Replies)
Discussion started by: gsiva
6 Replies

8. UNIX for Dummies Questions & Answers

Script to change/find/delete/install a specific file

Hi Very much a newbie to UNIX & scripting, but have identified an area within work that would benefit from being automated, as its repeated manually very often, and it looks like the ideal first script! What I need to do is change directory to a users home (cd ~), and then find and remove a... (6 Replies)
Discussion started by: Great Uncle Kip
6 Replies

9. UNIX for Dummies Questions & Answers

find and delete a file in one command

hi , let's assume i have a file that is located in the l*** directory and this file's name is t****_***s.php , how can i find this file and delete it using one single command ????? (3 Replies)
Discussion started by: wewee
3 Replies

10. Shell Programming and Scripting

find full directory and delete old dated file?

I want to write a code that will look for a particular directory which is full or more than 95% used, and will search older file and delete it. My code looks like this: df -k|while read f1 f2 f3 f4 f5 f6 do if expr '$f5' -ge '95%' || continue then echo $f5 else echo "there is no... (2 Replies)
Discussion started by: xramm
2 Replies
Login or Register to Ask a Question