Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

How to keep just one instance in input file?

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting How to keep just one instance in input file?
# 1  
Old 01-11-2015
How to keep just one instance in input file?
Hello, I have the following dataset

Code:
1
1
2
2
2
3
3
4

I want to keep just one instance of each number seen:

Code:
1
2
3
4

Is there a way to do this with sed to delete multiple instances?

Any help is appreciated!
# 2  
Old 01-11-2015
Maybe. Is there a special reason to use sed?

With awk it would be very simple:
Code:
awk '!A[$1]++' file

or with your example if it is sorted or at least grouped:
Code:
uniq file

or otherwise:
Code:
sort -nu file

Last edited by Scrutinizer; 01-11-2015 at 11:35 PM..
# 3  
Old 01-11-2015
Code:
perl -nle 'unless($hash{$_}++){print $_}' file

Perl of wisdom
# 4  
Old 01-12-2015
Hello Rabu,

Following may also help in same.
Code:
perl -ne 'print unless $dup{$_}++;'  Input_file

Output will be as follows in given input.
Code:
1
2
3
4

Thanks,
R. Singh
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Homework & Coursework Questions

Removing punctuations from file input or standard input

Just started learning Unix and received my first assignment recently. We haven't learned many commands and honestly, I'm stumped. I'd like to receive assistance/guidance/hints. 1. The problem statement, all variables and given/known data: How do I write a shell script that takes in a file or... (4 Replies)
Discussion started by: fozilla
4 Replies

2. Shell Programming and Scripting

Read input files and merge them in given order and write them to input one param or one file

Dear Friends, I am looking for a shell script to merge input files into one file .. here is my idea: 1st paramter would be outfile file (all input files content) read all input files and merge them to input param 1 ex: if I pass 6 file names to the script then 1st file name as output file... (4 Replies)
Discussion started by: hyd1234
4 Replies

3. Shell Programming and Scripting

Script to delete files with an input for directories and an input for path/file

Hello, I'm trying to figure out how best to approach this script, and I have very little experience, so I could use all the help I can get. :wall: I regularly need to delete files from many directories. A file with the same name may exist any number of times in different subdirectories.... (3 Replies)
Discussion started by: *ShadowCat*
3 Replies

4. Shell Programming and Scripting

How can I just grep one instance of a word in the file

I want to grep some information out of the dmidecode but when I type dmidecode | grep Memory I get several instances of the word. Is there a way I can just choose which instance I want to display? (8 Replies)
Discussion started by: jcnewton13
8 Replies

5. Shell Programming and Scripting

Search for last instance of word in a file

Hi I'm trying to search for the last instance of the word 'cache' in a HTML file that I have downloaded from YouTube. I'm using the following syntax, but an error is thrown when I try it. grep -f "cache" Also I wish to append the above grep command to the below so that the search for cache... (3 Replies)
Discussion started by: colmbell
3 Replies

6. Shell Programming and Scripting

Need script to take input from file, match on it in file 2 and input data

All, I am trying to figure out a script to run in windows that will allow me to match on First column in file1 to 8th Column in File2 then Insert file1 column2 to file2 column4 then create a new file. File1: 12345 Sam 12346 Bob 12347 Bill File2:... (1 Reply)
Discussion started by: darkoth
1 Replies

7. Shell Programming and Scripting

Search the last instance of a string in a file

I have a big database log file which keepsgrowing daily. OS is HP UX. Here is a small part of it: Tue Jan 27 04:03:22 2009 : add session begin for mfgeb on /dev/pts/th. : Converting relative path to absolute path. : add session end. Tue Jan 27 04:03:29... (6 Replies)
Discussion started by: dinesh1178
6 Replies

8. Shell Programming and Scripting

Reading specific contents from 1 input files and appending it to another input file

Hi guys, I am new to AWK and unix scripting. Please see below my problem and let me know if anyone you can help. I have 2 input files (example given below) Input file 2 is a standard file (it will not change) and we have to get the name (second column after comma) from it and append it... (5 Replies)
Discussion started by: sksahu
5 Replies

9. Shell Programming and Scripting

replace first instance(not first instance in line)

Alright, I think I know what I am doing with sed(which probably means I don't). But I cant figure out how to replace just the first occurance of a string. I have tried sed, ed, and grep but can't seem to figure it out. If you have any suggestions I am open to anything! (3 Replies)
Discussion started by: IronHorse7
3 Replies

10. Shell Programming and Scripting

first instance to end of log file

All, I looking for developing a script which would find the first instance of a date in a log file: 2004/07/15 19:16:09 Here are the next couple lines after the dat time stamp: backup completed daemon started. And I want to put the rest of the lines after the first instance in this... (2 Replies)
Discussion started by: bubba112557
2 Replies
Login or Register to Ask a Question