Login or Register to Ask a Question and Join Our Community


UNIX for Dummies Questions & Answers

Print unique lines without sort or unique

 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers Print unique lines without sort or unique
# 1  
Old 09-13-2013
Print unique lines without sort or unique
I would like to print unique lines without sort or unique. Unfortunately the server I am working on does not have sort or unique. I have not been able to contact the administrator of the server to ask him to add it for several weeks.
# 2  
Old 09-13-2013
If it doesn't have those very basic utilities, what does it have?

Code:
awk '! ($0 in X) { X[$0]; print }' inputfile

# 3  
Old 09-14-2013
Quote:
Originally Posted by Corona688
If it doesn't have those very basic utilities, what does it have?

Code:
awk '! ($0 in X) { X[$0]; print }' inputfile

It does have awk Smilie. I use awk on a regular basis. I would go crazy without awk.

What is the best way to check what basic utilities it has?
# 4  
Old 09-14-2013
Those missing programs are core utilities. Why would they be missing? Which OS is this?
# 5  
Old 09-14-2013
Quote:
Originally Posted by Scott
Those missing programs are core utilities. Why would they be missing? Which OS is this?
No idea. This seems like some bare minimum installation. I just discovered 7z is also missing.

Which OS is this?
No idea. The usual commands for checking are missing.

Code:
$ lsb_release -a
-bash: lsb_release: command not found
$ uname -a
Linux 2.6.32-358.18.1.el6.x86_64 #1 SMP Fri Aug 2 17:04:38 EDT 2013 x86_64 x86_64 x86_64 GNU/Linux

Its some distro that has yum.
# 6  
Old 09-15-2013
RHEL 6.4
This and the package could be further determined by
Code:
cat /etc/*release
rpm -qf /usr/bin/sort

# 7  
Old 09-15-2013
Sure that sort and uniq are not on your system? Incorrect PATH? Use the find command to check for these binaries.
 
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Print number of lines for files in directory, also print number of unique lines

I have a directory of files, I can show the number of lines in each file and order them from lowest to highest with: wc -l *|sort 15263 Image.txt 16401 reference.txt 40459 richtexteditor.txt How can I also print the number of unique lines in each file? 15263 1401 Image.txt 16401... (15 Replies)
Discussion started by: spacegoose
15 Replies

2. UNIX for Beginners Questions & Answers

Print lines based upon unique values in Nth field

For some reason I am having difficulty performing what should be a fairly easy task. I would like to print lines of a file that have a unique value in the first field. For example, I have a large data-set with the following excerpt: PS003,001 MZMWR/ L-DWD// * PS003,001... (4 Replies)
Discussion started by: jvoot
4 Replies

3. Shell Programming and Scripting

Sort unique

Hi, I have an input file that I have sorted in a previous stage by $1 and $4. I now need something that will take the first record from each group of data based on the key being $1 Input file 1000AAA|"ZZZ"|"Date"|"1"|"Y"|"ABC"|""|AA 1000AAA|"ZZZ"|"Date"|"2"|"Y"|"ABC"|""|AA... (2 Replies)
Discussion started by: Ads89
2 Replies

4. Shell Programming and Scripting

Print lines in which value in specified Col is NOT unique

Hi everyone, I have the following file, which is a 3 column tab-delineated. cat big 24 cat small 13 cat red 63 dog big 34 chicken plays 39 fish red 294 I would like to print only those lines, in which the value in Col2 is repeated. Thus, given the above input file, the desired... (7 Replies)
Discussion started by: owwow14
7 Replies

5. Shell Programming and Scripting

Change unique file names into new unique filenames

I have 84 files with the following names splitseqs.1, spliseqs.2 etc. and I want to change the .number to a unique filename. E.g. change splitseqs.1 into splitseqs.7114_1#24 and change spliseqs.2 into splitseqs.7067_2#4 So all the current file names are unique, so are the new file names.... (1 Reply)
Discussion started by: avonm
1 Replies

6. Shell Programming and Scripting

Compare multiple files and print unique lines

Hi friends, I have multiple files. For now, let's say I have two of the following style cat 1.txt cat 2.txt output.txt Please note that my files are not sorted and in the output file I need another extra column that says the file from which it is coming. I have more than 100... (19 Replies)
Discussion started by: jacobs.smith
19 Replies

7. Shell Programming and Scripting

Compare Tab Separated Field with AWK to all and print lines of unique fields.

Hi. I have a tab separated file that has a couple nearly identical lines. When doing: sort file | uniq > file.new It passes through the nearly identical lines because, well, they still are unique. a) I want to look only at field x for uniqueness and if the content in field x is the... (1 Reply)
Discussion started by: rocket_dog
1 Replies

8. Shell Programming and Scripting

Unique sort with two fields

I have a file with contents below 123,502 123,506 123,702 234,101 235,104 456,104 456,100 i want to sort such that i get a unique value in column A, and for those with multiple value in A, i want the lowest value in B. output should be 123,502 234,101 235,104 456,100 (3 Replies)
Discussion started by: dealerso
3 Replies

9. Shell Programming and Scripting

Awk sort and unique

Input file --------- 12:name1:|host1|host1|host2|host1 13:name2:|host1|host1|host2|host3 14:name3: ...... Required output --------------- 12:name1:host1(2)|host1(1) 13:name2:host1(2)|host2(1)|host3(1) 14:name3: where (x) - Count how many times field appears in last column ... (3 Replies)
Discussion started by: greycells
3 Replies

10. Shell Programming and Scripting

get part of file with unique & non-unique string

I have an archive file that holds a batch of statements. I would like to be able to extract a certain statement based on the unique customer # (ie. 123456). The end for each statement is noted by "ENDSTM". I can find the line number for the beginning of the statement section with sed. ... (5 Replies)
Discussion started by: andrewsc
5 Replies
Login or Register to Ask a Question