08-01-2010
Filtering Unique Lines
Use and complete the template provided. The entire template must be completed. If you don't, your post may be deleted!
1. The problem statement, all variables and given/known data:
The uniq command excludes consecutive duplicate lines. It has a -c option to display a count of the number of occurances of each line.
-using the man pages as necessary, analyze the following command:
history | cut -c 8-100 | cut -d " " -f 1 | sort | uniq -c
2. Relevant commands, code, scripts, algorithms:
3. The attempts at a solution (include all code and scripts):
This was the results from running the command. I have no clue what the command is doing and how the it got the results. Could someone please explain what this command is doing?
william.makarevich@socrates:/home/801NANBEA$ history | cut -c 8-100 | cut -d " " -f 1 | sort | uniq -c
2 ~
2 Andy
2 Anne
6 awk
5 cal
25 cat
11 cd
1 chgrp
2 Claudia
27 clear
1 ~clear
1 cls
7 cp
1 cut
2 %D
2 Danny
85 date
1 ~date
1 del
2 Dennis
3 diff
6 echo
1 env
8 exit
2 grep
1 groups
4 head
1 hist
16 history
2 hostname
1 id
2 Jeff
2 Lewis
2 Lisa
1 list
25 ls
51 man
1 ~man
2 Mickey
2 Mini
3 mkdir
1 more
1 msn
2 mv
1 ps
14 pwd
3 q
1 :q
1 Q
1 QQ
4 quit
1 :quit
2 Rex
4 rm
1 rmdir
2 Robert
2 Ruby
1 Sat,
2 Sherelene
4 sort
2 spell
4 tail
2 Tony
1 touch
2 Vermont
14 vi
5 wc
2 which
2 Wilbert
2 william.makarevich@socrates:~$
2 Williams
2 Yoga
2 ZZ
4. Complete Name of School (University), City (State), Country, Name of Professor, and Course Number (Link to Course):
Colorado Tech University
Colorado Springs, CO, USA
Professor: Matt Hodges
Course # CS146
Note: Without school/professor/course information, you will be banned if you post here! You must complete the entire template (not just parts of it).
10 More Discussions You Might Find Interesting
1. UNIX for Advanced & Expert Users
Does anybody know a command that filters duplicate lines out of a file. Similar to the uniq command but can handle duplicate lines no matter where they occur in a file? (9 Replies)
Discussion started by: AreaMan
9 Replies
2. Shell Programming and Scripting
Hi all! Im wondering if its possible to remove all lines between two lines. Im working with a document like this:
data1
data2
<Remove>
data3
data4
</Remove>
data5
data6
I need it to end up like this if that possible:
data1
data2
data5
data6
There are multiple instances of... (2 Replies)
Discussion started by: Grizzly
2 Replies
3. UNIX for Dummies Questions & Answers
I received this question for homework:
We have to write our program into a .sh file, with "#!/bin/bash" as the first line. And we have the list of access logs in a file, looking like this (it's nearly 10,000 lines long):
65.214.44.112 - - "GET /~user0/cgg/msg08400.html HTTP/1.0" 304 -... (1 Reply)
Discussion started by: Andrew9191
1 Replies
4. UNIX for Advanced & Expert Users
Hi All,
I have a very huge file (4GB) which has duplicate lines. I want to delete duplicate lines leaving unique lines. Sort, uniq, awk '!x++' are not working as its running out of buffer space.
I dont know if this works : I want to read each line of the File in a For Loop, and want to... (16 Replies)
Discussion started by: krishnix
16 Replies
5. Shell Programming and Scripting
Hi All,
I am trying to filter out a txt file containing 2 columns. Which has got website name and response time as below.
Website_name response_time
yahoo 22
google 21
yahoo 34
rediff 45
google 43
rediff 31
I want filter the above file with unique website names which has got highest... (5 Replies)
Discussion started by: anilreddy103
5 Replies
6. UNIX for Dummies Questions & Answers
I have a table to data which one of the columns include string of text
from within that, I am searching to include few lines but not others
for example I want to to include some combination of word address such as (address.| address? |the address | your address) but not (ip address | email... (17 Replies)
Discussion started by: A-V
17 Replies
7. Shell Programming and Scripting
Hello to all, happy new year 2013!
May somebody could help me, is about a very similar problem to the problem I've posted here where the member rdrtx1 and bipinajith helped me a lot.
https://www.unix.com/shell-programming-scripting/211147-map-values-blocks-single-line-2.html
It is very... (3 Replies)
Discussion started by: Ophiuchus
3 Replies
8. Shell Programming and Scripting
Hi Guys,
Would need your expert help with the following situation..
I have a comma seperated .csv file, with a header row and data as follows
H1,H2,H3,H4,H5..... (header row)
0,0,0,0,0,1,2.... (data rows follow)
0,0,0,0,0,0,1
.........
.........
i need a code... (10 Replies)
Discussion started by: dev.devil.1983
10 Replies
9. Shell Programming and Scripting
Experts Good day,
I want to filter multiple lines of same error of same day , to only 1 error of each day, the first line from the log.
Here is the file:
May 26 11:29:19 cmihpx02 vmunix: NFS write failed for server cmiauxe1: error 5 (RPC: Timed out)
May 26 11:29:19 cmihpx02 vmunix: NFS... (4 Replies)
Discussion started by: rveri
4 Replies
10. UNIX for Dummies Questions & Answers
I would like to print unique lines without sort or unique. Unfortunately the server I am working on does not have sort or unique. I have not been able to contact the administrator of the server to ask him to add it for several weeks. (7 Replies)
Discussion started by: cokedude
7 Replies
0intro 0intro intro 0intro 2a 2a 6a 2a 8a 2a ka 2a va 2a xa 2a 2c
2c 6c 2c 8c 2c kc 2c vc 2c xc 2c 2l 2l 6l 2l 8l 2l kl 2l vl 2l xl
2l 81/2 81/2 label 81/2 window 81/2 wloc 81/2 acid acid acme acme
awd acme win acme 8al alef alef alef kal alef val alef ar ar art
art art2pic art ascii ascii unicode ascii awk awk basename base-
name bc bc bind bind mount bind unmount bind bundle bundle c++
c++ c++/2c c++ c++/2l c++ c++/8c c++ c++/8l c++ c++/kc c++ c++/kl
c++ c++/vc c++ c++/vl c++ cal cal calendar calendar cat cat read
cat char char rschar char 1/2char char chgrp chgrp chmod chmod
cmp cmp comm comm con con cu con rx con telnet con xmr con xms
con cp cp mv cp cpp cpp cpu cpu date date db db dc dc dd dd dela-
tex deroff deroff deroff diff diff doctype doctype du du echo
echo ed ed emacs emacs eqn eqn factor factor primes factor file
file fmt fmt fone fone fortune fortune freq freq 4s games 5s
games ana games catclock games clock games festoon games fire-
works games fsim games games games life games mandel games plumb
games smiley games swar games grap grap graph graph grep grep gs
gs hoc hoc hp hp join join broke kill kill kill ktrans ktrans lex
lex look look lp lp lc ls ls ls aliasmail mail edmail mail mail
mail seemail mail sendmail mail smtp mail smtpd mail to mail vis-
mon mail vwhois mail lookman man man man mc mc membername mk mk
mk mkdir mkdir mothra mothra netstat netstat news news nm nm p p
page page netkey passwd passwd passwd pcc pcc pic pic tpic pic
plot plot ppp ppp pppclient ppp pppserver ppp pr pr kprof prof
prof prof proof proof ps ps psu ps pbd pwd pwd pwd cd rc eval rc
exec rc exit rc flag rc rc rc rfork rc shift rc wait rc whatis rc
~ rc rm rm B sam sam sam sam.save sam sed sed seq seq size size
sleep sleep sort sort spell spell sprog spell spin spin split
split start stop stop stop strings strings strip strip md5sum sum
sum sum syscall syscall tail tail 32vfs tapefs cpiofs tapefs
tapefs tapefs tapfs tapefs tarfs tapefs tpfs tapefs v10fs tapefs
v6fs tapefs tar tar tbl tbl tcs tcs tee tee iwhois tel tel tel
test test bibtex tex dvips tex dviselect tex latex tex mf tex tex
tex time time touch touch tr tr nroff troff troff troff tweak
tweak twig twig uniq uniq units units ki vi vi vi xi vi wc wc who
who whois who xd xd yacc yacc yesterday yesterday