Classwork (group project)


 
Thread Tools Search this Thread
Homework and Emergencies Homework & Coursework Questions Classwork (group project)
# 1  
Old 10-19-2015
Classwork (group project)

Use and complete the template provided. The entire template must be completed. If you don't, your post may be deleted!



1. The problem statement, all variables and given/known data:

Using the example.log file in your home directory, write a bash script that will collect the following information:

1. Date range of the log. This would be the first (oldest) and last (newest) date in the file.
2. Total number of hits during the date range. 1 hit = 1 line
3. Total number of unique visitors. A unique visitor = a unique IP address.
4. Top 10 users, and the number of times they visited.
5. Five most popular resource accessed, and the number of times they were accessed.
6. Ten most visited URLs, and the number of time accessed.
7. Number of visitors using Internet Explorer, broken down by version.
8. Number of people using Firefox, broken down by version.
9. Number of people using any other browser.

THE FORMAT OF THE LOG:

Each line in the log file is equal to one visit, or hit. The log is a tab-delimited file with 6 columns of data. It will be necessary to extract data from a specific column and break it down further. The columns in the log file are in the order below, and contain the following data:

1. IP address of the visitor.
2. Username of the visitor
3. Date and time the user visited.
4. The access method, resource, and protocol used.
5. URL accessed by the user.
6. The ‘User Agent’ string containing browser and other system information from the user.

2. Relevant commands, code, scripts, algorithms:
• bash
• head
• tail
• cut
• tr
• sed
• wc
• sort
• uniq
• grep
• printf
cut & sort) multiple times in one command chain

3. The attempts at a solution (include all code and scripts):
Code:
 $ head -n1 example.log | cut -d'h' -f1
 $ tail -n1 example.log | cut -d'h' -f1

(These for the date ranges)

Code:
grep -i "date" example.log | wc -l

(for the top hits but we believe this isnt completely correct)

also know that the sort and awk commands can be of use but dont know how to put it into a line.

Code:
cat /path/to/example.log |awk '{print $1}' | sort |uniq -c |sort -n |tail

(ip addresses, but not sure if correct either.)

4. Complete Name of School (University), City (State), Country, Name of Professor, and Course Number (Link to Course):

Pace university, New York, New York. United States. Professor Thomas Murphy, RH134
Note: Without school/professor/course information, you will be banned if you post here! You must complete the entire template (not just parts of it).

---------- Post updated at 10:54 AM ---------- Previous update was at 10:52 AM ----------

so it wont let me link the course =( and also wont let me link the data for this assignment. Anyone know a work around? says I need at least 5 post first

---------- Post updated at 10:55 AM ---------- Previous update was at 10:54 AM ----------

Code:
192.168.28.168	user143	[08/May/2010:09:52:52]	"GET /NoAuth/js/scriptaculous/scriptaculous.js?load=effects,controls HTTP/1.1"	"http://www.example.com/index.html"	"Mozilla/5.0 (Windows; U; Windows NT 5.1; rv:1.9.2.3) Gecko/20100401 Firefox/3.6.3 GTB7.0"
192.168.28.168	user147	[08/May/2010:09:52:52]	"GET /NoAuth/js/prototype/prototype.js HTTP/1.1"	"http://www.example.com/index.html"	"Mozilla/5.0 (Windows; U; Windows NT 5.1; rv:1.9.2.3) Gecko/20100401 Firefox/3.6.3 GTB7.0"
192.168.28.168	user174	[08/May/2010:09:52:52]	"GET /NoAuth/js/ahah.js HTTP/1.1"	"http://www.example.com/index.html"	"Mozilla/5.0 (Windows; U; Windows NT 5.1; rv:1.9.2.3) Gecko/20100401 Firefox/3.6.3 GTB7.0"
192.168.28.168	user82	[08/May/2010:09:52:52]	"GET /NoAuth/js/titlebox-state.js HTTP/1.1"	"http://www.example.com/index.html"	"Mozilla/5.0 (Windows; U; Windows NT 5.1; rv:1.9.2.3) Gecko/20100401 Firefox/3.6.3 GTB7.0"
192.168.28.168	user14	[08/May/2010:09:52:52]	"GET /NoAuth/css/validation.css HTTP/1.1"	"http://www.example.com/index.html"	"Mozilla/5.0 (Windows; U; Windows NT 5.1; rv:1.9.2.3) Gecko/20100401 Firefox/3.6.3 GTB7.0"
192.168.28.168	user129	[08/May/2010:09:52:52]	"GET /NoAuth/js/util.js HTTP/1.1"	"http://www.example.com/index.html"	"Mozilla/5.0 (Windows; U; Windows NT 5.1; rv:1.9.2.3) Gecko/20100401 Firefox/3.6.3 GTB7.0"
192.168.28.168	user162	[08/May/2010:09:52:52]	"GET /NoAuth/css/print.css HTTP/1.1"	"http://www.example.com/index.html"	"Mozilla/5.0 (Windows; U; Windows NT 5.1; rv:1.9.2.3) Gecko/20100401 Firefox/3.6.3 GTB7.0"
192.168.28.168	user35	[08/May/2010:09:52:52]	"GET /NoAuth/css/web2/main-squished.css HTTP/1.1"	"http://www.example.com/index.html"	"Mozilla/5.0 (Windows; U; Windows NT 5.1; rv:1.9.2.3) Gecko/20100401 Firefox/3.6.3 GTB7.0"
192.168.149.163	user44	[08/May/2010:09:51:30]	"GET /index.html HTTP/1.1"	"http://www.example.com/Ticket/Display.html?id=236821&results=54058c6bb77364e805a28b05cf401789"	"Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; .NET CLR 2.0.50727; .NET CLR 1.1.4322; .NET CLR 1.0.3705;)"
192.168.149.163	user137	[08/May/2010:09:51:30]	"GET / HTTP/1.1"	"http://www.example.com/Ticket/Display.html?id=236821&results=54058c6bb77364e805a28b05cf401789"	"Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; .NET CLR 2.0.50727; .NET CLR 1.1.4322; .NET CLR 1.0.3705;)"
192.168.149.163	user15	[08/May/2010:09:51:13]	"GET /Ticket/Display.html?id=236821&results=54058c6bb77364e805a28b05cf401789 HTTP/1.1"	"http://www.example.com/Ticket/Update.html?Action=Comment&id=236821"	"Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; .NET CLR 2.0.50727; .NET CLR 1.1.4322; .NET CLR 1.0.3705;)"
192.168.149.163	user101	[08/May/2010:09:51:12]	"POST /Ticket/Update.html HTTP/1.1"	"http://www.example.com/Ticket/Update.html?Action=Comment&id=236821"	"Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; .NET CLR 2.0.50727; .NET CLR 1.1.4322; .NET CLR 1.0.3705;)"
192.168.170.132	user195	[08/May/2010:09:43:52]	"GET /index.html HTTP/1.1"	"http://www.example.com/Ticket/Display.html?id=238759&results=16189b033b19ffdba5b07b0dddc11b85"	"Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; .NET CLR 1.0.3705; .NET CLR 1.1.4322; .NET CLR 2.0.50727; InfoPath.2)"
192.168.170.132	user43	[08/May/2010:09:43:52]	"GET / HTTP/1.1"	"http://www.example.com/Ticket/Display.html?id=238759&results=16189b033b19ffdba5b07b0dddc11b85"	"Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; .NET CLR 1.0.3705; .NET CLR 1.1.4322; .NET CLR 2.0.50727; InfoPath.2)"
192.168.170.132	user38	[08/May/2010:09:43:41]	"GET /Ticket/Display.html?id=238759&results=16189b033b19ffdba5b07b0dddc11b85 HTTP/1.1"	"http://www.example.com/Ticket/Update.html?Action=Comment&id=238759"	"Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; .NET CLR 1.0.3705; .NET CLR 1.1.4322; .NET CLR 2.0.50727; InfoPath.2)"
192.168.170.132	user192	[08/May/2010:09:43:41]	"POST /Ticket/Update.html HTTP/1.1"	"http://www.example.com/Ticket/Update.html?Action=Comment&id=238759"	"Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; .NET CLR 1.0.3705; .NET CLR 1.1.4322; .NET CLR 2.0.50727; InfoPath.2)"
192.168.170.132	user58	[08/May/2010:09:43:12]	"GET /Ticket/Update.html?Action=Comment&id=238759 HTTP/1.1"	"http://www.example.com/Ticket/Display.html?id=238759"	"Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; .NET CLR 1.0.3705; .NET CLR 1.1.4322; .NET CLR 2.0.50727; InfoPath.2)"
192.168.170.132	user11	[08/May/2010:09:43:07]	"GET /Ticket/Display.html?id=238759 HTTP/1.1"	"http://www.example.com/index.html"	"Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; .NET CLR 1.0.3705; .NET CLR 1.1.4322; .NET CLR 2.0.50727; InfoPath.2)"


Here is a little snippet of what the data I'm working with looks like.

Last edited by rbatte1; 10-19-2015 at 01:13 PM.. Reason: Added CODE tags for code & data block
# 2  
Old 10-19-2015
- With the sample given, the assumption for the first question on first and last line is wrong. On top, the delimiter for cut should be reconsidered.
- Your attempt to find the top hits (whatever this means) looks for the string constant "date"; not sure this is what you want.
- Your attempt to answer question 3 can be simplified and doesn't meet the question's kernel.

What about questions 4 - 10?
# 3  
Old 01-19-2016
1. Date range of the log. Hope they are sorted

Check the manual page for cut using

$ man cut

Code:
cut -f 
     -f list
             The list specifies fields, separated in the input by the field
             delimiter character (see the -d option).  Output fields are sepa-
             rated by a single occurrence of the field delimiter character.

So if we dont use -d just use -f then each word is separated by space

Code:
 $ tail -n1 exp.log | cut -f 3

Gets the date but with a [] sign .. can we use 'tr' command now ??
# 4  
Old 01-19-2016
Quote:
Originally Posted by chakrapani
[,,]
So if we dont use -d just use -f then each word is separated by space
[..]
That should be: by TAB character.
Code:
     -d delim
             Use delim as the field delimiter character instead of the tab character.

to use a single space: -d " "
# 5  
Old 01-20-2016
i tested same log (copy paste) and -f option and it worked for me with space.

on FreeBSD with /usr/bin/cut
# 6  
Old 01-21-2016
According to the man page of FreeBSD 11.0 :
Code:
     -d delim
	     Use delim as the field delimiter character instead of the tab character.

     -f list
	     The list specifies fields, separated in the input by the field delimiter character
	     (see the -d option).  Output fields are separated by a single occurrence of the
	     field delimiter character.

https://www.unix.com/man-page/freebsd/1/cut/

Code:
$ printf "foo\tbar boo \tbaz\tbae\n" | cut -f3
baz
$

--
Quote:
Originally Posted by chakrapani
i tested same log (copy paste) and -f option and it worked for me with space.

on FreeBSD with /usr/bin/cut
The log file sample is TAB separated, so that is why it works in this case..

Last edited by Scrutinizer; 01-21-2016 at 06:08 AM..
# 7  
Old 01-21-2016
Quote:
Originally Posted by Scrutinizer
The log file sample is TAB separated, so that is why it works in this case..
Got it ... Thanks
Login or Register to Ask a Question

Previous Thread | Next Thread

6 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

FINDING DUPLICATE PROJECT ( directory project )

I have a project tree like that. after running find command with the -no -empty option, i am able to have a list of non empty directory DO_MY_SEARCH="find . -type d -not -empty -print0" MY_EXCLUDE_DIR1=" -e NOT_IN_USE -e RTMAP -e NOT_USEFULL " echo " " > $MY_TEMP_RESULT_1 while... (2 Replies)
Discussion started by: jcdole
2 Replies

2. Post Here to Contact Site Administrators and Moderators

Apparently my post is Homework/Classwork?

I previously posted a question in the Shell Scripting forum a few minutes ago. I recieved a message telling me I had breached a rule, apparently because my post was a homework infraction. Well, im currently trying to figure out how to use Raspberry Pi's (as im a starter) I asked one of my... (1 Reply)
Discussion started by: Waggie14
1 Replies

3. News, Links, Events and Announcements

A new project was posted on The UNIX and Linux Forums project board.

A new project was posted on your project board. Project title: Bash Shell Tutoring Estimated Budget: $50/hr Start date: Immediately Required skills: Linux, Bash, Shell, UNIX I work as a datawarehouse designer and developer. Although I usually stick to the role of an analyst,... (0 Replies)
Discussion started by: Neo
0 Replies

4. Shell Programming and Scripting

need a one liner to grep a group info from /etc/group and use that result to search passwd file

/etc/group tiadm::345:mk789,po312,jo343,ju454,ko453,yx879,iy345,hn453 bin::2:root,daemon sys::3:root,bin,adm adm::4:root,daemon uucp::5:root /etc/passwd mk789:x:234:1::/export/home/dummy:/bin/sh po312:x:234:1::/export/home/dummy:/bin/sh ju454:x:234:1::/export/home/dummy:/bin/sh... (6 Replies)
Discussion started by: chidori
6 Replies

5. Solaris

what is the use of /etc/project file and project administration commands?

i have two doubts.. 1. what is the use /etc/project file. i renamed this file and when i tried to switch user or login with some user account the login was happening slowly. but when i renamed it to original name it was working fine... why so? 2. unix already has useradd and grouadd for... (4 Replies)
Discussion started by: chidori
4 Replies

6. Solaris

SSH doesn't pick up user's project from /etc/project

We have a system running ssh. When a user logs in, they do not get the project they are assigned to (they run under "system"). I verify the project using the command "ps -e -o user,pid,ppid,args,project". If you do a "su - username", the user does get the project they are assigned to (and all... (2 Replies)
Discussion started by: kurgan
2 Replies
Login or Register to Ask a Question