Working with grep and Bash


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Working with grep and Bash
# 1  
Old 02-16-2013
Working with grep and Bash

Hi, I am currently working on a Bash shell script that
- Downloads a webpage, in this case youtube.com
- Extracts Number of views, Extracts Title of video, Extracts User who made it, and lastly Duration. Then I have to Out put this into columns.

To me this sounds like crazyness. I'm very new to bash and even using grep/cut commands So far this is all I got. I decided to pass all my commands to variables to try and make a table then output but its very messy. What I am trying to do is

make tables for

Views | Title | User | Time

0000 blah me 0:10

etc but I have no idea how to cut out information from the html and organize it.
Code:
#! /bin/bash

wget -O Link.html youtube.com

echo [Views]            [User]          

x=$(grep views Link.html | grep -v 'div' | cut -d ' ' -f5 | sort -n | tr -d ',')
y=$(grep title Link.html | grep -v 'div' | grep -v span |cut -d '>' -f1 | grep -v class | grep -v '<')


IFS="'echo'"
for line in $x
        do
                echo $line
        done

#echo $y
~                                                                                         
~                                                                                         
~                                                                                         
~

Its probably really messy but I'm lost on how to extract the correct titles which are usually in the " " ex. title="Harlem Shake (Matt and Kim Edition)"

Also I have no idea how to organize or sort it so it matches the correct amount of views. I'm not looking for the answer I'm looking for the solution. So I can do the same for the other parts of this sh myself.

Once again I am very new to this stuff lol.

The best can get are the views to show up organized. The titles are all over the place.

Last edited by Njzangel; 02-16-2013 at 10:10 PM..
# 2  
Old 02-16-2013
I don't think using BASH for extracting data from downloaded file is a going to be an easy task.

I suggest to use AWK instead which is designed specifically for data extraction and reporting.
# 3  
Old 02-17-2013
On top of bipinajith's statement, it would be helpful if you posted (edited) samples of what you've got, e.g. a few meaningful lines of your downloads - DON'T post the videos! It's difficult to guess the line layout from the greps you've posted above.
# 4  
Old 02-18-2013
Hi sorry guys for the late response I managed to figure out how to extract all my information and send them to variables. It maybe sloppy but right now it doesn't matter.

What I was looking for is lets say I have some lines like this
Code:
<div class="feed-item-content-wrapper clearfix context-data-item" data-context-item-title="Mike Chang's Superbowl Workout - Part II" data-context-item-user="sixpackshortcuts" data-context-item-views="114,925 views" data-context-item-id="V1l1TGUUaj0" data-context-item-type="video" data-context-item-time="8:44" data-context-item-actionuser="sixpackshortcuts">

I would need to Extract the Views, duration(time) username, and title. Which I did using these simple codes and passed them to a variable
Code:
#! /bin/bash

#wget -O web2.html youtube.com

echo [Views]            [time/Duration] 
echo ----------------------------------
echo    

views=$(grep views web2.html | grep -v 'div' | cut -d ' ' -f5 | tr -d ',')

users=$(grep item-user web2.html | cut -d '"' -f6)

duration=$(grep item-time web2.html | grep -o '"[^"]*"' | grep : | grep -vi '[a-z]' | tr -d '"')

title=$(grep item-title web2.html | cut -d '"' -f4)

So lets say I echo "$views" it will show something like

11
2244
2423532
2342
2324

echo $title

I'm hungry
Pewdiepie rocks

My issue now is as you can see in the code its incomplete, I have to construct a table with columns and display all my data in columns. I have no idea what soever where to start I need it to show up like

[Views]---------[Title]--------------------[user]
12 ------------Pewdiepie--------------Pewdiepie
1212 --------IDONTCARE!--------Somedeadguy
1212 --------BARRELS!!-----------idonthavewifi
1 --------------Poop]--------------------etc

(minus the '-' it was used for an example to look more clear)
and so on with the time duration. How can i go about outputting the contents of the variables in such a way?
# 5  
Old 02-19-2013
You might want to consider using awk. As a starting point, working for your sample, try and then adapt to your needs:
Code:
awk     '/item-(user|views|time|title)/ {getline Ar[++i]}
         END {for (n in Ar) printf "%15s ", Ar[n]
              printf "\n"}
        ' RS="[=\"]*" file
Mike Chang's Superbowl Workout - Part II sixpackshortcuts   114,925 views            8:44

Not every awk implementation will accept above RS construct, so you might need to experiment a bit.
# 6  
Old 02-20-2013
In my case I cant use awk yet, if I get a quiz or exam in the near few days/week it will be using quick and dirty methods of grep, cut, expr etc.

Is it even possible to read in my $views, $titles, $users etc into an array that will list them going down? or will i have to find another method in doing this?
# 7  
Old 02-20-2013
Oh, this is homework? Then pls follow the forum rules; #6.
And, yes, it is possible, although not effeicient.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Programming

Best way to get a bash script working in C

Ahoy friends. Currently i got a bash script running to manage my minecraft servers. All of them are stored in /home/minecraft_servers directory. Using my script im able to start a server (e.g. ./minecraft start ftb_continuum) because server name and server name are the same.(e.g.... (2 Replies)
Discussion started by: Knogle
2 Replies

2. Shell Programming and Scripting

Grep not working on mac

Hi all, I got a new mac and can't get grep, awk etc to work. I tried the following command: grep DICER test.txt output: AGOER text.txt looks like this: DICER DICER AGOWhat is wrong? Please use code tags (23 Replies)
Discussion started by: Palgrave
23 Replies

3. UNIX for Dummies Questions & Answers

Bash - CLI - grep - Passing result to grep through pipe

Hello. I want to get all modules which are loaded and which name are exactly 2 characters long and not more than 2 characters and begin with "nv" lsmod | (e)grep '^nv???????????? I want to get all modules which are loaded and which name begin with "nv" and are 2 to 7 characters long ... (1 Reply)
Discussion started by: jcdole
1 Replies

4. UNIX for Dummies Questions & Answers

grep for word not working

Hi All..I need a help i am trying to find a word using below script whereas the word exists in my file nitin.txt as a directory but still i am getting "word not found" output..Your suggestions welcomed.: #to check for existence of nitin #!/bin/bash cd /apps/uat1/deploy/app ls -lrt >... (4 Replies)
Discussion started by: nattynitin
4 Replies

5. UNIX for Dummies Questions & Answers

grep -f not working

Hello, I'm going crazy about this. I'm using grep to filter some values as in pas -ef | grep asterisk. When I use the same with -f somefile something weird happens, if somefile is created with vi it'll work, if somefile is created with vi but values are pasted from an Excell file it will not work.... (2 Replies)
Discussion started by: seveman
2 Replies

6. Shell Programming and Scripting

Working with bash and date

Hello all, I'm trying to substract 1 minute from the current date and take the hour and minute (for filename purpose). 1) If I want hour an minute from current time I can use: timetmp=$(date +"%H:%M") 2) To substract 1 minute from current time I can use: timetmp=$(date --date "$dte -1... (8 Replies)
Discussion started by: Lord Spectre
8 Replies

7. Shell Programming and Scripting

grep not working ????

Hi, I've prob in doing grep. I want to grep line staring with number 531250 in the 1st column from a file (example in picture attached below) using command grep -w "531250" file my ideal result should be 531250 1 21 42.1 100 1e-05 ... (8 Replies)
Discussion started by: masterpiece
8 Replies

8. HP-UX

Why Bash is not working in HP-UX ?

Why Bash is not working in HP-UX ? What is similiar exe which is in HP_UX as Bash? (9 Replies)
Discussion started by: girija
9 Replies

9. UNIX for Dummies Questions & Answers

grep not working

This condition is not able to grep , can any one tell what's wrong with this part. I am able to see from unix command but not with host script. echo "Checking for Loader Status " >> $REPFILE if test $? = 0 then echo "Successful termination of SQL*Loader "$LOADER1 >>... (5 Replies)
Discussion started by: u263066
5 Replies

10. Solaris

grep -r isn't working

Hi, I was trying to use this particular option of grep grep -r 'Search_pattern' * This command should ideally search all the occurrences of Search_pattern recursively within a directory & print it on shell prompt. But this command is not doing what is expected. It just displays nothin! ... (8 Replies)
Discussion started by: harishmitty
8 Replies
Login or Register to Ask a Question