Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

SED Search in YouTube file not working.

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting SED Search in YouTube file not working.
# 1  
Old 07-08-2010
SED Search in YouTube file not working.
Hi

I've succesfully downloaded a HTML file from youtube (for a specific video id)
I'm trying to use the SED command to search for double and single quotes and convert them to a line break (new line).
However i'm getting end of file errors when I run this script, can someone help?

I've included the whole script below, with comments for each line.

Code:
#!/bin/bash

echo ENTER THE YOUTUBE URL TO COPY:
read VIDEOID
#The above 2 lines read in the full URL link from the end user.
echo VIDEO ID TO  DOWNLOAD IS $VIDEOID #This line echos back to the user, what he just entered.
#YOURLINK=$FILENAME
file="youtube.html"
wget -q -O - $VIDEOID > $file #gets the file from youtube.
vidid=`cat $file | grep "fmt_url_map" | sed "s/'/\n"  | sed "s/"/\n" > youtube2.html
# this line is not working first grep is a search for the text within the file, second 2 sed's should convert single and double spaces to a new line.

Last edited by pludi; 07-08-2010 at 11:45 AM.. Reason: code tags, please...
# 2  
Old 07-08-2010
Hi,

to download youtube-videos from the commandline use cclive.

Your line:
Code:
vidid=`cat $file | grep "fmt_url_map" | sed "s/'/\n"  | sed "s/"/\n" > youtube2.html

can be simplified to:
Code:
vidid=$(sed -n "/fmt_url_map/{s/[\'\"]/\n/g;p}"  $file > youtube2.html)

HTH Chris
Last edited by Christoph Spohr; 07-08-2010 at 11:57 AM.. Reason: Mistake
# 3  
Old 07-08-2010
Code:
# sed -n '/fmt_url_map/p'  $file | sed -e 's/"/\n/g' -e "s/'/\n/g" > youtube2.html

# 4  
Old 07-08-2010
Hi

Thanks for your response, it worked, and broke the quotes up for me.
The below is what I ultimatley want to end up, i've tried to add the cache line, but it broke the search, can u have a look please?

Search for "fmt_url_map"
Break new lines at single-quotes, double-quotes and pipe (DONE)
grep for http addresses with the string "cache" (Added but not working)
Get the last such line. This is the http address of the actual video, which can be downloaded with wget.

Thanks
# 5  
Old 07-08-2010
if you can give me sample input file and desired output , i can help you Smilie
# 6  
Old 07-08-2010
Hi Ygemici

Basically I start by logging onto YouTube, find a video clip i want, then copy the URL of that video clip such as
YouTube - RC-Heli-Action: Tag der offenen Tür bei Vario
I then download this through a wget to my PC.
What I want to do is then search the downloaded HTML file.

Search for "fmt_url_map"
Break new lines at single-quotes, double-quotes and pipe.
grep for http addresses with the string "cache"
Get the last such line. This is the http address of the actual video, which can be downloaded with wget.
End Result should be a downloaded FLV file of the choosen video clip from YouTube.

C.
# 7  
Old 07-08-2010
hi

i download youtube video page url as html but i cant find "flv" in source code..
flv informations should be located server side and not client side..
but i write something Smilie but i cant find flv address Smilie

Code:
# sed -n '/fmt_url_map/p'  YouTube\ -\ RC-Heli-Action\ Tag\ der\ offenen\ Tür\ bei\ Vario.htm | sed -e 's/"/\n/g' -e "s/'/\n/g" -e 's:|:/n:g' | grep  "cache" | sed -n -e '$p' |sed 's/.*\(http.*\)http.*/\1/'
 
http://v23.lscache1.c.youtube.com/videoplayback?ip=0.0.0.0&sparams=id%2Cexpire%2Cip%2Cipbits%2Citag%2Calgorithm%2Cburst%2Cfactor%2Coc%3AU0dWTVZPVl9FSkNNNl9OSVpB&algorithm=throttle-factor&itag=34&ipbits=0&burst=40&sver=3&expire=1278648000&key=yt1&signature=2C7FF8A961208A9536EDC5F7BDE3B90937A51BEE.80DEA95F5B5DF092C5A3CDA92BA599CDC73814C8&factor=1.25&id=fba0731f5dc7af9b/n/ntc.v23.cache1.c.youtube.com,5/n

Code:
# sed -n '/fmt_url_map/p'  YouTube\ -\ RC-Heli-Action\ Tag\ der\ offenen\ Tür\ bei\ Vario.htm | sed -e 's/"/\n/g' -e "s/'/\n/g" -e 's:|:/n:g' | grep  "video_id" | sed -e 's/.*\(http.*\)http.*/\1/' -e '$d'
 
http%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3Dv1RmD8lnDUc&rv.7.url=
http%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3Dv1RmD8lnDUc&rv.7.url=
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. What is on Your Mind?

YouTube: Search Engine Optimization | How To Fix Soft 404 Errors and A.I. Tales from Google Search

Getting a bit more comfortable making quick YT videos in 4K, here is: Search Engine Optimization | How To Fix Soft 404 Errors and A.I. Tales from Google Search Console https://youtu.be/I6b9T2qcqFo (0 Replies)
Discussion started by: Neo
0 Replies

2. Shell Programming and Scripting

sed command not working for me to change text in a file

UNIX gurus I need your help with the following (The server is an AIX box). I have a text file with the following information: ******************************************************** SOME LINES case :WORD1 SOME LINES :WORD2 SOME LINES :WORD3 SOME LINES esac SOME LINES... (7 Replies)
Discussion started by: curiousmal
7 Replies

3. Shell Programming and Scripting

sed to search and replace - iterating on a file

Hi, I am new to sed scriping but do have some basic scripting skills. I have a properties file props.conf with name/value pairs written in it. Sample is Name1=test Name2=notest Then I have a template file (can be xml or simple text) which will have 'keys' embedded in it. Keys can... (2 Replies)
Discussion started by: tigerinopen
2 Replies

4. Shell Programming and Scripting

Search and replace is not working by sed or awk

Hi , I have one file and in this file i have one like TEST1 KEY0=AAC040R1;AAC041R1ISE;AAC041R2ISE;AAC370R1;ADR0500;ADR0600;AME245R1;AME245R2;BAP0135;BAP0300;PPINVDTD*;PPJERPTD*;PPJERPT*;PRBSUMM*;: i want to replace this line with the following line TEST1... (4 Replies)
Discussion started by: ashissau
4 Replies

5. Shell Programming and Scripting

sed and awk not working on a large record file

Hi All, I have a very large single record file. abc;date||bcd;efg|......... pqr;stu||record_count;date when i do wc -l on this file it gives me "0" records, coz of missing line feed. my problem is there is an extra pipe that is coming at the end of this record like... (6 Replies)
Discussion started by: Gurkamal83
6 Replies

6. Shell Programming and Scripting

Search pattern within file using sed..

Hi, Could someone help me in figuring out a way using which i can search for a specific pattern. Eg. JUSTDOIT..I have to print just the word "DO" from "JUSTDOIT" If the same word JUSTDOIT is print n number of times (6 Replies)
Discussion started by: sankasu
6 Replies

7. Shell Programming and Scripting

Sed emptying file when i use for search replace operation

Hi Friends I am new to sed programming , i found that the below code can search for the $ToSearch and Replace it with $ToReplace ( $ToSearch and $ToReplace are my variables in my script ) sed "s/$ToSearch/$ToReplace/" $file > $output mv $output $file In testing the script i found that... (3 Replies)
Discussion started by: rakeshkumar
3 Replies

8. Shell Programming and Scripting

sed help - search/copy from one file and search/paste to another

I am a newbie and would like some help with the following - Trying to search fileA for a string similar to - AS11000022010 30.4 31.7 43.7 53.8 60.5 71.1 75.2 74.7 66.9 56.6 42.7 32.5 53.3 I then want to replace that string with a string from fileB - ... (5 Replies)
Discussion started by: ncwxpanther
5 Replies

9. Shell Programming and Scripting

Awk/Sed: Search Pattern from file and Print

Hi, I want to search for patterns (from a file) in a file and print the line matching the patterns and the line before it. I have to search for 100s of patterns from a file. Any help with AWK or Sed. Thanks! (2 Replies)
Discussion started by: saint2006
2 Replies

10. UNIX for Dummies Questions & Answers

How to search and replace a particular line in file with sed command

Hello, I have a file and in that, I want to search for a aprticular word and then replace another word in the same line with something else. Example: In file abc.txt, there is a line <host oa_var="s_hostname">test</host> I want to search with s_hostname text and then replace test with... (2 Replies)
Discussion started by: sshah1001
2 Replies
Login or Register to Ask a Question