Login or Register to Ask a Question and Join Our Community


Programming

Python Script Issues

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Programming Python Script Issues
# 1  
Old 05-09-2013
Python Script Issues
Gents,

This is a follow up to my previous post:

https://www.unix.com/shell-programmin...ion-regex.html

Here the latest function that I am having issues with:

Code:
def find_all_flvs(url):
    soup = BeautifulSoup(urllib2.urlopen(url))
    flvs = []
    for link in soup.findAll(onclick=re.compile("doShowCHys=1*")):
        link = str(link)
        #startpos = link.find("lpk4=") + 5
        #endpos   = link.find("&amp")
        vidnum   = re.findall("(?<==)\d{5,7}", link, re.U)
        vidurl   = "http://www.blah.com/home/GetPlayerXML.aspx?lpk4=%s" % vidnum[0]

    for hashval_url in BeautifulSoup(urllib2.urlopen(vidurl)).findAll("flv"):
            flvs.append(hashval_url.text)

    return flvs

in that I cant understand why when manually looking at my truple
Code:
flvs = []

it should have all the urls that were scraped such as:
Code:
[u'http://blah.blah.com/secure/courses/94343/f4v_H264/94343_00_01_SC03_welcome.f4v?hashval=1368124621_35861c278ff5c700797b0bf8fc04c554', u'http://blah.blah.com/secure/courses/94343/f4v_H264/94343_00_01_SC03_welcome.f4v?hashval=1368124623_67da0c443fe1b220332f1bd9e50e5d6c'

it only adding a single url to the truple:
Code:
[u'http://blah.blah.com/secure/courses/94343/f4v_H264/94343_00_01_SC03_welcome.f4v?hashval=1368124621_35861c278ff5c700797b0bf8fc04c554'

I have a deep feeling it has to do with my list reference:
Code:
vidurl   = "http://www.blah.com/home/GetPlayerXML.aspx?lpk4=%s" % vidnum[0]

Someone please make me feel stupid.
# 2  
Old 05-10-2013
Hi,

Try printing out vidum, then proceed from there.

Code:
vidnum   = re.findall("(?<==)\d{5,7}", link, re.U) 
print vidnum

# 3  
Old 05-10-2013
thank you for the reply. It actually turned out to be my nested "for" loop that appeared to be the issue. So from
Code:
def find_all_flvs(url):
    soup = BeautifulSoup(urllib2.urlopen(url))
    flvs = []
    for link in soup.findAll(onclick=re.compile("doShowCHys=1*")):
        link = str(link)
        #startpos = link.find("lpk4=") + 5
        #endpos   = link.find("&amp")
        vidnum   = re.findall("(?<==)\d{5,7}", link, re.U)
        vidurl   = "http://www.blah.com/home/GetPlayerXML.aspx?lpk4=%s" % vidnum[0]

    for hashval_url in BeautifulSoup(urllib2.urlopen(vidurl)).findAll("flv"):
            flvs.append(hashval_url.text)

    return flvs

to

Code:
def find_all_flvs(url):
    soup = BeautifulSoup(urllib2.urlopen(url))
    flvs = []
    for link in soup.findAll(onclick=re.compile("doShowCHys=1*")):
        link = str(link)
        #startpos = link.find("lpk4=") + 5
        #endpos   = link.find("&amp")
        vidnum   = re.findall("(?<==)\d{5,7}", link, re.U)
        vidurl   = "http://www.blah.com/home/GetPlayerXML.aspx?lpk4=%s" % vidnum[0]

         for hashval_url in BeautifulSoup(urllib2.urlopen(vidurl)).findAll("flv"):
            flvs.append(hashval_url.text)

    return flvs

Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. Programming

Create a C source and compile inside Python 1.4.0 to 3.7.0 in Python for ALL? platforms...

Hi all... As you know I like making code backwards compatible for as many platforms as possible. This Python script was in fact dedicated for the AMIGA A1200 using Pythons 1.4.0, 1.5.2, 1.6.0, 2.0.1, and 2.4.6 as that is all we have for varying levels of upgrades from a HDD and 4MB FastRam... (1 Reply)
Discussion started by: wisecracker
1 Replies

2. Windows & DOS: Issues & Discussions

How to execute python script on remote with python way..?

Hi all, I am trying to run below python code for connecting remote windows machine from unix to run an python file exist on that remote windows machine.. Below is the code I am trying: #!/usr/bin/env python import wmi c = wmi.WMI("xxxxx", user="xxxx", password="xxxxxxx")... (1 Reply)
Discussion started by: onenessboy
1 Replies

3. Shell Programming and Scripting

Capture run time of python script executed inside shell script

I have bash shell script which is internally calling python script.I would like to know how long python is taking to execute.I am not allowed to do changes in python script.Please note i need to know execution time of python script which is getting executed inside shell .I need to store execution... (2 Replies)
Discussion started by: Adfire
2 Replies

4. Shell Programming and Scripting

Python script called by a shell script

experts, i wrote a python script to do a certain job, i tried it and it is working fine, i want this script to be executed automatically after a ksh script, the problem is when i execute the ksh script my python script runes perfectly after the ksh script as I have include it at the end of the ksh... (1 Reply)
Discussion started by: q8devilish
1 Replies

5. Shell Programming and Scripting

Need a Python script

I work on various messages received from server and want to write a python script that can sort messages with unique flag values and give me the output in a text file. I get these messages in the form of .zcap file from server, in order to get messages from those files; I use an internal tool:... (0 Replies)
Discussion started by: Vijeta Laad
0 Replies

6. Shell Programming and Scripting

Script issues

#!/bin/bash glist=`cat /etc/group | cut -d ":" -f1,4` ulist=`cat /etc/passwd | cut -d ":" -f1,6` for i in $glist do echo "$glist" done for i in $ulist do echo "$ulist" done chkgrp=`cat /etc/group | cut -d ":" -f1` for a in chkgrp do (4 Replies)
Discussion started by: mduduzi
4 Replies

7. Shell Programming and Scripting

Passing variable from shell script to python script

I have a shell script main.sh which inturn call the python script ofdm.py, I want to pass two variables from shell script to python script for its execution. How do i achieve this ????? Eg: main.sh a=3 b=3; c= a+b exec python ofdm.py ofdm.py d=c+a Thanks in Anticipation (4 Replies)
Discussion started by: shashi792
4 Replies

8. Shell Programming and Scripting

Hi Python and shell script,the script hangs

Hi I need to run a shell script from a TCL script,the shell script in trun will run a python script 1.Tcl script set filename "./GopiRun.sh" 2.GopiRun.sh python ./psi.py $MYSB/test_scripts/delivery/gpy1.py 3.I have my gpy1.py script. Here the problem i am facing is on running... (0 Replies)
Discussion started by: nathgopi214
0 Replies

9. Shell Programming and Scripting

How to run python script from a Tcl script

Hi I have a python script,i need to run this script from a tcl script.Can anyone let me know how to do this (1 Reply)
Discussion started by: nathgopi214
1 Replies
Login or Register to Ask a Question