Sponsored Content
Top Forums Programming extract the username and the url and print it using Python 3 and BeautifulSoup 4 Post 303045287 by bob123 on Sunday 15th of March 2020 04:01:52 PM
Old 03-15-2020
extract the username and the url and print it using Python 3 and BeautifulSoup 4

using python 3 and BeautifulSoup 4
my code when run returns a big list like the txt below
but 100 times more i have only posted one as an example


Code:
urls = ['https://example.com']
for url in urls:
    response = requests.get(url)
    soup = BeautifulSoup(response.content, "html.parser")
    for text in soup.find_all(text= True):
           print(text)


code above returns
this

Code:
{"users":{"userId":222222222,"username":"some_name","hlsPreviewUrl":"https://example.com/322/example-origin-live/some_name-322-33babf1b-6030-479a-ac41-b90bd8313678_aac/playlist.m3u8 }

i want to extract the username and the url and print it

i have tried to add this piece of code but i get nothing

Code:
 if 'username' in ['text']:

what code am i missing
to print just the username and the url
with python 3 and BeautifulSoup 4

thanks

Last edited by bob123; 03-15-2020 at 05:07 PM..
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

SED extract url - please help a lamer

Hello everybody. I have lines that looks something like this: <done16=""118"" done18=""$ title=""thisisatitle"" href=""/JoeBanana" alt=""Joe""><done16=""118"" done18=""$ title=""thisisatitle"" href=""/GeraldGiraffe" alt=""Gerald""> What kind of SED command would I need to use to extract... (4 Replies)
Discussion started by: digi
4 Replies

2. Shell Programming and Scripting

How to extract url from html page?

for example, I have an html file, contain <a href="http://awebsite" id="awebsite" class="first">website</a>and sometime a line contains more then one link, for example <a href="http://awebsite" id="awebsite" class="first">website</a><a href="http://bwebsite" id="bwebsite"... (36 Replies)
Discussion started by: 14th
36 Replies

3. Shell Programming and Scripting

help, extract username from file

there is a text file like this text here text here text here text here USER=someusername here text here text here text here how can i extract the someusername from this file which has a path like /home/myfile.txt? i want to store it in a variable $u (3 Replies)
Discussion started by: vanessafan99
3 Replies

4. UNIX for Dummies Questions & Answers

Awk: print all URL addresses between iframe tags without repeating an already printed URL

Here is what I have so far: find . -name "*php*" -or -name "*htm*" | xargs grep -i iframe | awk -F'"' '/<iframe*/{gsub(/.\*iframe>/,"\"");print $2}' Here is an example content of a PHP or HTM(HTML) file: <iframe src="http://ADDRESS_1/?click=5BBB08\" width=1 height=1... (18 Replies)
Discussion started by: striker4o
18 Replies

5. UNIX for Dummies Questions & Answers

Launch a URL,validate username and password using wget or curl

Hi All, I want to launch "ex: http://gmail.com" from the cmd window and validate the credentials with username and password, is it possible? I have found something like this "wget --http-user=USER' --http-password=PASSWORD http://gmail.com" am new to this and unable to find a solution, i... (0 Replies)
Discussion started by: harsha85
0 Replies

6. UNIX for Dummies Questions & Answers

Mailx: extract username from a file

Hey guys, I am trying to send email to the usernames listed in a ascii file. I know about the .mailrc file. I want to know if I there is a way to extract the usernames from a file and email them. I tried something like this: mailx -s "subject" '~cat... (2 Replies)
Discussion started by: Nirav4
2 Replies

7. Shell Programming and Scripting

Newbie Python Url Scraper

I setup Zoneminder and have been playing around with setting up a couple of Wanscam PTZ ip cameras in which I have been running into road blocks with streaming and etc. I cant find much information on the camera and its webserver that sits on it and wanted to get a an absolute directory structure... (5 Replies)
Discussion started by: metallica1973
5 Replies

8. Shell Programming and Scripting

Connect to Remote DB2 ,using URL ,username,Password

Hi All, I am new to Shell scripting, I want to connect to DB2 database through URL,username and password Please help me out. I read so many posts in that there is no where it is mentioned about how to connect to a remote database. Thanks in advance :) (0 Replies)
Discussion started by: Rohit G
0 Replies

9. Shell Programming and Scripting

Python BeautifulSoup Re Finding Digits Within Tags

I am writing a little python script that needs to grab version numbers between "<td>4.2.2</td>" within the tbody of the page: blah blah blah blah blah Is it possible to use a one-liner to scrap only the digits between the tags: "<td>4.2.2</td>" so it spits out: 4.2.2 4.2.1 etc..... (2 Replies)
Discussion started by: metallica1973
2 Replies

10. Programming

[Python] BeautifulSoup tags > </a>

using BeautifulSoup how can i get the txt between all the > </a> example >The Student .mp4</a> thanks (10 Replies)
Discussion started by: bob123
10 Replies
Net::Google::AuthSub::Response(3pm)			User Contributed Perl Documentation		       Net::Google::AuthSub::Response(3pm)

NAME
Net::Google::AuthSub::Response - a response from a Net::Google::AuthSub request SYNOPSIS
my $response = $auth->login($user, $pass); if ($response->is_success) { print "Yay! "; } else { if ($response->error eq 'CaptchaRequired') { print "Captcha Image ".$response->captchaurl; } } METHODS
new "HTTP::Response" "base url" Create a new response. is_success Returns whether the response was a sucess or not. SUCCESS METHODS
Methods available if the response was a success. auth The authorisation token if the response is a success. sid Not used yet. lsid Not used yet. ERROR METHODS
Methods available if the response was an error. error The error code. Can be one of BadAuthentication The login request used a username or password that is not recognized. NotVerified The account email address has not been verified. The user will need to access their Google account directly to resolve the issue before logging in using a non-Google application. TermsNotAgreed The user has not agreed to terms. The user will need to access their Google account directly to resolve the issue before logging in using a non-Google application. CaptchaRequired A CAPTCHA is required. (A response with this error code will also contain an image URL and a CAPTCHA token.) Unknown The error is unknown or unspecified; the request contained invalid input or was malformed. AccountDeleted The user account has been deleted. AccountDisabled The user account has been disabled. ServiceDisabled The user's access to the specified service has been disabled. (The user account may still be valid.) ServiceUnavailable The service is not available; try again later. url The url of a page describing the error. captchatoken The token required to authenticate a captcha. captchaurl The full url of the captcha image. perl v5.10.0 2009-05-09 Net::Google::AuthSub::Response(3pm)
All times are GMT -4. The time now is 04:03 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy