Sponsored Content
Top Forums Programming extract the username and the url and print it using Python 3 and BeautifulSoup 4 Post 303045287 by bob123 on Sunday 15th of March 2020 04:01:52 PM
Old 03-15-2020
extract the username and the url and print it using Python 3 and BeautifulSoup 4

using python 3 and BeautifulSoup 4
my code when run returns a big list like the txt below
but 100 times more i have only posted one as an example


Code:
urls = ['https://example.com']
for url in urls:
    response = requests.get(url)
    soup = BeautifulSoup(response.content, "html.parser")
    for text in soup.find_all(text= True):
           print(text)


code above returns
this

Code:
{"users":{"userId":222222222,"username":"some_name","hlsPreviewUrl":"https://example.com/322/example-origin-live/some_name-322-33babf1b-6030-479a-ac41-b90bd8313678_aac/playlist.m3u8 }

i want to extract the username and the url and print it

i have tried to add this piece of code but i get nothing

Code:
 if 'username' in ['text']:

what code am i missing
to print just the username and the url
with python 3 and BeautifulSoup 4

thanks

Last edited by bob123; 03-15-2020 at 05:07 PM..
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

SED extract url - please help a lamer

Hello everybody. I have lines that looks something like this: <done16=""118"" done18=""$ title=""thisisatitle"" href=""/JoeBanana" alt=""Joe""><done16=""118"" done18=""$ title=""thisisatitle"" href=""/GeraldGiraffe" alt=""Gerald""> What kind of SED command would I need to use to extract... (4 Replies)
Discussion started by: digi
4 Replies

2. Shell Programming and Scripting

How to extract url from html page?

for example, I have an html file, contain <a href="http://awebsite" id="awebsite" class="first">website</a>and sometime a line contains more then one link, for example <a href="http://awebsite" id="awebsite" class="first">website</a><a href="http://bwebsite" id="bwebsite"... (36 Replies)
Discussion started by: 14th
36 Replies

3. Shell Programming and Scripting

help, extract username from file

there is a text file like this text here text here text here text here USER=someusername here text here text here text here how can i extract the someusername from this file which has a path like /home/myfile.txt? i want to store it in a variable $u (3 Replies)
Discussion started by: vanessafan99
3 Replies

4. UNIX for Dummies Questions & Answers

Awk: print all URL addresses between iframe tags without repeating an already printed URL

Here is what I have so far: find . -name "*php*" -or -name "*htm*" | xargs grep -i iframe | awk -F'"' '/<iframe*/{gsub(/.\*iframe>/,"\"");print $2}' Here is an example content of a PHP or HTM(HTML) file: <iframe src="http://ADDRESS_1/?click=5BBB08\" width=1 height=1... (18 Replies)
Discussion started by: striker4o
18 Replies

5. UNIX for Dummies Questions & Answers

Launch a URL,validate username and password using wget or curl

Hi All, I want to launch "ex: http://gmail.com" from the cmd window and validate the credentials with username and password, is it possible? I have found something like this "wget --http-user=USER' --http-password=PASSWORD http://gmail.com" am new to this and unable to find a solution, i... (0 Replies)
Discussion started by: harsha85
0 Replies

6. UNIX for Dummies Questions & Answers

Mailx: extract username from a file

Hey guys, I am trying to send email to the usernames listed in a ascii file. I know about the .mailrc file. I want to know if I there is a way to extract the usernames from a file and email them. I tried something like this: mailx -s "subject" '~cat... (2 Replies)
Discussion started by: Nirav4
2 Replies

7. Shell Programming and Scripting

Newbie Python Url Scraper

I setup Zoneminder and have been playing around with setting up a couple of Wanscam PTZ ip cameras in which I have been running into road blocks with streaming and etc. I cant find much information on the camera and its webserver that sits on it and wanted to get a an absolute directory structure... (5 Replies)
Discussion started by: metallica1973
5 Replies

8. Shell Programming and Scripting

Connect to Remote DB2 ,using URL ,username,Password

Hi All, I am new to Shell scripting, I want to connect to DB2 database through URL,username and password Please help me out. I read so many posts in that there is no where it is mentioned about how to connect to a remote database. Thanks in advance :) (0 Replies)
Discussion started by: Rohit G
0 Replies

9. Shell Programming and Scripting

Python BeautifulSoup Re Finding Digits Within Tags

I am writing a little python script that needs to grab version numbers between "<td>4.2.2</td>" within the tbody of the page: blah blah blah blah blah Is it possible to use a one-liner to scrap only the digits between the tags: "<td>4.2.2</td>" so it spits out: 4.2.2 4.2.1 etc..... (2 Replies)
Discussion started by: metallica1973
2 Replies

10. Programming

[Python] BeautifulSoup tags > </a>

using BeautifulSoup how can i get the txt between all the > </a> example >The Student .mp4</a> thanks (10 Replies)
Discussion started by: bob123
10 Replies
IPCHECK(8)						  System Administration Utilities						IPCHECK(8)

NAME
ipcheck - dyndns.org client DESCRIPTION
Usage : ipcheck.py [options] Username Password Hostnames or ipcheck.py [options] --acctfile acct_info_file Options: -a address manually specify the address -r URL NAT router, use web IP detection -A text scan syslog for an IP after text (Unix only) -F filename guess the WAN IP from a firewall log -b backup mx option ON (default OFF) -c custom dns option (default dynamic) -d dir directory for data files (default current) -e script execute script after a successful update -f force update regardless of current state -g NAT router, let dyndns guess your IP (do not use this in a cronjob, try -r) -h print this help text --help print all available help text -i interface interface for local address (default ppp0) -j disable https -l log debugging text to ipcheck.log file --syslog log debugging text to syslog (Unix only) -m mxhost mx host (default NOCHG) -o set dyndns offline mode -p proxy bypass on port 8245 -q quiet mode (unless there is an error) -s static dns option (default dynamic) -t test run, do not send the update -v verbose mode -w wildcard mode ON (default OFF) --makedat create the ipcheck.dat file by dns lookup --devices print router options (Linksys, Netgear, etc) -n ip the router IP address on the internal lan For help with different options: python ipcheck.py -h For supported devices listing : python ipcheck.py --devices For long detailed help text : python ipcheck.py --help Make sure you can write to the current directory for data files and that you always run from the same directory. The first time you run the script, you will be asked to run with the --makedat option in addition to any other options. This will create the data files for the hostnames and options your specify on the command line. You should only do this once. Subsequent runs should be made without the --make- dat option. For help with different options: python ipcheck.py -h For supported devices listing : python ipcheck.py --devices For long detailed help text : python ipcheck.py --help Example 1: the external IP is on eth0 of the current machine python ipcheck.py -l -i eth0 username password hostnames Example 2: you are using the Linksys routing device python ipcheck.py -l -L linksyspassword username password hostnames Example 3: you want to use web based ip detection python ipcheck.py -l -r checkip.dyndns.org:8245 ... where ... is your dyndns username password and hostnames. Hostnames should be comma delimited if there are more than one. Debian GNU/Linux January 2012 IPCHECK(8)
All times are GMT -4. The time now is 10:58 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy