Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: extract the username and the url and print it using Python 3 and BeautifulSoup 4
Top Forums Programming extract the username and the url and print it using Python 3 and BeautifulSoup 4 Post 303045287 by bob123 on Sunday 15th of March 2020 04:01:52 PM
Old 03-15-2020
extract the username and the url and print it using Python 3 and BeautifulSoup 4
using python 3 and BeautifulSoup 4
my code when run returns a big list like the txt below
but 100 times more i have only posted one as an example


Code:
urls = ['https://example.com']
for url in urls:
    response = requests.get(url)
    soup = BeautifulSoup(response.content, "html.parser")
    for text in soup.find_all(text= True):
           print(text)


code above returns
this

Code:
{"users":{"userId":222222222,"username":"some_name","hlsPreviewUrl":"https://example.com/322/example-origin-live/some_name-322-33babf1b-6030-479a-ac41-b90bd8313678_aac/playlist.m3u8 }

i want to extract the username and the url and print it

i have tried to add this piece of code but i get nothing

Code:
 if 'username' in ['text']:

what code am i missing
to print just the username and the url
with python 3 and BeautifulSoup 4

thanks
Last edited by bob123; 03-15-2020 at 05:07 PM..
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

SED extract url - please help a lamer

Hello everybody. I have lines that looks something like this: <done16=""118"" done18=""$ title=""thisisatitle"" href=""/JoeBanana" alt=""Joe""><done16=""118"" done18=""$ title=""thisisatitle"" href=""/GeraldGiraffe" alt=""Gerald""> What kind of SED command would I need to use to extract... (4 Replies)
Discussion started by: digi
4 Replies

2. Shell Programming and Scripting

How to extract url from html page?

for example, I have an html file, contain <a href="http://awebsite" id="awebsite" class="first">website</a>and sometime a line contains more then one link, for example <a href="http://awebsite" id="awebsite" class="first">website</a><a href="http://bwebsite" id="bwebsite"... (36 Replies)
Discussion started by: 14th
36 Replies

3. Shell Programming and Scripting

help, extract username from file

there is a text file like this text here text here text here text here USER=someusername here text here text here text here how can i extract the someusername from this file which has a path like /home/myfile.txt? i want to store it in a variable $u (3 Replies)
Discussion started by: vanessafan99
3 Replies

4. UNIX for Dummies Questions & Answers

Awk: print all URL addresses between iframe tags without repeating an already printed URL

Here is what I have so far: find . -name "*php*" -or -name "*htm*" | xargs grep -i iframe | awk -F'"' '/<iframe*/{gsub(/.\*iframe>/,"\"");print $2}' Here is an example content of a PHP or HTM(HTML) file: <iframe src="http://ADDRESS_1/?click=5BBB08\" width=1 height=1... (18 Replies)
Discussion started by: striker4o
18 Replies

5. UNIX for Dummies Questions & Answers

Launch a URL,validate username and password using wget or curl

Hi All, I want to launch "ex: http://gmail.com" from the cmd window and validate the credentials with username and password, is it possible? I have found something like this "wget --http-user=USER' --http-password=PASSWORD http://gmail.com" am new to this and unable to find a solution, i... (0 Replies)
Discussion started by: harsha85
0 Replies

6. UNIX for Dummies Questions & Answers

Mailx: extract username from a file

Hey guys, I am trying to send email to the usernames listed in a ascii file. I know about the .mailrc file. I want to know if I there is a way to extract the usernames from a file and email them. I tried something like this: mailx -s "subject" '~cat... (2 Replies)
Discussion started by: Nirav4
2 Replies

7. Shell Programming and Scripting

Newbie Python Url Scraper

I setup Zoneminder and have been playing around with setting up a couple of Wanscam PTZ ip cameras in which I have been running into road blocks with streaming and etc. I cant find much information on the camera and its webserver that sits on it and wanted to get a an absolute directory structure... (5 Replies)
Discussion started by: metallica1973
5 Replies

8. Shell Programming and Scripting

Connect to Remote DB2 ,using URL ,username,Password

Hi All, I am new to Shell scripting, I want to connect to DB2 database through URL,username and password Please help me out. I read so many posts in that there is no where it is mentioned about how to connect to a remote database. Thanks in advance :) (0 Replies)
Discussion started by: Rohit G
0 Replies

9. Shell Programming and Scripting

Python BeautifulSoup Re Finding Digits Within Tags

I am writing a little python script that needs to grab version numbers between "<td>4.2.2</td>" within the tbody of the page: blah blah blah blah blah Is it possible to use a one-liner to scrap only the digits between the tags: "<td>4.2.2</td>" so it spits out: 4.2.2 4.2.1 etc..... (2 Replies)
Discussion started by: metallica1973
2 Replies

10. Programming

[Python] BeautifulSoup tags > </a>

using BeautifulSoup how can i get the txt between all the > </a> example >The Student .mp4</a> thanks (10 Replies)
Discussion started by: bob123
10 Replies

LEARN ABOUT DEBIAN

update-python-modules

UPDATE-PYTHON-MODULES(8)				      System Manager's Manual					  UPDATE-PYTHON-MODULES(8)

NAME

       update-python-modules - byte-compile python modules

SYNOPSIS

       update-python-modules [options] [package.public [...]]

DESCRIPTION

       The update-python-modules command is part of the python-support bundle.

       update-python-modules  is  responsible for the byte-compilation of python modules. When used in byte-compilation mode, it will byte-compile
       modules references in the files given on the command line. When used in	installation  mode,  it  will  byte-compile  them  once  for  each
       installed python version, and will make them available for all of them.

       At  each  invocation,  with  or without arguments, update-python-modules will check for installed python versions, and will build or remove
       modules according to new or removed versions since the last time it was run.

ARGUMENTS

       Arguments must be files or directories located in the /usr/share/python-support directory. Depending on their extension, they  are  treated
       differently.

       package.public
	      A file listing public modules to install for each version. These modules must lie in /usr/share/pyshared or /usr/lib/pyshared.

       package.private
	      A file listing private modules to byte-compile for one Python version.

       package/
	      A legacy directory hierarchy of public modules. The C extensions and portions that change with the Python version are to be found in
	      /usr/lib/python-support/pythonX.Y/package/.

       package.dirs
	      A legacy file listing directories where to find private modules.

OPTIONS

       -h, --help
	      Show summary of options.

       -v, --verbose
	      Detail all actions while they are performed.

       -b, --bytecompile
	      Only accept to work on private modules. When given this option, update-python-modules will fail if passed references to public  mod-
	      ules.

       -i, --install
	      Only  accept  to work on public modules. When given this option, update-python-modules will fail if passed references to public mod-
	      ules.

       -c, --clean
	      Clean modules instead of installing them.

       -a, --rebuild-all
	      Rebuild all private modules. This is necessary when the default python version was changed, for example.

       -f, --force-rebuild-all
	      Clean all modules (public and private), and rebuild them.

FILES

       /usr/lib/pymodules/pythonX.Y/
	      These directories contain the byte-compiled modules for version pythonX.Y.

SEE ALSO

       /usr/share/doc/python-support/README.gz

AUTHOR

       Josselin Mouette <joss@debian.org>.

								    18 Feb 2009 					  UPDATE-PYTHON-MODULES(8)
All times are GMT -4. The time now is 02:52 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy