Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: extract the username and the url and print it using Python 3 and BeautifulSoup 4
Top Forums Programming extract the username and the url and print it using Python 3 and BeautifulSoup 4 Post 303045287 by bob123 on Sunday 15th of March 2020 04:01:52 PM
Old 03-15-2020
extract the username and the url and print it using Python 3 and BeautifulSoup 4
using python 3 and BeautifulSoup 4
my code when run returns a big list like the txt below
but 100 times more i have only posted one as an example


Code:
urls = ['https://example.com']
for url in urls:
    response = requests.get(url)
    soup = BeautifulSoup(response.content, "html.parser")
    for text in soup.find_all(text= True):
           print(text)


code above returns
this

Code:
{"users":{"userId":222222222,"username":"some_name","hlsPreviewUrl":"https://example.com/322/example-origin-live/some_name-322-33babf1b-6030-479a-ac41-b90bd8313678_aac/playlist.m3u8 }

i want to extract the username and the url and print it

i have tried to add this piece of code but i get nothing

Code:
 if 'username' in ['text']:

what code am i missing
to print just the username and the url
with python 3 and BeautifulSoup 4

thanks
Last edited by bob123; 03-15-2020 at 05:07 PM..
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

SED extract url - please help a lamer

Hello everybody. I have lines that looks something like this: <done16=""118"" done18=""$ title=""thisisatitle"" href=""/JoeBanana" alt=""Joe""><done16=""118"" done18=""$ title=""thisisatitle"" href=""/GeraldGiraffe" alt=""Gerald""> What kind of SED command would I need to use to extract... (4 Replies)
Discussion started by: digi
4 Replies

2. Shell Programming and Scripting

How to extract url from html page?

for example, I have an html file, contain <a href="http://awebsite" id="awebsite" class="first">website</a>and sometime a line contains more then one link, for example <a href="http://awebsite" id="awebsite" class="first">website</a><a href="http://bwebsite" id="bwebsite"... (36 Replies)
Discussion started by: 14th
36 Replies

3. Shell Programming and Scripting

help, extract username from file

there is a text file like this text here text here text here text here USER=someusername here text here text here text here how can i extract the someusername from this file which has a path like /home/myfile.txt? i want to store it in a variable $u (3 Replies)
Discussion started by: vanessafan99
3 Replies

4. UNIX for Dummies Questions & Answers

Awk: print all URL addresses between iframe tags without repeating an already printed URL

Here is what I have so far: find . -name "*php*" -or -name "*htm*" | xargs grep -i iframe | awk -F'"' '/<iframe*/{gsub(/.\*iframe>/,"\"");print $2}' Here is an example content of a PHP or HTM(HTML) file: <iframe src="http://ADDRESS_1/?click=5BBB08\" width=1 height=1... (18 Replies)
Discussion started by: striker4o
18 Replies

5. UNIX for Dummies Questions & Answers

Launch a URL,validate username and password using wget or curl

Hi All, I want to launch "ex: http://gmail.com" from the cmd window and validate the credentials with username and password, is it possible? I have found something like this "wget --http-user=USER' --http-password=PASSWORD http://gmail.com" am new to this and unable to find a solution, i... (0 Replies)
Discussion started by: harsha85
0 Replies

6. UNIX for Dummies Questions & Answers

Mailx: extract username from a file

Hey guys, I am trying to send email to the usernames listed in a ascii file. I know about the .mailrc file. I want to know if I there is a way to extract the usernames from a file and email them. I tried something like this: mailx -s "subject" '~cat... (2 Replies)
Discussion started by: Nirav4
2 Replies

7. Shell Programming and Scripting

Newbie Python Url Scraper

I setup Zoneminder and have been playing around with setting up a couple of Wanscam PTZ ip cameras in which I have been running into road blocks with streaming and etc. I cant find much information on the camera and its webserver that sits on it and wanted to get a an absolute directory structure... (5 Replies)
Discussion started by: metallica1973
5 Replies

8. Shell Programming and Scripting

Connect to Remote DB2 ,using URL ,username,Password

Hi All, I am new to Shell scripting, I want to connect to DB2 database through URL,username and password Please help me out. I read so many posts in that there is no where it is mentioned about how to connect to a remote database. Thanks in advance :) (0 Replies)
Discussion started by: Rohit G
0 Replies

9. Shell Programming and Scripting

Python BeautifulSoup Re Finding Digits Within Tags

I am writing a little python script that needs to grab version numbers between "<td>4.2.2</td>" within the tbody of the page: blah blah blah blah blah Is it possible to use a one-liner to scrap only the digits between the tags: "<td>4.2.2</td>" so it spits out: 4.2.2 4.2.1 etc..... (2 Replies)
Discussion started by: metallica1973
2 Replies

10. Programming

[Python] BeautifulSoup tags > </a>

using BeautifulSoup how can i get the txt between all the > </a> example >The Student .mp4</a> thanks (10 Replies)
Discussion started by: bob123
10 Replies

LEARN ABOUT OSX

locale::codes::langfam

Locale::Codes::LangFam(3pm)				 Perl Programmers Reference Guide			       Locale::Codes::LangFam(3pm)

NAME

       Locale::Codes::LangFam - standard codes for language extension identification

SYNOPSIS

	  use Locale::Codes::LangFam;

	  $lext = code2langfam('apa');		       # $lext gets 'Apache languages'
	  $code = langfam2code('Apache languages');    # $code gets 'apa'

	  @codes   = all_langfam_codes();
	  @names   = all_langfam_names();

DESCRIPTION

       The "Locale::Codes::LangFam" module provides access to standard codes used for identifying language families, such as those as defined in
       ISO 639-5.

       Most of the routines take an optional additional argument which specifies the code set to use. If not specified, the default ISO 639-5
       language family codes will be used.

SUPPORTED CODE SETS

       There are several different code sets you can use for identifying language families. A code set may be specified using either a name, or a
       constant that is automatically exported by this module.

       For example, the two are equivalent:

	  $lext = code2langfam('apa','alpha');
	  $lext = code2langfam('apa',LOCALE_LANGFAM_ALPHA);

       The codesets currently supported are:

       alpha
	   This is the set of three-letter (lowercase) codes from ISO 639-5 such as 'apa' for Apache languages.

	   This is the default code set.

ROUTINES

       code2langfam ( CODE [,CODESET] )
       langfam2code ( NAME [,CODESET] )
       langfam_code2code ( CODE ,CODESET ,CODESET2 )
       all_langfam_codes ( [CODESET] )
       all_langfam_names ( [CODESET] )
       Locale::Codes::LangFam::rename_langfam  ( CODE ,NEW_NAME [,CODESET] )
       Locale::Codes::LangFam::add_langfam  ( CODE ,NAME [,CODESET] )
       Locale::Codes::LangFam::delete_langfam  ( CODE [,CODESET] )
       Locale::Codes::LangFam::add_langfam_alias  ( NAME ,NEW_NAME )
       Locale::Codes::LangFam::delete_langfam_alias  ( NAME )
       Locale::Codes::LangFam::rename_langfam_code  ( CODE ,NEW_CODE [,CODESET] )
       Locale::Codes::LangFam::add_langfam_code_alias  ( CODE ,NEW_CODE [,CODESET] )
       Locale::Codes::LangFam::delete_langfam_code_alias  ( CODE [,CODESET] )
	   These routines are all documented in the Locale::Codes::API man page.

SEE ALSO

       Locale::Codes
	   The Locale-Codes distribution.

       Locale::Codes::API
	   The list of functions supported by this module.

       http://www.loc.gov/standards/iso639-5/id.php
	   ISO 639-5 .

AUTHOR

       See Locale::Codes for full author history.

       Currently maintained by Sullivan Beck (sbeck@cpan.org).

COPYRIGHT

	  Copyright (c) 2011-2012 Sullivan Beck

       This module is free software; you can redistribute it and/or modify it under the same terms as Perl itself.

perl v5.16.2							    2012-10-11					       Locale::Codes::LangFam(3pm)
All times are GMT -4. The time now is 09:57 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy