Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: extract the username and the url and print it using Python 3 and BeautifulSoup 4
Top Forums Programming extract the username and the url and print it using Python 3 and BeautifulSoup 4 Post 303045287 by bob123 on Sunday 15th of March 2020 04:01:52 PM
Old 03-15-2020
extract the username and the url and print it using Python 3 and BeautifulSoup 4
using python 3 and BeautifulSoup 4
my code when run returns a big list like the txt below
but 100 times more i have only posted one as an example


Code:
urls = ['https://example.com']
for url in urls:
    response = requests.get(url)
    soup = BeautifulSoup(response.content, "html.parser")
    for text in soup.find_all(text= True):
           print(text)


code above returns
this

Code:
{"users":{"userId":222222222,"username":"some_name","hlsPreviewUrl":"https://example.com/322/example-origin-live/some_name-322-33babf1b-6030-479a-ac41-b90bd8313678_aac/playlist.m3u8 }

i want to extract the username and the url and print it

i have tried to add this piece of code but i get nothing

Code:
 if 'username' in ['text']:

what code am i missing
to print just the username and the url
with python 3 and BeautifulSoup 4

thanks
Last edited by bob123; 03-15-2020 at 05:07 PM..
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

SED extract url - please help a lamer

Hello everybody. I have lines that looks something like this: <done16=""118"" done18=""$ title=""thisisatitle"" href=""/JoeBanana" alt=""Joe""><done16=""118"" done18=""$ title=""thisisatitle"" href=""/GeraldGiraffe" alt=""Gerald""> What kind of SED command would I need to use to extract... (4 Replies)
Discussion started by: digi
4 Replies

2. Shell Programming and Scripting

How to extract url from html page?

for example, I have an html file, contain <a href="http://awebsite" id="awebsite" class="first">website</a>and sometime a line contains more then one link, for example <a href="http://awebsite" id="awebsite" class="first">website</a><a href="http://bwebsite" id="bwebsite"... (36 Replies)
Discussion started by: 14th
36 Replies

3. Shell Programming and Scripting

help, extract username from file

there is a text file like this text here text here text here text here USER=someusername here text here text here text here how can i extract the someusername from this file which has a path like /home/myfile.txt? i want to store it in a variable $u (3 Replies)
Discussion started by: vanessafan99
3 Replies

4. UNIX for Dummies Questions & Answers

Awk: print all URL addresses between iframe tags without repeating an already printed URL

Here is what I have so far: find . -name "*php*" -or -name "*htm*" | xargs grep -i iframe | awk -F'"' '/<iframe*/{gsub(/.\*iframe>/,"\"");print $2}' Here is an example content of a PHP or HTM(HTML) file: <iframe src="http://ADDRESS_1/?click=5BBB08\" width=1 height=1... (18 Replies)
Discussion started by: striker4o
18 Replies

5. UNIX for Dummies Questions & Answers

Launch a URL,validate username and password using wget or curl

Hi All, I want to launch "ex: http://gmail.com" from the cmd window and validate the credentials with username and password, is it possible? I have found something like this "wget --http-user=USER' --http-password=PASSWORD http://gmail.com" am new to this and unable to find a solution, i... (0 Replies)
Discussion started by: harsha85
0 Replies

6. UNIX for Dummies Questions & Answers

Mailx: extract username from a file

Hey guys, I am trying to send email to the usernames listed in a ascii file. I know about the .mailrc file. I want to know if I there is a way to extract the usernames from a file and email them. I tried something like this: mailx -s "subject" '~cat... (2 Replies)
Discussion started by: Nirav4
2 Replies

7. Shell Programming and Scripting

Newbie Python Url Scraper

I setup Zoneminder and have been playing around with setting up a couple of Wanscam PTZ ip cameras in which I have been running into road blocks with streaming and etc. I cant find much information on the camera and its webserver that sits on it and wanted to get a an absolute directory structure... (5 Replies)
Discussion started by: metallica1973
5 Replies

8. Shell Programming and Scripting

Connect to Remote DB2 ,using URL ,username,Password

Hi All, I am new to Shell scripting, I want to connect to DB2 database through URL,username and password Please help me out. I read so many posts in that there is no where it is mentioned about how to connect to a remote database. Thanks in advance :) (0 Replies)
Discussion started by: Rohit G
0 Replies

9. Shell Programming and Scripting

Python BeautifulSoup Re Finding Digits Within Tags

I am writing a little python script that needs to grab version numbers between "<td>4.2.2</td>" within the tbody of the page: blah blah blah blah blah Is it possible to use a one-liner to scrap only the digits between the tags: "<td>4.2.2</td>" so it spits out: 4.2.2 4.2.1 etc..... (2 Replies)
Discussion started by: metallica1973
2 Replies

10. Programming

[Python] BeautifulSoup tags > </a>

using BeautifulSoup how can i get the txt between all the > </a> example >The Student .mp4</a> thanks (10 Replies)
Discussion started by: bob123
10 Replies

LEARN ABOUT OSX

locale::codes::langvar

Locale::Codes::LangVar(3pm)				 Perl Programmers Reference Guide			       Locale::Codes::LangVar(3pm)

NAME

       Locale::Codes::LangVar - standard codes for language variation identification

SYNOPSIS

	  use Locale::Codes::LangVar;

	  $lvar = code2langvar('acm');		       # $lvar gets 'Mesopotamian Arabic'
	  $code = langvar2code('Mesopotamian Arabic'); # $code gets 'acm'

	  @codes   = all_langvar_codes();
	  @names   = all_langvar_names();

DESCRIPTION

       The "Locale::Codes::LangVar" module provides access to standard codes used for identifying language variations, such as those as defined in
       the IANA language registry.

       Most of the routines take an optional additional argument which specifies the code set to use. If not specified, the default IANA language
       registry codes will be used.

SUPPORTED CODE SETS

       There are several different code sets you can use for identifying language variations. A code set may be specified using either a name, or
       a constant that is automatically exported by this module.

       For example, the two are equivalent:

	  $lvar = code2langvar('en','alpha-2');
	  $lvar = code2langvar('en',LOCALE_CODE_ALPHA_2);

       The codesets currently supported are:

       alpha
	   This is the set of alphanumeric codes from the IANA language registry, such as 'arevela' for Eastern Armenian.

	   This code set is identified with the symbol "LOCALE_LANGVAR_ALPHA".

	   This is the default code set.

ROUTINES

       code2langvar ( CODE [,CODESET] )
       langvar2code ( NAME [,CODESET] )
       langvar_code2code ( CODE ,CODESET ,CODESET2 )
       all_langvar_codes ( [CODESET] )
       all_langvar_names ( [CODESET] )
       Locale::Codes::LangVar::rename_langvar  ( CODE ,NEW_NAME [,CODESET] )
       Locale::Codes::LangVar::add_langvar  ( CODE ,NAME [,CODESET] )
       Locale::Codes::LangVar::delete_langvar  ( CODE [,CODESET] )
       Locale::Codes::LangVar::add_langvar_alias  ( NAME ,NEW_NAME )
       Locale::Codes::LangVar::delete_langvar_alias  ( NAME )
       Locale::Codes::LangVar::rename_langvar_code  ( CODE ,NEW_CODE [,CODESET] )
       Locale::Codes::LangVar::add_langvar_code_alias  ( CODE ,NEW_CODE [,CODESET] )
       Locale::Codes::LangVar::delete_langvar_code_alias  ( CODE [,CODESET] )
	   These routines are all documented in the Locale::Codes::API man page.

SEE ALSO

       Locale::Codes
	   The Locale-Codes distribution.

       Locale::Codes::API
	   The list of functions supported by this module.

       http://www.iana.org/assignments/language-subtag-registry
	   The IANA language subtag registry.

AUTHOR

       See Locale::Codes for full author history.

       Currently maintained by Sullivan Beck (sbeck@cpan.org).

COPYRIGHT

	  Copyright (c) 2011-2012 Sullivan Beck

       This module is free software; you can redistribute it and/or modify it under the same terms as Perl itself.

perl v5.16.2							    2012-10-11					       Locale::Codes::LangVar(3pm)
All times are GMT -4. The time now is 04:57 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy