Sponsored Content
Top Forums Programming extract the username and the url and print it using Python 3 and BeautifulSoup 4 Post 303045287 by bob123 on Sunday 15th of March 2020 04:01:52 PM
Old 03-15-2020
extract the username and the url and print it using Python 3 and BeautifulSoup 4

using python 3 and BeautifulSoup 4
my code when run returns a big list like the txt below
but 100 times more i have only posted one as an example


Code:
urls = ['https://example.com']
for url in urls:
    response = requests.get(url)
    soup = BeautifulSoup(response.content, "html.parser")
    for text in soup.find_all(text= True):
           print(text)


code above returns
this

Code:
{"users":{"userId":222222222,"username":"some_name","hlsPreviewUrl":"https://example.com/322/example-origin-live/some_name-322-33babf1b-6030-479a-ac41-b90bd8313678_aac/playlist.m3u8 }

i want to extract the username and the url and print it

i have tried to add this piece of code but i get nothing

Code:
 if 'username' in ['text']:

what code am i missing
to print just the username and the url
with python 3 and BeautifulSoup 4

thanks

Last edited by bob123; 03-15-2020 at 05:07 PM..
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

SED extract url - please help a lamer

Hello everybody. I have lines that looks something like this: <done16=""118"" done18=""$ title=""thisisatitle"" href=""/JoeBanana" alt=""Joe""><done16=""118"" done18=""$ title=""thisisatitle"" href=""/GeraldGiraffe" alt=""Gerald""> What kind of SED command would I need to use to extract... (4 Replies)
Discussion started by: digi
4 Replies

2. Shell Programming and Scripting

How to extract url from html page?

for example, I have an html file, contain <a href="http://awebsite" id="awebsite" class="first">website</a>and sometime a line contains more then one link, for example <a href="http://awebsite" id="awebsite" class="first">website</a><a href="http://bwebsite" id="bwebsite"... (36 Replies)
Discussion started by: 14th
36 Replies

3. Shell Programming and Scripting

help, extract username from file

there is a text file like this text here text here text here text here USER=someusername here text here text here text here how can i extract the someusername from this file which has a path like /home/myfile.txt? i want to store it in a variable $u (3 Replies)
Discussion started by: vanessafan99
3 Replies

4. UNIX for Dummies Questions & Answers

Awk: print all URL addresses between iframe tags without repeating an already printed URL

Here is what I have so far: find . -name "*php*" -or -name "*htm*" | xargs grep -i iframe | awk -F'"' '/<iframe*/{gsub(/.\*iframe>/,"\"");print $2}' Here is an example content of a PHP or HTM(HTML) file: <iframe src="http://ADDRESS_1/?click=5BBB08\" width=1 height=1... (18 Replies)
Discussion started by: striker4o
18 Replies

5. UNIX for Dummies Questions & Answers

Launch a URL,validate username and password using wget or curl

Hi All, I want to launch "ex: http://gmail.com" from the cmd window and validate the credentials with username and password, is it possible? I have found something like this "wget --http-user=USER' --http-password=PASSWORD http://gmail.com" am new to this and unable to find a solution, i... (0 Replies)
Discussion started by: harsha85
0 Replies

6. UNIX for Dummies Questions & Answers

Mailx: extract username from a file

Hey guys, I am trying to send email to the usernames listed in a ascii file. I know about the .mailrc file. I want to know if I there is a way to extract the usernames from a file and email them. I tried something like this: mailx -s "subject" '~cat... (2 Replies)
Discussion started by: Nirav4
2 Replies

7. Shell Programming and Scripting

Newbie Python Url Scraper

I setup Zoneminder and have been playing around with setting up a couple of Wanscam PTZ ip cameras in which I have been running into road blocks with streaming and etc. I cant find much information on the camera and its webserver that sits on it and wanted to get a an absolute directory structure... (5 Replies)
Discussion started by: metallica1973
5 Replies

8. Shell Programming and Scripting

Connect to Remote DB2 ,using URL ,username,Password

Hi All, I am new to Shell scripting, I want to connect to DB2 database through URL,username and password Please help me out. I read so many posts in that there is no where it is mentioned about how to connect to a remote database. Thanks in advance :) (0 Replies)
Discussion started by: Rohit G
0 Replies

9. Shell Programming and Scripting

Python BeautifulSoup Re Finding Digits Within Tags

I am writing a little python script that needs to grab version numbers between "<td>4.2.2</td>" within the tbody of the page: blah blah blah blah blah Is it possible to use a one-liner to scrap only the digits between the tags: "<td>4.2.2</td>" so it spits out: 4.2.2 4.2.1 etc..... (2 Replies)
Discussion started by: metallica1973
2 Replies

10. Programming

[Python] BeautifulSoup tags > </a>

using BeautifulSoup how can i get the txt between all the > </a> example >The Student .mp4</a> thanks (10 Replies)
Discussion started by: bob123
10 Replies
Prophet::Test(3pm)					User Contributed Perl Documentation					Prophet::Test(3pm)

   set_editor($code)
       Sets the subroutine that Prophet should use instead of "Prophet::CLI::Command::edit_text" (as this routine invokes an interactive editor)
       to $code.

   set_editor_script SCRIPT
       Sets the editor that Proc::InvokeEditor uses.

       This should be a non-interactive script found in t/scripts.

   import_extra($class, $args)
   in_gladiator($code)
       Run the given code using Devel::Gladiator.

   repo_path_for($username)
       Returns a path on disk for where $username's replica is stored.

   repo_uri_for($username)
       Returns a file:// URI for $USERNAME'S replica (with the correct replica type prefix).

   replica_uuid
       Returns the UUID of the test replica.

   database_uuid
       Returns the UUID of the test database.

   replica_last_rev
       Returns the sequence number of the last change in the test replica.

   as_user($username, $coderef)
       Run this code block as $username.  This routine sets up the %ENV hash so that when we go looking for a repository, we get the user's repo.

   replica_uuid_for($username)
       Returns the UUID of the given user's test replica.

   database_uuid_for($username)
       Returns the UUID of the given user's test database.

   ok_added_revisions( { CODE }, $numbers_of_new_revisions, $msg)
       Checks that the given code block adds the given number of changes to the test replica. $msg is optional and will be printed with the test
       if given.

   serialize_conflict($conflict_obj)
       Returns a simple, serialized version of a Prophet::Conflict object suitable for comparison in tests.

       The serialized version is a hash reference containing the following keys:
	   meta => { original_source_uuid => 'source_replica_uuid' }
	   records => { 'record_uuid' =>
			  { change_type => 'type',
			    props => { propchange_name => { source_old => 'old_val',
							    source_new => 'new_val',
							    target_old => 'target_val',
							  }
				     }
			  },
			'another_record_uuid' =>
			  { change_type => 'type',
			    props => { propchange_name => { source_old => 'old_val',
							    source_new => 'new_val',
							    target_old => 'target_val',
							  }
				     }
			  },
		      }

   serialize_changeset($changeset_obj)
       Returns a simple, serialized version of a Prophet::ChangeSet object suitable for comparison in tests (a hash).

   run_command($command, @args)
       Run the given command with (optionally) the given args using a new Prophet::CLI object. Returns the standard output of that command in
       scalar form or, in array context, the STDOUT in scalar form *and* the STDERR in scalar form.

       Examples:

	   run_command('create', '--type=Foo');

   load_record($type, $uuid)
       Loads and returns a record object for the record with the given type and uuid.

   as_alice CODE, as_bob CODE, as_charlie CODE, as_david CODE
       Runs CODE as alice, bob, charlie or david.

perl v5.10.1							    2009-09-02							Prophet::Test(3pm)
All times are GMT -4. The time now is 12:38 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy