Sponsored Content
Top Forums UNIX for Dummies Questions & Answers Awk: print all URL addresses between iframe tags without repeating an already printed URL Post 302602859 by striker4o on Tuesday 28th of February 2012 02:12:10 PM
Old 02-28-2012
Yeah, thanks. I sorted out "sort -u" almost immediately after my dumb question Smilie

So here is the finalist:

Code:
find . -name "*php*" -or -name "*htm*" -type f| awk '/http/' RS=\" *.* | sort -u

 

10 More Discussions You Might Find Interesting

1. UNIX for Advanced & Expert Users

url calling and parameter passing to url in script

Hi all, I need to write a unix script in which need to call a url. Then need to pass parameters to that url. please help. Regards, gander_ss (1 Reply)
Discussion started by: gander_ss
1 Replies

2. Shell Programming and Scripting

url calling and parameter passing to url in script

Hi all, I need to write a unix script in which need to call a url. Then need to pass parameters to that url. please help. Regards, gander_ss (1 Reply)
Discussion started by: gander_ss
1 Replies

3. UNIX for Dummies Questions & Answers

ReDirecting a URL to another URL - Linux

Hello, I need to redirect an existing URL, how can i do that? There's a current web address to a GUI that I have to redirect to another webaddress. Does anyone know how to do this? This is on Unix boxes Linux. example: https://m45.testing.address.net/host.php make it so the... (3 Replies)
Discussion started by: SkySmart
3 Replies

4. UNIX for Advanced & Expert Users

Need to grab URL and place between <A></A> Tags

my output looks like: <A HREF="http://support.apple.com/kb/HT1629"> </A> <A HREF="http://support.apple.com/kb/HT1200"> </A> <A HREF="http://old.nabble.com/AFP-eating-up-CPU-td19976358.html"> </A> <A HREF="http://jochsner.dyndns.org/scripts/NHR.html"> </A> <A... (3 Replies)
Discussion started by: glev2005
3 Replies

5. Shell Programming and Scripting

how to judge wether a url is valid or not using awk

rt 3ks:confused: (6 Replies)
Discussion started by: rainboisterous
6 Replies

6. Shell Programming and Scripting

Extract URL from RSS Feed in AWK

Hi, I have following data file; <outline title="Matt Cutts" type="rss" version="RSS" xmlUrl="http://www.mattcutts.com/blog/feed/" htmlUrl="http://www.mattcutts.com/blog"/> <outline title="Stone" text="Stone" type="rss" version="RSS" xmlUrl="http://feeds.feedburner.com/STC-Art"... (8 Replies)
Discussion started by: fahdmirza
8 Replies

7. Web Development

Regex to rewrite URL to another URL based on HTTP_HOST?

I am trying to find a way to test some code, but I need to rewrite a specific URL only from a specific HTTP_HOST The call goes out to http://SUB.DOMAIN.COM/showAssignment/7bde10b45efdd7a97629ef2fe01f7303/jsmodule/Nevow.Athena The ID in the middle is always random due to the cookie. I... (5 Replies)
Discussion started by: EXT3FSCK
5 Replies

8. UNIX for Dummies Questions & Answers

URL decoding with awk

The challenge: Decode URL's, i.e. convert %HEX to the corresponding special characters, using only UNIX base utilities, and without having to type out each special character. I have an anonymous C code snippet where the author assigns each hex digit a number from 0 to 16 and then does some... (2 Replies)
Discussion started by: uiop44
2 Replies

9. Shell Programming and Scripting

awk and or sed command to sum the value in repeating tags in a XML

I have a XML in which <Amt Ccy="EUR">3.1</Amt> tag repeats. This is under another tag <Main>. I need to sum all the values of <Amt Ccy=""> (Ccy may vary) coming under <Main> using awk and or sed command. can some help? Sample looks like below <root> <Main> ... (6 Replies)
Discussion started by: bk_12345
6 Replies

10. Shell Programming and Scripting

Reading URL using Mechanize and dump all the contents of the URL to a file

Hello, Am very new to perl , please help me here !! I need help in reading a URL from command line using PERL:: Mechanize and needs all the contents from the URL to get into a file. below is the script which i have written so far , #!/usr/bin/perl use LWP::UserAgent; use... (2 Replies)
Discussion started by: scott_cog
2 Replies
Tree::Simple::Visitor::LoadDirectoryTree(3pm)		User Contributed Perl Documentation	     Tree::Simple::Visitor::LoadDirectoryTree(3pm)

NAME
Tree::Simple::Visitor::LoadDirectoryTree - A Visitor for loading the contents of a directory into a Tree::Simple object SYNOPSIS
use Tree::Simple::Visitor::LoadDirectoryTree; # create a Tree::Simple object whose # node is path to a directory my $tree = Tree::Simple->new("./"); # create an instance of our visitor my $visitor = Tree::Simple::Visitor::LoadDirectoryTree->new(); # set the directory sorting style $visitor->setSortStyle($visitor->SORT_FILES_FIRST); # create node filter to filter # out certain files and directories $visitor->setNodeFilter(sub { my ($item) = @_; return 0 if $item =~ /CVS/; return 1; }); # pass the visitor to a Tree::Simple object $tree->accept($visitor); # the tree now mirrors the structure of the directory DESCRIPTION
This visitor can be used to load a directory tree into a Tree::Simple hierarchy. METHODS
new There are no arguments to the constructor the object will be in its default state. You can use the "setNodeFilter" and "setSortStyle" methods to customize its behavior. setNodeFilter ($filter_function) This method accepts a CODE reference as its $filter_function argument and throws an exception if it is not a code reference. This code reference is used to filter the tree nodes as they are created. The function is given the current directory or file being added to the tree, and it is expected to return either true(1) of false(0) to determine if that directory should be traversed or file added to the tree. setSortStyle ($sort_function) This method accepts a CODE reference as its $sort_function argument and throws an exception if it is not a code reference. This function is used to sort the individual levels of the directory tree right before it is added to the tree being built. The function is passed the the current path, followed by the two items being sorted. The reason for passing the path in is so that sorting operations can be performed on the entire path if desired. Two pre-built functions are supplied and described below. SORT_FILES_FIRST This sorting function will sort files before directories, so that files are sorted alphabetically first in the list followed by directories sorted alphabetically. Here is example of how that would look: Tree/ Simple.pm Simple/ Visitor.pm VisitorFactory.pm Visitor/ PathToRoot.pm SORT_DIRS_FIRST This sorting function will sort directories before files, so that directories are sorted alphabetically first in the list followed by files sorted alphabetically. Here is example of how that would look: Tree/ Simple/ Visitor/ PathToRoot.pm Visitor.pm VisitorFactory.pm Simple.pm visit ($tree) This is the method that is used by Tree::Simple's "accept" method. It can also be used on its own, it requires the $tree argument to be a Tree::Simple object (or derived from a Tree::Simple object), and will throw and exception otherwise. The node value of the $tree argument (gotten by calling "getNodeValue") is considered the root directory from which we begin our traversal. We use File::Spec to keep our paths cross-platform, but it is expected that you will feed in a valid path for your OS. If the path either does not exist, or is not a directory, then an exception is thrown. The $tree argument which is passed to "visit" must be a leaf node. This is because this Visitor will create all the sub-nodes for this tree. If the tree is not a leaf, an exception is thrown. We do not require the tree to be a root though, and this Visitor will not affect any nodes above the $tree argument. BUGS
None that I am aware of. Of course, if you find a bug, let me know, and I will be sure to fix it. CODE COVERAGE
See the CODE COVERAGE section in Tree::Simple::VisitorFactory for more inforamtion. SEE ALSO
These Visitor classes are all subclasses of Tree::Simple::Visitor, which can be found in the Tree::Simple module, you should refer to that module for more information. AUTHOR
stevan little, <stevan@iinteractive.com> COPYRIGHT AND LICENSE
Copyright 2004, 2005 by Infinity Interactive, Inc. <http://www.iinteractive.com> This library is free software; you can redistribute it and/or modify it under the same terms as Perl itself. perl v5.10.1 2005-07-14 Tree::Simple::Visitor::LoadDirectoryTree(3pm)
All times are GMT -4. The time now is 03:29 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy