Login or Register to Ask a Question and Join Our Community


UNIX for Dummies Questions & Answers

using Lynx and Grep to return search page rank - help

 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers using Lynx and Grep to return search page rank - help
# 1  
Old 09-18-2007
using Lynx and Grep to return search page rank - help
I am writing a script which will read in search terms from a text file and pass each line to Lynx. Lynx will grab the source html, then I want grep/tr, whatever to search for the first occurance of a term (mydomain.name), then delete from that 1st occurance on, creating a new end of file.

Then I want to count a certain marker <class=L> in the remaining source to determine the search engine page rank until end of file.

This is what I have so far. My primary issue is that google returns all search html source as 1 line, which is why I need to count the style tag <class=L> (in this case lowercase L), what I have right now grab the search terms and the results, but I'm unsure of where to go from here.

#!/bin/bash
cat ${1} | while read searchTerm; do
#echo "${searchTerm}"
lynx -source -accept_all_cookies "http://www.google.com/search?q=$searchTerm">> /path/to/dir/archive.txt
done

Thanks in Advance!
 
Login or Register to Ask a Question

Previous Thread | Next Thread

7 More Discussions You Might Find Interesting

1. What is on Your Mind?

The Return of the Show Post Page

You may recall we used to have a "Show Post" link in each post that referenced the post and had a link to this post in page. I am going to bring back this feature and and renovate the page: https://www.unix.com/members/1-albums215-picture1013.png So that page has a "Under Renovation"... (1 Reply)
Discussion started by: Neo
1 Replies

2. What is on Your Mind?

Update to Advanced Search Page (Phase 1)

Update: I have completed the first phase of revamping the "Advanced Search" page using Bootstrap (desktop not mobile yet): https://www.unix.com/search.php https://www.unix.com/search.php I may change this to a Bootstrap modal later and change the CSS a bit more; but for now it is much... (0 Replies)
Discussion started by: Neo
0 Replies

3. Web Development

Fix For Google Page Rank: Wordpress List Rank Dashboard Widget

Here is the fix for the recent Google changes to their pagerank API. For example, in the List Rank Dashboard Widget Wordpress Plugin (Version 1.7), in this plugin file: list-rank-dashboard-widget/wp-list-rank-class.php in this function: function getGooglePR($url) Change this line: ... (0 Replies)
Discussion started by: Neo
0 Replies

4. Solaris

How to search man page (pdf file)

I'm not sure is it just only me or something. I try to download man page documentation from SUN.COM. However, it seems I can't search what I looking for in SUN man page. I try to search "passwd" but it return me a word "less" why this pdf can't search or is it require specific plugin to... (3 Replies)
Discussion started by: Smith
3 Replies

5. UNIX for Dummies Questions & Answers

Lynx Grep Pattern Match 2 conditions Print from Start to End

I am working on a scraping project and I am stuck at this tiny grep pattern match. Sample text : FPA List. FPA List. FPA List. FPA List. FPA List. FPA List. FPA List. FPA List. ABC Personal Planning Catherine K. Wat Cath Wat Catherine K. Wat Catherine K. Wat IFRAME:... (8 Replies)
Discussion started by: kkiran
8 Replies

6. UNIX for Dummies Questions & Answers

| help | unix | grep - Can I use grep to return a string with exactly n matches?

Hello, I looking to use grep to return a string with exactly n matches. I'm building off this: ls -aLl /bin | grep '^.\{9\}x' | tr -s ' ' -rwxr-xr-x 1 root root 632816 Nov 25 2008 vi -rwxr-xr-x 1 root root 632816 Nov 25 2008 view -rwxr-xr-x 1 root root 16008 May 25 2008... (7 Replies)
Discussion started by: MykC
7 Replies

7. UNIX for Advanced & Expert Users

Man page search issue

I have an issue with my man page configuration. I can able to see man pages for 1st section. But for not the rest of the sections. But If it give section number, man page is working properly Following are the details echo $MANPATH... (4 Replies)
Discussion started by: praveenkumar_l
4 Replies
Login or Register to Ask a Question

LEARN ABOUT DEBIAN

inssegmbkrds

INSSEGMBKRDS(3) 					      RDS PHYSICAL FUNCTIONS						   INSSEGMBKRDS(3)

NAME

       inssegmbkrds - adds in RDS instance all the segments of MBK instance

ORIGIN

       This  software  belongs	to  the  ALLIANCE CAD SYSTEM developed by the ASIM team at LIP6 laboratory of Universite Pierre et Marie CURIE, in
       Paris, France.

       Web : http://asim.lip6.fr/recherche/alliance/
       E-mail : alliance-users@asim.lip6.fr

SYNOPSYS

       #include "rdsnnn.h"
	rdsrec_list *inssegmbkrds( InstanceMbk, ModelMbk, InstanceRds, Lynx )
	   phins_list  *InstanceMbk;
	   phfig_list  *ModelMbk;
	   rdsins_list *InstanceRds;
	   char 	Lynx;

PARAMETER

       InstanceMbk	   The MBK instance which contains the segments to convert.

       ModelMbk 	   Pointer to the model of the MBK instance.

       InstanceRds	   The instance which has to receive converted segments from MBK to RDS format.

       Lynx		   Flag used for the segment conversion. If the parameter Lynx is set to 0 then thi s is the normal  conversion  mode.	If
			   the parameter Lynx is set to 1 then the rds structure generated permits to extract equipotentials rectangles.

DESCRIPTION

       The inssegmbkrds function adds in the RDS instance all segments of the MBK instance converted to RDS format. All newly created segments are
       chained each other (circulary list) using the USER field of the rdsrec_list structure.

RETURN VALUE

       Pointer to the head of list of the added rectangles to the RDS instance.

ERRORS

       "Rds202: rdsalloc error, can't continue !"
	      it's impossible to allocate the memory size desired

EXAMPLE

	       #include "mphnnn.h"
	       #include "mutnnn.h"
	       #include "rdsnnn.h"
	       #include "rfmnnn.h"
	       #include "rtlnnn.h"
	       typedef struct UserStruct
		{
		  char *STRING;
		  void *USER1;
		} UserStruct;
	       main()
		 {
		   phfig_list  *MbkFigure;
		   phins_list  *MbkInstance;
		   phfig_list  *MbkModelInstance;
		   rdsfig_list *RdsFigure;
		   rdsins_list *RdsInstance;
		   rdsrec_list *RdsSegmentList;
		   rdsrec_list *ScanSegment;
		   mbkenv();
		   rdsenv();
		   loadrdsparam();
		   /* creates MbkFigure Named "core" */
		   MbkFigure	    = addphfig("core");
		   /* creates RdsFigure named "core_2" */
		   RdsFigure	    = addrdsfig("core_2",sizeof(UserStruct));
		   /* adds Mbk instance "n1_y" to MbkFigure named "core" */
		   MbkInstance	    = addphins(MbkFigure,"n1_y","inv_1",NOSYM,4,9);
		   /* Gets Model of instance n1_y */
		   MbkModelInstance = getphfig("n1_y",'A');
		   /* creates Rds instance named "block_1" added to RdsFigure */
		   RdsInstance =
		      addrdsins(RdsFigure,"block","block_1",RDS_NOSYM,8,6);
		   /*
			 adds segments of "inv_1" instance Mbk in Rds instance
			 named "block_1"
		   */
		   RdsSegmentList   = inssegmbkrds ( MbkInstance,
						     MbkModelInstance,
						     RdsInstance,0 );
		   viewrdsrec ( RdsSegmentList );
		   for ( ScanSegment  = RdsSegmentList->USER ;
			 ScanSegment != RdsSegmentList ;
			 ScanSegment  = ScanSegment->USER )
		    {
		      viewrdsrec ( (rdsrec_list *) ScanSegment );
		    }
		 }

SEE ALSO

       librfm, librds, loadrdsparam

BUG REPORT

       This tool is under development at the ASIM department of the LIP6 laboratory.
       We need your feedback to improve documentation and tools.

ASIM
/LIP6							  October 1, 1997						   INSSEGMBKRDS(3)