Hello. Invalid would mean 'broken links'; sites that no longer exist or are unreachable, etc.
I can extract the http links from html files (firefox) with sed easily enough, producing a list to loop through. I'm just not sure whether to use wget or ??? to attempt to reach the sites and get the error msgs. I want to do it in the console, not the browser. I thought I'd ask around for a script or advice from someone who might have already done this.
Hmm. The biggest problem I see is how many sites redirect bad links to valid pages. Try https://www.unix.com/qwerty/ui/op for an example. It's one thing to catch a 404 -- another to check that the contents were what you wanted.
wget does the deed nicely. I found that there are some false positives. Some websites just dont appreciate robots, so they return 'broken link' when the site is actually alive and reachable. But for my intentions that's no big deal.
What I have so far...
Code:
#!/bin/bash
SORTED_LIST=./urls.lst
BOOKMARKS=./bookmarks.html
toDEL=./del.lst
> $toDEL
while read XX
do
log=$(wget --tries=1 -T 5 --spider -nv $XX 2>&1)
if [[ $(echo $log | grep -c "broken link") -gt 0 ]]; then
echo "$log"
echo "$XX" >> $toDEL
# sed -n "s|$XX|&|p" $BOOKMARKS
sed -i "s|^.*$XX.*$||g" $BOOKMARKS
fi
done < $SORTED_LIST
I am using Internet Explorer v10 at work and regularly need to import my personal Firefox bookmarks over. Long story short, I have found the import falling over on any bookmark elements which are over 256 characters.
The bookmark file contains bookmarks of this format:
<DT><A... (4 Replies)
Hello to all,
I have a *.ps file, which needs to be converted to a pdf file. The file is huge & i would like it to have bookmarked according to main titles (already provided).
What is the option used in ps2pdf command, that will generate bookmarks from a *.ps file ? Kindly help. Thanks.
... (0 Replies)
This is the code:
while test 1 -eq 1
do
read a
$a
if test $a = stop
then
break
fi
done
I read a command on every loop an execute it.
I check if the string equals the word stop to end the loop,but it say that I gave too many arguments to test.
For example echo hello.
Now the... (1 Reply)
This is probably one of the dumber questions I've posted on ANY Linux forum, but I'm in a fog and can't answer it myself. My attempts at searching the forums go no where.
I have Win XP and Ubuntu dual booting (via WUBI install). I want to use the Ubuntu side more but I've socked away a LOT of... (1 Reply)
Ok, so I want to push out Firefox bookmarks to all the student accounts on my Mac server, but we all know how fun-loving Firefox profiles are, with their random czg6hi72.default profile names.
My question is how would I send the bookmarks.html file out to /Students/<username>/Library/Application... (5 Replies)
