Sponsored Content
Full Discussion: Wget help
Top Forums UNIX for Dummies Questions & Answers Wget help Post 302894266 by sea on Monday 24th of March 2014 03:01:22 PM
Old 03-24-2014
How about crawling the whole server, retrieving all findings as a list only.
In the next step, parse the list for your files (zip,rar) and then download those findings to current dir.

eg something like:
Code:
wget -r --spider example.com > retrieved.txt
for FOUND in $(grep -e rar -e zip retrieved.txt);do
  wget -nc $FOUND > $(basename $FOUND)
done

 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

wget -r

I have noticed a lot of expensive books appearing online so I have decided to copy them to CD. I was going to write a program in java to do this, but remembered that wget GNU program some of you guys were talking about. Instead of spending two hours or so writing a program to do this.... (1 Reply)
Discussion started by: photon
1 Replies

2. Shell Programming and Scripting

wget help

i am trying to ftp files/dirs with wget. i am having an issue where the path always takes me to my home dir even when i specify something else. For example: wget -m ftp://USER:PASS@IP_ADDRESS/Path/on/remote/box ...but if that path on the remote box isn't in my home dir it doesn't change to... (0 Replies)
Discussion started by: djembeplayer
0 Replies

3. Shell Programming and Scripting

Help with wget

Hi, i need temperature hourly from a web page Im using wget to get the web page. I would like to save the page downloaded in a file called page. I check the file everytime i run the wget function but its not saving but instead creates a wx.php file....Each time i run it...a new wx.php file is... (2 Replies)
Discussion started by: vadharah
2 Replies

4. Shell Programming and Scripting

wget

Hi I want to download some files using wget , and want to save in a specified directory. Is there any way to save it.Please suggest me. (1 Reply)
Discussion started by: mnmonu
1 Replies

5. Shell Programming and Scripting

wget help?

can someone please help in understanding this shell script? wget --progress=dot:mega --cut-dirs=4 -r -c -nH -np --reject index.html*,icons/*.gif \ http://*****.oz.xxxxx.com:<portnum>/omcsm/releases/dew/${UPGRADE_VERSION}/ (1 Reply)
Discussion started by: dnam9917
1 Replies

6. UNIX for Dummies Questions & Answers

Wget

...... (1 Reply)
Discussion started by: hoo
1 Replies

7. Shell Programming and Scripting

WGET help!

Hi Friends, I have an url like this https://www.unix.com/help/ In this help directory, I have more than 300 directories which contains file or files. So, the 300 directories are like this http://unix.com/help/ dir1 file1 dir2 file2 dir3 file3_1 file3_2... (4 Replies)
Discussion started by: jacobs.smith
4 Replies

8. Red Hat

Wget

If I run the following command wget -r --no-parent --reject "index.html*" 10.11.12.13/backups/ A local directory named 10.11.12.13/backups with the content of web site data is created. What I want to do is have the data placed in a local directory called $HOME/backups. Thanks for... (1 Reply)
Discussion started by: popeye
1 Replies

9. Shell Programming and Scripting

Wget and gz

Can wget be used to goto a site and piped into a .gz extrated command? wget ftp://ftp.ncbi.nlm.nih.gov/pub/clinvar/vcf_GRCh37 | gunzip -d clinvar_20150603.vcf.gz (1 Reply)
Discussion started by: cmccabe
1 Replies

10. Shell Programming and Scripting

Wget - working in browser but cannot download from wget

Hi, I need to download a zip file from my the below US govt link. https://www.sam.gov/SAMPortal/extractfiledownload?role=WW&version=SAM&filename=SAM_PUBLIC_MONTHLY_20160207.ZIP I only have wget utility installed on the server. When I use the below command, I am getting error 403... (2 Replies)
Discussion started by: Prasannag87
2 Replies
GIT-LOST-FOUND(1)						    Git Manual							 GIT-LOST-FOUND(1)

NAME
git-lost-found - Recover lost refs that luckily have not yet been pruned SYNOPSIS
git lost-found DESCRIPTION
NOTE: this command is deprecated. Use git-fsck(1) with the option --lost-found instead. Finds dangling commits and tags from the object database, and creates refs to them in the .git/lost-found/ directory. Commits and tags that dereference to commits are stored in .git/lost-found/commit, and other objects are stored in .git/lost-found/other. OUTPUT
Prints to standard output the object names and one-line descriptions of any commits or tags found. EXAMPLE
Suppose you run git tag -f and mistype the tag to overwrite. The ref to your tag is overwritten, but until you run git prune, the tag itself is still there. $ git lost-found [1ef2b196d909eed523d4f3c9bf54b78cdd6843c6] GIT 0.99.9c ... Also you can use gitk to browse how any tags found relate to each other. $ gitk $(cd .git/lost-found/commit && echo ??*) After making sure you know which the object is the tag you are looking for, you can reconnect it to your regular .git/refs hierarchy. $ git cat-file -t 1ef2b196 tag $ git cat-file tag 1ef2b196 object fa41bbce8e38c67a218415de6cfa510c7e50032a type commit tag v0.99.9c tagger Junio C Hamano <junkio@cox.net> 1131059594 -0800 GIT 0.99.9c This contains the following changes from the "master" branch, since ... $ git update-ref refs/tags/not-lost-anymore 1ef2b196 $ git rev-parse not-lost-anymore 1ef2b196d909eed523d4f3c9bf54b78cdd6843c6 GIT
Part of the git(1) suite Git 1.7.10.4 11/24/2012 GIT-LOST-FOUND(1)
All times are GMT -4. The time now is 07:01 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy