Sponsored Content
Top Forums Shell Programming and Scripting Downloading jpgs from a gallery type website Post 302899542 by workisnotfun on Tuesday 29th of April 2014 06:24:42 PM
Old 04-29-2014
I am having the most trouble with step two I think.

I'm assuming the -oh is the option -o and -h?

Step 1 downloads the files fine but they're stored in my directory as 1.html *dot* , 2.html *dot* etc, with a dot right after (not a period), not sure if this is a problem but Step 2 doesn't seem to be able to find any .html files

and so Step 3 fails because there is no urls.txt. What could be the problem?

---------- Post updated at 05:24 PM ---------- Previous update was at 04:55 PM ----------

I think I might have found a different problem actually.

Running this in terminal works fine
Code:
wget -nd -H -p -A jpg,jpeg,png,gif -e robots=off www.url.example

but when I put this in my bash script and run it I get awaiting response... 404 Not Found. At the end of the jpg url, for some reason a %0D gets appended to the end which I'm thinking makes wget go to the wrong url.

I've been trying a different approach than my earlier one since I couldn't get that working. What could be the problem now so that I can automate the downloading?

Last edited by workisnotfun; 04-29-2014 at 08:13 PM..
 

8 More Discussions You Might Find Interesting

1. UNIX Desktop Questions & Answers

Recommendations for good shell utility to resize JPGs?

Not sure if this is the right place to be posting this. If not, let me know where it fits. I am running RedHat Linux 8.0. I've recently acquired a Sony Mavica digital camera. In short, this thing is awesome (uses cd-rw!). I have been taking high quality images, but I now have the need to try... (2 Replies)
Discussion started by: deckard
2 Replies

2. UNIX for Dummies Questions & Answers

Lynx - Downloading - extension handling - changing mime type?

Using Lynx, when I try to download a .rar, it confirms I want to download and its got it as an appication/rar file. However, split archives that end in .r## (.r00, .r01 ...) are not recognized as an appication/rar file and it reads the file like a .txt or .html. How can I fix this? Thanks! (2 Replies)
Discussion started by: yitzle
2 Replies

3. Shell Programming and Scripting

String type to date type

Can one string type variable changed into the date type variable. (1 Reply)
Discussion started by: rinku
1 Replies

4. Programming

array type has incomplete element type

Dear colleagues, One of my friend have a problem with c code. While compiling a c program it displays a message like "array type has incomplete element type". Any body can provide a solution for it. Jaganadh.G (1 Reply)
Discussion started by: jaganadh
1 Replies

5. Shell Programming and Scripting

Downloading info from website to database

Hi guys! I created a database using mysql in bash now i would like to download weather info from the data.(temp, date and time)...and just store this in the database to display after every 3 hours or so... i have tried to get the website using wget and now dont exactly now how to go from here... (0 Replies)
Discussion started by: vadharah
0 Replies

6. Shell Programming and Scripting

File Management: How do I move all JPGS in a folder structure to a single folder?

This is the file structure: DESKTOP/Root of Photo Folders/Folder1qweqwasdfsd/*jpg DESKTOP/Root of Photo Folders/Folder2asdasdasd/*jpg DESKTOP/Root of Photo Folders/Folder3asdadfhgasdf/*jpg DESKTOP/Root of Photo Folders/Folder4qwetwdfsdfg/*jpg DESKTOP/Root of Photo... (4 Replies)
Discussion started by: guptaxpn
4 Replies

7. Windows & DOS: Issues & Discussions

Downloading a file from Website to a Windows Folder

Hi, Is it possible to download a file using Wget or some other command from a Windows machine? Say I want to download something from https server to C:\ABC\abc.xls Any ideas, Thanks. (4 Replies)
Discussion started by: dohko
4 Replies

8. Shell Programming and Scripting

Wget error while downloading from https website

Hi, I would like to download a file from a https website. I don't have the file name as it changes every day. I am using the following command: wget --no-check-certificate -r -np --user=ABC --password=DEF -O temp.txt https://<website/directory> I am getting followin error in my... (9 Replies)
Discussion started by: pinnacle
9 Replies
GKRELLKAM-LIST(5)						File Formats Manual						 GKRELLKAM-LIST(5)

NAME
gkrellkam-list - format of the list description files used by GKrellKam DESCRIPTION
GKrellKam lists describe a series of image sources, used in order or at random by the GKrellKam plugin. Online lists (downloadable by GKrellKam with HTTP) use this same syntax. Blank lines and comments (beginning with '#') in GKrellKam lists are ignored. Every line of text that does not begin with a tab character is considered an image source. An image source can be one of several types: image Sources of type image consist of a complete local filename to an image file. url Type url is just that; a ftp:// or http:// URL pointing at an online image file. script Type script is a system command, executed in a shell by GKrellKam when it's time to get this image. The output of the command is assumed to be a complete filename of a local image file, optionally terminated with newlines/whitespace. The corresponding image is loaded. list Type list causes a different GKrellKam list to be included in the current one. Each image source line should look like this: type: name The "type: " part of the line can be omitted for images and URLs, to make these list files backwards compatible with earlier versions of GKrellKam, but it is suggested that you specify them. It will enhance the readability of the list file. There are also properties that can be applied to types image, url, and script. Properties must follow the image source line that they mod- ify, and must begin with a tab character. These include: tooltip This sets the message shown when the mouse is kept over the image panel in GKrellKam for a few seconds. If not set, the image's filename or URL is shown instead. seconds This changes the number of seconds that an image will remain displayed. If not set, the "Default number of seconds" set for that panel in the GKrellM configuration window will be used. refresh If set, this sets the minimum length of time that must elapse before the image is re-loaded. If the image cycles up before that time, the old image will be used. For script types, this means that the script will not be re-run. For url types, the image will not be fetched using wget. When this property is not set, url images will be downloaded every time they are displayed, and scripts will be run every time. EXAMPLE
This is a simple example of a GKrellKam list. For a better example, see example.list from this distribution. # .krellkam.list # This list rotates through ~/pics/mypic[1-4].jpg along with a # webcam, and then displays the pictures in ~/lists/sub.list image: /home/paul/pics/mypic1.jpg /home/paul/pics/mypic2.jpg image: /home/paul/pics/mypic3.jpg http://www.usu.edu/webcam/fullsize.jpg [tab] tooltip: This is the building where I work image: /home/paul/pics/mypic4.jpg [tab] seconds: 4 [tab] tooltip: Don't display this one very long list: /home/paul/lists/sub.list FILES
$HOME/.krellkam.list - The default source for the first GKrellKam panel example.list - A sample GKrellKam list AUTHOR
GKrellKam was written by paul cannon <paul@cannon.cs.usu.edu>. This manual page was written by the same author for inclusion in the GKrel- lKam distribution, and may be used by others. SEE ALSO
wget(8) Dec 7, 2001 GKRELLKAM-LIST(5)
All times are GMT -4. The time now is 06:53 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy