Sponsored Content
Top Forums Shell Programming and Scripting Downloading jpgs from a gallery type website Post 302899538 by workisnotfun on Tuesday 29th of April 2014 05:39:08 PM
Old 04-29-2014
Downloading jpgs from a gallery type website

Can someone explain what this does step by step? I found this script on stackoverflow and want to customize it for personal use in downloading jpg images from a website.

Code:
# get all pages 
curl 'http://domain.com/id/[1-151468]' -o '#1.html' 

# get all images 
grep -oh 'http://pics.domain.com/pics/original/.*jpg' *.html >urls.txt 

# download all images 
sort -u urls.txt | wget -i-

1. What I think the first like does is download the pages of domain with curl but what does the '#1.html' mean?

2. Why in .*jpg is the * after the '.'? Also what is this trying to do? I attempted altering this using a different website but there's an error grep: *.html: No such file or directory even though the first command is downloading the html files just fine.

3. I think the third option is just organizing the results and wget goes to the jpg's website and downloads the jpgs.

Moderator's Comments:
Mod Comment Code tags for code, please.

Last edited by Corona688; 04-29-2014 at 06:47 PM..
 

8 More Discussions You Might Find Interesting

1. UNIX Desktop Questions & Answers

Recommendations for good shell utility to resize JPGs?

Not sure if this is the right place to be posting this. If not, let me know where it fits. I am running RedHat Linux 8.0. I've recently acquired a Sony Mavica digital camera. In short, this thing is awesome (uses cd-rw!). I have been taking high quality images, but I now have the need to try... (2 Replies)
Discussion started by: deckard
2 Replies

2. UNIX for Dummies Questions & Answers

Lynx - Downloading - extension handling - changing mime type?

Using Lynx, when I try to download a .rar, it confirms I want to download and its got it as an appication/rar file. However, split archives that end in .r## (.r00, .r01 ...) are not recognized as an appication/rar file and it reads the file like a .txt or .html. How can I fix this? Thanks! (2 Replies)
Discussion started by: yitzle
2 Replies

3. Shell Programming and Scripting

String type to date type

Can one string type variable changed into the date type variable. (1 Reply)
Discussion started by: rinku
1 Replies

4. Programming

array type has incomplete element type

Dear colleagues, One of my friend have a problem with c code. While compiling a c program it displays a message like "array type has incomplete element type". Any body can provide a solution for it. Jaganadh.G (1 Reply)
Discussion started by: jaganadh
1 Replies

5. Shell Programming and Scripting

Downloading info from website to database

Hi guys! I created a database using mysql in bash now i would like to download weather info from the data.(temp, date and time)...and just store this in the database to display after every 3 hours or so... i have tried to get the website using wget and now dont exactly now how to go from here... (0 Replies)
Discussion started by: vadharah
0 Replies

6. Shell Programming and Scripting

File Management: How do I move all JPGS in a folder structure to a single folder?

This is the file structure: DESKTOP/Root of Photo Folders/Folder1qweqwasdfsd/*jpg DESKTOP/Root of Photo Folders/Folder2asdasdasd/*jpg DESKTOP/Root of Photo Folders/Folder3asdadfhgasdf/*jpg DESKTOP/Root of Photo Folders/Folder4qwetwdfsdfg/*jpg DESKTOP/Root of Photo... (4 Replies)
Discussion started by: guptaxpn
4 Replies

7. Windows & DOS: Issues & Discussions

Downloading a file from Website to a Windows Folder

Hi, Is it possible to download a file using Wget or some other command from a Windows machine? Say I want to download something from https server to C:\ABC\abc.xls Any ideas, Thanks. (4 Replies)
Discussion started by: dohko
4 Replies

8. Shell Programming and Scripting

Wget error while downloading from https website

Hi, I would like to download a file from a https website. I don't have the file name as it changes every day. I am using the following command: wget --no-check-certificate -r -np --user=ABC --password=DEF -O temp.txt https://<website/directory> I am getting followin error in my... (9 Replies)
Discussion started by: pinnacle
9 Replies
MRENAME(1)						      General Commands Manual							MRENAME(1)

NAME
mrename - program to rename files SYNOPSIS
mrename 'pattern' prefix [option] DESCRIPTION
This manual page documents briefly the mrename command. This manual page was written for the Debian GNU/Linux distribution because the original program does not have a manual page. mrename is a tool for easy and automatic renaming of many files. The 'pattern' is the pattern to search files to rename (quoted to avoid that bash resolve it), and prefix is the prefix that will be added to the name of each file. The two alternative options for copying or moving files in the new name are explained below. All parameters are needed, and you have to stay and launch the script in the same direc- tory of the files to be renamed. The program should be able to write in this directory. OPTIONS
There are only the following three options. -c The option -c will copy each file with the new filename. -m The option -m will move each file in the new filename. -h Display help. EXAMPLE
If you have a directory with two jpeg images prof.jpg and forp.jpg and you want to add them a prefix like item0, item1 etc.. (that is item0prof.jpg, item1forp.jpg etc..) do this: cd /path/to/the/images mrename '*.jpg' item -c to copy each matching file into another with the new name mrename '*.jpg' item -m to rename each file without keeping a copy with the previous name Word-Wide-Web: http://alfalinux.sourceforge.net/mrename.php3 AUTHOR
: Giancarlo -rofus- Erra e-mail: rofus@mindless.com This manual page was written by Dr. Guenter Bechly <gbechly@debian.org>, for the Debian GNU/Linux system (but may be used by others). It is distributed under the GPL just like mrename itself. October 22, 2000 MRENAME(1)
All times are GMT -4. The time now is 07:19 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy