Sponsored Content
Top Forums Shell Programming and Scripting Downloading jpgs from a gallery type website Post 302899538 by workisnotfun on Tuesday 29th of April 2014 05:39:08 PM
Old 04-29-2014
Downloading jpgs from a gallery type website

Can someone explain what this does step by step? I found this script on stackoverflow and want to customize it for personal use in downloading jpg images from a website.

Code:
# get all pages 
curl 'http://domain.com/id/[1-151468]' -o '#1.html' 

# get all images 
grep -oh 'http://pics.domain.com/pics/original/.*jpg' *.html >urls.txt 

# download all images 
sort -u urls.txt | wget -i-

1. What I think the first like does is download the pages of domain with curl but what does the '#1.html' mean?

2. Why in .*jpg is the * after the '.'? Also what is this trying to do? I attempted altering this using a different website but there's an error grep: *.html: No such file or directory even though the first command is downloading the html files just fine.

3. I think the third option is just organizing the results and wget goes to the jpg's website and downloads the jpgs.

Moderator's Comments:
Mod Comment Code tags for code, please.

Last edited by Corona688; 04-29-2014 at 06:47 PM..
 

8 More Discussions You Might Find Interesting

1. UNIX Desktop Questions & Answers

Recommendations for good shell utility to resize JPGs?

Not sure if this is the right place to be posting this. If not, let me know where it fits. I am running RedHat Linux 8.0. I've recently acquired a Sony Mavica digital camera. In short, this thing is awesome (uses cd-rw!). I have been taking high quality images, but I now have the need to try... (2 Replies)
Discussion started by: deckard
2 Replies

2. UNIX for Dummies Questions & Answers

Lynx - Downloading - extension handling - changing mime type?

Using Lynx, when I try to download a .rar, it confirms I want to download and its got it as an appication/rar file. However, split archives that end in .r## (.r00, .r01 ...) are not recognized as an appication/rar file and it reads the file like a .txt or .html. How can I fix this? Thanks! (2 Replies)
Discussion started by: yitzle
2 Replies

3. Shell Programming and Scripting

String type to date type

Can one string type variable changed into the date type variable. (1 Reply)
Discussion started by: rinku
1 Replies

4. Programming

array type has incomplete element type

Dear colleagues, One of my friend have a problem with c code. While compiling a c program it displays a message like "array type has incomplete element type". Any body can provide a solution for it. Jaganadh.G (1 Reply)
Discussion started by: jaganadh
1 Replies

5. Shell Programming and Scripting

Downloading info from website to database

Hi guys! I created a database using mysql in bash now i would like to download weather info from the data.(temp, date and time)...and just store this in the database to display after every 3 hours or so... i have tried to get the website using wget and now dont exactly now how to go from here... (0 Replies)
Discussion started by: vadharah
0 Replies

6. Shell Programming and Scripting

File Management: How do I move all JPGS in a folder structure to a single folder?

This is the file structure: DESKTOP/Root of Photo Folders/Folder1qweqwasdfsd/*jpg DESKTOP/Root of Photo Folders/Folder2asdasdasd/*jpg DESKTOP/Root of Photo Folders/Folder3asdadfhgasdf/*jpg DESKTOP/Root of Photo Folders/Folder4qwetwdfsdfg/*jpg DESKTOP/Root of Photo... (4 Replies)
Discussion started by: guptaxpn
4 Replies

7. Windows & DOS: Issues & Discussions

Downloading a file from Website to a Windows Folder

Hi, Is it possible to download a file using Wget or some other command from a Windows machine? Say I want to download something from https server to C:\ABC\abc.xls Any ideas, Thanks. (4 Replies)
Discussion started by: dohko
4 Replies

8. Shell Programming and Scripting

Wget error while downloading from https website

Hi, I would like to download a file from a https website. I don't have the file name as it changes every day. I am using the following command: wget --no-check-certificate -r -np --user=ABC --password=DEF -O temp.txt https://<website/directory> I am getting followin error in my... (9 Replies)
Discussion started by: pinnacle
9 Replies
IGREP(1)                                                      General Commands Manual                                                     IGREP(1)

NAME
igrep - search images for matching metadata SYNOPSIS
igrep [options] pattern files... DESCRIPTION
The igrep utility is invoked as follows: igrep [options] pattern filename... Where pattern is a POSIX.2 regular expression (just like the Unix/Linux grep(1) command), and filename (and any following names) specify images or directories that should be searched. An image file will "match" if any of its metadata contains values contain substring that are recognized regular expression. The image files may be of any format recognized by OpenImageIO (i.e., for which ImageInput plugins are available). Example: $ igrep Jack *.jpg bar.jpg: Keywords = Carly; Jack foo.jpg: Keywords = Jack test7.jpg: ImageDescription = Jack on vacation For a complete description, see /usr/share/doc/libopenimageio-dev/openimageio.pdf.gz. OPTIONS
--help Print help message -i Ignore upper/lower case distinctions -v Invert match (select non-matching files) -E Pattern is an extended regular expression -f Match against file name as well as metadata -l List the matching files (no detail) -r Recurse into directories -d Print directories (when recursive) -a Search all subimages of each file SEE ALSO iconvert(1), idiff(1), iinfo(1), iprocess(1), iv(1), maketx(1), oiiotool(1). AUTHOR
OpenImageIO was written by Larry Gritz and the other authors and contributors. This manual page was written by IRIE Shinsuke <irieshinsuke@yahoo.co.jp>, for the Debian project (and may be used by others). May 19, 2012 IGREP(1)
All times are GMT -4. The time now is 08:32 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy