Sponsored Content
Top Forums Shell Programming and Scripting Downloading jpgs from a gallery type website Post 302899538 by workisnotfun on Tuesday 29th of April 2014 05:39:08 PM
Old 04-29-2014
Downloading jpgs from a gallery type website

Can someone explain what this does step by step? I found this script on stackoverflow and want to customize it for personal use in downloading jpg images from a website.

Code:
# get all pages 
curl 'http://domain.com/id/[1-151468]' -o '#1.html' 

# get all images 
grep -oh 'http://pics.domain.com/pics/original/.*jpg' *.html >urls.txt 

# download all images 
sort -u urls.txt | wget -i-

1. What I think the first like does is download the pages of domain with curl but what does the '#1.html' mean?

2. Why in .*jpg is the * after the '.'? Also what is this trying to do? I attempted altering this using a different website but there's an error grep: *.html: No such file or directory even though the first command is downloading the html files just fine.

3. I think the third option is just organizing the results and wget goes to the jpg's website and downloads the jpgs.

Moderator's Comments:
Mod Comment Code tags for code, please.

Last edited by Corona688; 04-29-2014 at 06:47 PM..
 

8 More Discussions You Might Find Interesting

1. UNIX Desktop Questions & Answers

Recommendations for good shell utility to resize JPGs?

Not sure if this is the right place to be posting this. If not, let me know where it fits. I am running RedHat Linux 8.0. I've recently acquired a Sony Mavica digital camera. In short, this thing is awesome (uses cd-rw!). I have been taking high quality images, but I now have the need to try... (2 Replies)
Discussion started by: deckard
2 Replies

2. UNIX for Dummies Questions & Answers

Lynx - Downloading - extension handling - changing mime type?

Using Lynx, when I try to download a .rar, it confirms I want to download and its got it as an appication/rar file. However, split archives that end in .r## (.r00, .r01 ...) are not recognized as an appication/rar file and it reads the file like a .txt or .html. How can I fix this? Thanks! (2 Replies)
Discussion started by: yitzle
2 Replies

3. Shell Programming and Scripting

String type to date type

Can one string type variable changed into the date type variable. (1 Reply)
Discussion started by: rinku
1 Replies

4. Programming

array type has incomplete element type

Dear colleagues, One of my friend have a problem with c code. While compiling a c program it displays a message like "array type has incomplete element type". Any body can provide a solution for it. Jaganadh.G (1 Reply)
Discussion started by: jaganadh
1 Replies

5. Shell Programming and Scripting

Downloading info from website to database

Hi guys! I created a database using mysql in bash now i would like to download weather info from the data.(temp, date and time)...and just store this in the database to display after every 3 hours or so... i have tried to get the website using wget and now dont exactly now how to go from here... (0 Replies)
Discussion started by: vadharah
0 Replies

6. Shell Programming and Scripting

File Management: How do I move all JPGS in a folder structure to a single folder?

This is the file structure: DESKTOP/Root of Photo Folders/Folder1qweqwasdfsd/*jpg DESKTOP/Root of Photo Folders/Folder2asdasdasd/*jpg DESKTOP/Root of Photo Folders/Folder3asdadfhgasdf/*jpg DESKTOP/Root of Photo Folders/Folder4qwetwdfsdfg/*jpg DESKTOP/Root of Photo... (4 Replies)
Discussion started by: guptaxpn
4 Replies

7. Windows & DOS: Issues & Discussions

Downloading a file from Website to a Windows Folder

Hi, Is it possible to download a file using Wget or some other command from a Windows machine? Say I want to download something from https server to C:\ABC\abc.xls Any ideas, Thanks. (4 Replies)
Discussion started by: dohko
4 Replies

8. Shell Programming and Scripting

Wget error while downloading from https website

Hi, I would like to download a file from a https website. I don't have the file name as it changes every day. I am using the following command: wget --no-check-certificate -r -np --user=ABC --password=DEF -O temp.txt https://<website/directory> I am getting followin error in my... (9 Replies)
Discussion started by: pinnacle
9 Replies
PUZZLE-DIFF(1)															    PUZZLE-DIFF(1)

NAME
puzzle-diff - Compare pictures with libpuzzle SYNOPSIS
[-b <contrast barrier for cropping>] [-c] [-C <max cropping ratio>] [-e] [-E <similarity threshold>] [-h] [-H <max height>] [-l <lambdas>] [-n <noise cutoff>] [-p <p ratio>] [-t] [-W <max width>] <file 1> <file 2> DESCRIPTION
puzzle-diff compares two pictures and outputs the normalized distance. Try puzzle-diff -h for more info. EXAMPLES
Output distance between two images: $ puzzle-diff pic-a-0.jpg pics-a-1.jpg 0.102286 Compare two images, exit with 10 if they look the same, exit with 20 if they don't (may be useful for scripts): $ puzzle-diff -e pic-a-0.jpg pics-a-1.jpg $ echo $? 10 Compute distance, without cropping and with computing the average intensity of the whole blocks: $ puzzle-diff -p 1.0 -c pic-a-0.jpg pic-a-1.jpg 0.0523151 AUTHORS
Frank DENIS libpuzzle at pureftpd dot org SEE ALSO
libpuzzle(3), puzzle_set(3) 2012-05-09 PUZZLE-DIFF(1)
All times are GMT -4. The time now is 03:52 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy