Sponsored Content
Top Forums Shell Programming and Scripting Downloading jpgs from a gallery type website Post 302899538 by workisnotfun on Tuesday 29th of April 2014 05:39:08 PM
Old 04-29-2014
Downloading jpgs from a gallery type website

Can someone explain what this does step by step? I found this script on stackoverflow and want to customize it for personal use in downloading jpg images from a website.

Code:
# get all pages 
curl 'http://domain.com/id/[1-151468]' -o '#1.html' 

# get all images 
grep -oh 'http://pics.domain.com/pics/original/.*jpg' *.html >urls.txt 

# download all images 
sort -u urls.txt | wget -i-

1. What I think the first like does is download the pages of domain with curl but what does the '#1.html' mean?

2. Why in .*jpg is the * after the '.'? Also what is this trying to do? I attempted altering this using a different website but there's an error grep: *.html: No such file or directory even though the first command is downloading the html files just fine.

3. I think the third option is just organizing the results and wget goes to the jpg's website and downloads the jpgs.

Moderator's Comments:
Mod Comment Code tags for code, please.

Last edited by Corona688; 04-29-2014 at 06:47 PM..
 

8 More Discussions You Might Find Interesting

1. UNIX Desktop Questions & Answers

Recommendations for good shell utility to resize JPGs?

Not sure if this is the right place to be posting this. If not, let me know where it fits. I am running RedHat Linux 8.0. I've recently acquired a Sony Mavica digital camera. In short, this thing is awesome (uses cd-rw!). I have been taking high quality images, but I now have the need to try... (2 Replies)
Discussion started by: deckard
2 Replies

2. UNIX for Dummies Questions & Answers

Lynx - Downloading - extension handling - changing mime type?

Using Lynx, when I try to download a .rar, it confirms I want to download and its got it as an appication/rar file. However, split archives that end in .r## (.r00, .r01 ...) are not recognized as an appication/rar file and it reads the file like a .txt or .html. How can I fix this? Thanks! (2 Replies)
Discussion started by: yitzle
2 Replies

3. Shell Programming and Scripting

String type to date type

Can one string type variable changed into the date type variable. (1 Reply)
Discussion started by: rinku
1 Replies

4. Programming

array type has incomplete element type

Dear colleagues, One of my friend have a problem with c code. While compiling a c program it displays a message like "array type has incomplete element type". Any body can provide a solution for it. Jaganadh.G (1 Reply)
Discussion started by: jaganadh
1 Replies

5. Shell Programming and Scripting

Downloading info from website to database

Hi guys! I created a database using mysql in bash now i would like to download weather info from the data.(temp, date and time)...and just store this in the database to display after every 3 hours or so... i have tried to get the website using wget and now dont exactly now how to go from here... (0 Replies)
Discussion started by: vadharah
0 Replies

6. Shell Programming and Scripting

File Management: How do I move all JPGS in a folder structure to a single folder?

This is the file structure: DESKTOP/Root of Photo Folders/Folder1qweqwasdfsd/*jpg DESKTOP/Root of Photo Folders/Folder2asdasdasd/*jpg DESKTOP/Root of Photo Folders/Folder3asdadfhgasdf/*jpg DESKTOP/Root of Photo Folders/Folder4qwetwdfsdfg/*jpg DESKTOP/Root of Photo... (4 Replies)
Discussion started by: guptaxpn
4 Replies

7. Windows & DOS: Issues & Discussions

Downloading a file from Website to a Windows Folder

Hi, Is it possible to download a file using Wget or some other command from a Windows machine? Say I want to download something from https server to C:\ABC\abc.xls Any ideas, Thanks. (4 Replies)
Discussion started by: dohko
4 Replies

8. Shell Programming and Scripting

Wget error while downloading from https website

Hi, I would like to download a file from a https website. I don't have the file name as it changes every day. I am using the following command: wget --no-check-certificate -r -np --user=ABC --password=DEF -O temp.txt https://<website/directory> I am getting followin error in my... (9 Replies)
Discussion started by: pinnacle
9 Replies
QVIDEOOB(1)						      General Commands Manual						       QVIDEOOB(1)

NAME
qvideoob - Qt application to search and play videos on various websites SYNOPSIS
qvideoob [-h] [-dqv] [-b backends] ... qvideoob [--help] [--version] DESCRIPTION
Qt application allowing to search videos on various websites and play them. Supported websites: * arte (Arte French and German TV) * canalplus (Canal Plus French TV) * cappedtv (Capped.tv demoscene website) * dailymotion (Dailymotion video streaming website) * francetelevisions (France Televisions video website) * ina (INA French TV video archives) * nolifetv (NolifeTV French video streaming website) * radiofrance (Radios of Radio France: Inter, Info, Bleu, Culture, Musique, FIP, Le Mouv') * youjizz (YouJizz pornographic video streaming website) * youporn (YouPorn pornographic video streaming website) * youtube (YouTube video streaming website) OPTIONS
--version show program's version number and exit -h, --help show this help message and exit -b BACKENDS, --backends=BACKENDS what backend(s) to enable (comma separated) LOGGING OPTIONS
-d, --debug display debug messages -q, --quiet display only error messages -v, --verbose display info messages --logging-file=LOGGING_FILE file to save logs -a, --save-responses save every response COPYRIGHT
Copyright(C) 2010-2011 Romain Bignon For full COPYRIGHT see COPYING file with weboob package. FILES
"~/.config/weboob/backends" "~/.config/weboob/qvideoob" SEE ALSO
Home page: http://weboob.org/applications/qvideoob qvideoob 0.c 04 May 2012 QVIDEOOB(1)
All times are GMT -4. The time now is 05:27 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy