Wget structure


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Wget structure
# 1  
Old 12-06-2012
Wget structure

Im reading a lot about wget and have really enjoyed using it. However I see a lot of tutorials that will show several ways. I want to be able to write scripts in gedit so I was wondering what is the best process to do this instead of just running them in the terminal??
# 2  
Old 12-07-2012
Wget is no different than any command in a script, you can test the results using the exit status $? (highly recommended, but move it into a variable ret=$? immediately as everything sets it) or grep in log files of stderr 2>logfile for magic words (hard to anticipate every fatal log message). The files it writes should have size [ -s data_file ], too. The idea of scripts and wget supporting them is unattended operation like from cron.
# 3  
Old 12-07-2012
is the proper way of writing .sh files still to declare #!/bin/bash in the header?

Ive noticed many files are
Code:
wget /
  procedure /
  procedure /
  procedure /

I would assume the backslash is required for every line?? Also I see wget has three ways to do it but I never see the short coding structure. IE: -X list but rather than --exclude list/ Can you shed light on this? Last how can you define what should go first? I know the address is always last but if you want a lot of rules how can you choose which to go with first?
# 4  
Old 12-07-2012
The #! thing is actually exec() not shell in particular. You can write your own file interpreter and get it called by scripts in your language with #!. See man execvp. The path must be absolute and you are allowed one argument, for commands like sed and awk that need a '-f to introduce a script not on stdin. Without the #!, text files are sent to stdin on sh. Execute permissions are a good idea, too.

You have to read bash scripts as bash, and wget and other external command calls in sh/ksh/bash are from the command to unquoted ';', '&', '<', '>', '#>' or end of line not escaped by '\'. The '&', '>', '<' or '#>' are directions to the shell on how to call the command. A bash tutorial might be a good read. There are many online. The bash man page is both long and insufficient in explanation for a novice, more a reference as you grow into it.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Wget - working in browser but cannot download from wget

Hi, I need to download a zip file from my the below US govt link. https://www.sam.gov/SAMPortal/extractfiledownload?role=WW&version=SAM&filename=SAM_PUBLIC_MONTHLY_20160207.ZIP I only have wget utility installed on the server. When I use the below command, I am getting error 403... (2 Replies)
Discussion started by: Prasannag87
2 Replies

2. Shell Programming and Scripting

wget a directory structure question

Can you tell me how to download the directory tree just starting from "project1/" in this URL? "https://somesite.com/projects/t/project1/" This command does not seem to do what I want as it downloads also files from the upper hierarchy: wget --no-check-certificate --http-user=user... (4 Replies)
Discussion started by: majormark
4 Replies

3. UNIX for Dummies Questions & Answers

Directory Structure

Hi... I have a directory which has multiple directories and sub directories inside... what command should i use to get a list of all these directories, without the filenames.... (2 Replies)
Discussion started by: saharookiedba
2 Replies

4. UNIX for Dummies Questions & Answers

Size of Structure

How can we find size of a structure with out using sizeof operator? Thanks, Harika (2 Replies)
Discussion started by: harikamamidala
2 Replies

5. Programming

Search attributes in one structure using the values from another structure

Hello Groups I am trying to find out ways of comparing a value from a 'c' structure to a value in another 'C' structure. the 'C' structure can be a List or liked list as it contains lot many records. if we loop it in both the structures it is going to consume time. I am looking for a simple... (3 Replies)
Discussion started by: dhanamurthy
3 Replies

6. Shell Programming and Scripting

file structure

Hi, Is there a command that can help me to determain a unix file structure ? Thanks (7 Replies)
Discussion started by: yoavbe
7 Replies

7. Shell Programming and Scripting

Case structure

Hi, Can anyone help me with the following case structure? echo "Please enter the date for which you want the Report:\c" read dat d1=`echo $dat|cut -c7-8` m1=`echo $dat|cut -c5-6` y1=`echo $dat|cut -c1-4` yr=`expr "$d1" - 1` case "$yr" in 0) MONTH=`expr "$m1" - 1`... (4 Replies)
Discussion started by: kamitsin
4 Replies

8. UNIX for Advanced & Expert Users

MV files from one directory structure(multiple level) to other directory structure

Hi, I am trying to write a script that will move all the files from source directory structure(multiple levels might exist) to destination directory structure. If a sub folder is source doesnot exist in destination then I have to skip and goto next level. I also need to delete the files in... (4 Replies)
Discussion started by: srmadab
4 Replies

9. UNIX for Dummies Questions & Answers

Copying a Directory Structure to a new structure

Hi all Is it possible to copy a structure of a directory only. e.g. I have a file with the following entries that is a result of a find :- /dir1/dir2/file.dbf /dir1/dir2/dir3/file1.dbf /dir1/file.dbf I want to copy these to a directory and keep the structure however starting at a new dir... (8 Replies)
Discussion started by: jhansrod
8 Replies

10. UNIX for Dummies Questions & Answers

if then else structure

echo name the file that you want to read read answer if then echo you must enter a file name fi cat $answer im trying to catch the error if user forget to enter the name of the file anyone can help me ? thanks:confused: (4 Replies)
Discussion started by: props
4 Replies
Login or Register to Ask a Question