Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Store filenames for wget in bash
Top Forums Shell Programming and Scripting Store filenames for wget in bash Post 302966201 by sea on Tuesday 9th of February 2016 10:49:11 AM
Old 02-09-2016
Try either one or both:
Code:
# Method one: list contains only filenames
while read F;do
	wget -q --user=xxxx --password=xxxxx --xxxxx \
		https://www.xxxx.com/$F
done<<~/list.txt

# Method two: list contains full url
wget -q --user=xxxx --password=xxxxx --xxxxx -i list.txt

hth
This User Gave Thanks to sea For This Post:
cmccabe
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How to script wget in bash?

The script below is giving me grief! The error message says /download.bash: line 16: syntax error near unexpected token `else' ./download.bash: line 16: `else wget "http://downloads.sourceforge.net/hibernate/hibernate-3.2.5.ga.zip?modtime=1185893922&big_mirror=1" ' I think it must be a... (1 Reply)
Discussion started by: siegfried
1 Replies

2. Shell Programming and Scripting

bash: reading filenames from file

Hi, I'm trying to write a script that reads filenames from a file and use these filenames in a loop. The filenames are all on one line and the problem is that these filenames have wildcards like * and braces like in them. Right now what I'm doing is something like this: echo "reading from... (0 Replies)
Discussion started by: warp17
0 Replies

3. Shell Programming and Scripting

Whitespace in filenames in for loop in bash script

I'm trying to search all .odt files in a directory for a string in the text of the file. I've found a bash script that works, except that it can't handle whitespace in the filenames. #!/bin/bash if ; then echo "Usage: searchodt searchterm" exit 1 fi for file in $(ls *.odt); do ... (4 Replies)
Discussion started by: triplemaya
4 Replies

4. Shell Programming and Scripting

Bash script - stripping away characters that can't be used in filenames

I want to create a temp file which is named based on a search string. The search string may contain spaces or characters that aren't supposed to be used in filenames so I want to strip those out. My thought was to use 'tr' with but the result is the opposite of what I want: $ echo "test... (5 Replies)
Discussion started by: mglenney
5 Replies

5. Shell Programming and Scripting

New file should store all the 7 existing filenames and their record counts and ftp th

Hi, I need help regarding below concern. There is a script and it has 7 existing files(in a path say,. usr/appl/temp/file1.txt) and I need to create one new blank file say “file_count.txt” in the same script itself. Then the new file <file_count.txt> should store all the 7 filenames and... (1 Reply)
Discussion started by: pr293
1 Replies

6. IP Networking

Using Apache2 to Store Files for Accessing with wget?

Hello All, I have a Virtual Machine that I basically use for just testing stuff on. It is running SLES 11.1 and Apache2. I was able to get Apache2 set-up and working... I was able to insert a basic index.html page (i.e. a simple "Hello World" html page) just to check and make sure I can... (4 Replies)
Discussion started by: mrm5102
4 Replies

7. Shell Programming and Scripting

How to take the filenames from a directory and store into a file??

hi, how can i take the file names from a directory and store only the filenames in the file. suppose i have a directory which contains the following files and subdirectories. $ ls -ltr total 16 -rw-rw-r-- 1 adm etc 4 Aug 6 20:37 s1.txt -rw-rw-r-- 1 adm etc 4 Aug 6 20:37 s2.txt... (11 Replies)
Discussion started by: Little
11 Replies

8. Shell Programming and Scripting

Wget in bash

I am attempting to write a bash that starts by using wget and getting the following errors: Stand-alone code that works: wget -O getCSV.txt http://172.24.xxx.xxx/data/getCSV.csv c:\cygwin\home\cmccabe\NGS.sh: line 2: $'\r': command not found : No such file or directorysh: line 3:... (2 Replies)
Discussion started by: cmccabe
2 Replies

9. Shell Programming and Scripting

Wget in bash using sed and awk

In the bash below when the program is opened the download function runs and downloads the getCSV file and on the screen "Downloading getCSV.csv:%" displays and when it completes the menu function is called. However, as of now the bash opens and closes after a few seconds and I'm not sure... (4 Replies)
Discussion started by: cmccabe
4 Replies

10. Shell Programming and Scripting

Bash: Picking up filenames then moving to next directory

I have a collection of directories, for example as below I want to create a loop that goes in the first directory and picks up the *hhz*.sac.pzs filename in a variable, and the other files matching *hhz*.sac in another variable (however I do not want to pick the *hhz*.sac.pzs). This is because... (4 Replies)
Discussion started by: kristinu
4 Replies

LEARN ABOUT REDHAT

dget

DGET(1) 																   DGET(1)

NAME

       dget -- Download Debian source and binary packages

SYNOPSIS

       dget [options] URL ...
       dget [options] package[=version]

DESCRIPTION

       dget downloads Debian packages.	In the first form, dget fetches the requested URLs.  If this is a .dsc or .changes file, then dget acts as
       a source-package aware form of wget: it also fetches any files referenced in the .dsc/.changes file.  The downloaded source is then checked
       with dscverify and, if successful, unpacked by dpkg-source.

       In the second form, dget downloads a binary package (i.e., a .deb file) from the Debian mirror configured in /etc/apt/sources.list(.d).
       Unlike apt-get install -d, it does not require root privileges, writes to the current directory, and does not download dependencies.  If a
       version number is specified, this version of the package is requested.

       In both cases dget is capable of getting several packages and/or URLs at once.

       (Note that .udeb packages used by debian-installer are located in separate packages files from .deb packages. In order to use .udebs with
       dget, you will need to have configured apt to use a packages file for component/debian-installer).

       Before downloading files listed in .dsc and .changes files, and before downloading binary packages, dget checks to see whether any of these
       files already exist.  If they do, then their md5sums are compared to avoid downloading them again unnecessarily.  dget also looks for
       matching files in /var/cache/apt/archives and directories given by the --path option or specified in the configuration files (see below).
       Finally, if downloading (.orig).tar.gz or .diff.gz files fails, dget consults apt-get source --print-uris.  Download backends used are curl
       and wget, looked for in that order.

       dget was written to make it easier to retrieve source packages from the web for sponsor uploads.  For checking the package with debdiff,
       the last binary version is available via dget package, the last source version via apt-get source package.

OPTIONS

       -b, --backup
	   Move files that would be overwritten to ./backup.

       -q, --quiet
	   Suppress wget/curl non-error output.

       -d, --download-only
	   Do not run dpkg-source -x on the downloaded source package.	This can only be used with the first method of calling dget.

       -x, --extract
	   Run dpkg-source -x on the downloaded source package to unpack it.  This option is the default and can only be used with the first
	   method of calling dget.

       -u, --allow-unauthenticated
	   Do not attempt to verify the integrity of downloaded source packages using dscverify.

       --build
	   Run dpkg-buildpackage -b -uc on the downloaded source package.

       --path DIR[:DIR ...]
	   In addition to /var/cache/apt/archives, dget uses the colon-separated list given as argument to --path to find files with a matching
	   md5sum.  For example: "--path /srv/pbuilder/result:/home/cb/UploadQueue".  If DIR is empty (i.e., "--path ''" is specified), then any
	   previously listed directories or directories specified in the configuration files will be ignored.  This option may be specified
	   multiple times, and all of the directories listed will be searched; hence, the above example could have been written as: "--path
	   /srv/pbuilder/result --path /home/cb/UploadQueue".

       --insecure
	   Allow SSL connections to untrusted hosts.

       --no-cache
	   Bypass server-side HTTP caches by sending a Pragma: no-cache header.

       -h, --help
	   Show a help message.

       -V, --version
	   Show version information.

CONFIGURATION VARIABLES

       The two configuration files /etc/devscripts.conf and ~/.devscripts are sourced by a shell in that order to set configuration variables.
       Command line options can be used to override configuration file settings.  Environment variable settings are ignored for this purpose.  The
       currently recognised variable is:

       DGET_PATH
	   This can be set to a colon-separated list of directories in which to search for files in addition to the default
	   /var/cache/apt/archives.  It has the same effect as the --path command line option.	It is not set by default.

       DGET_UNPACK
	   Set to 'no' to disable extracting downloaded source packages.  Default is 'yes'.

       DGET_VERIFY
	   Set to 'no' to disable checking signatures of downloaded source packages.  Default is 'yes'.

BUGS AND COMPATIBILITY

       dget package should be implemented in apt-get install -d.

       Before devscripts version 2.10.17, the default was not to extract the downloaded source. Set DGET_UNPACK=no to revert to the old behaviour.

AUTHOR

       This program is Copyright (C) 2005-08 by Christoph Berg <myon@debian.org>.  Modifications are Copyright (C) 2005-06 by Julian Gilbey
       <jdg@debian.org>.

       This program is licensed under the terms of the GPL, either version 2 of the License, or (at your option) any later version.

SEE ALSO

       apt-get(1), debcheckout(1), debdiff(1), dpkg-source(1), curl(1), wget(1).

Debian Utilities						    2013-12-23								   DGET(1)
All times are GMT -4. The time now is 04:07 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy