Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Wget from multiple paths
Top Forums Shell Programming and Scripting Wget from multiple paths Post 302938005 by Devyn on Wednesday 11th of March 2015 10:05:21 AM
Old 03-11-2015
Wget from multiple paths
if I have these wildcards to download from:

Code:
path1/*.txt
path2/*.txt
path3/*.txt
path4/*.txt
path5/*.txt

under a link such as this:

Code:
http://abc .com/]abc.com

Can wget be written in such a way to extract only those files and create the corresponding paths under a target folder? So the result looks like this:

Code:
/downloads/path1/*.txt
...
/downloads/path5/*.txt

I just need to download the above in that specific pattern and folder structure.

Cheers
DH

---------- Post updated at 10:05 AM ---------- Previous update was at 10:02 AM ----------

I pasted abc.com and it interpreted the link (lol). was just an example.


hehehe
Last edited by Don Cragun; 05-05-2015 at 03:05 AM.. Reason: Add CODE and ICODE tags.
 

10 More Discussions You Might Find Interesting

1. Filesystems, Disks and Memory

Multiple Paths to SAN with LVM in Linux?

I'm setting up a CentOS 5 server that will be connected to an HP EVA4000. My previous experience is with an HP VA7400 and HP-UX. In HP-UX I can add "alternate paths" to a volume group in order to have redundant paths to the SAN via dual fiber channel HBAs just by either adding them with... (3 Replies)
Discussion started by: deckard
3 Replies

2. Shell Programming and Scripting

Enabling a script to run in multiple paths

I have a script that i need to run from different paths. for example mypc/path1/path2/, mypc/path1/path2/path3/, and mypc/path1/path2/path3/path4 How do i set up that script so that it can execute in any of the above paths or how can i make it run on any path on my computer?? (5 Replies)
Discussion started by: dowell
5 Replies

3. Shell Programming and Scripting

find multiple paths

How do i find files in more than one directory? I searched through forums, but could not land into the right thread. I tried something like find dir1|dir2 -name file1 but it doesn't work. Please suggest. (5 Replies)
Discussion started by: krishmaths
5 Replies

4. Shell Programming and Scripting

KSH - Find paths of multiple files in CC (dir and sub-dir))

Dear Members, I have a list of xml files like abc.xml.table prq.xml.table ... .. . in a txt file. Now I have to search the file(s) in all directories and sub-directories and print the full path of file in a output txt file. Please help me with the script or command to do so. ... (11 Replies)
Discussion started by: Yoodit
11 Replies

5. Shell Programming and Scripting

Replace directory paths in multiple files at once

I need to update about 2400 files in a directory subtree, with a new directory path inside the files I need to change this occurence in all files: /d2/R12AB/VIS/apps/tech_st/10.1.2 with this: /u01/PROD/apps/apps_st/10.1.3 I know how to change single words using "find . -type f -print0 |... (6 Replies)
Discussion started by: wicus
6 Replies

6. Shell Programming and Scripting

Script to delete files older than x days and also taking an input for multiple paths

Hi , I am a newbie!!! I want to develop a script for deleting files older than x days from multiple paths. Now I could reach upto this piece of code which deletes files older than x days from a particular path. How do I enhance it to have an input from a .txt file or a .dat file? For eg:... (12 Replies)
Discussion started by: jhilmil
12 Replies

7. Shell Programming and Scripting

Storing multiple file paths in a variable

I am working on a script for Mac OS X that, among many other things, gets a list of all the installed Applications. I am pulling the list from the system_profiler command and formatting it using grep and awk. The problem is that I want to be able to use each result individually later in the script.... (3 Replies)
Discussion started by: cranfordio
3 Replies

8. Shell Programming and Scripting

Download multiple files uing wget

Need Assistance . Using wget how can i download multiple files from http site. Http doesnt has wild card (*) but FTP has it . Any ideas will be appreciative. wget --timeout=120 --append-output=output.txt --no-directories --cut-dirs=1 -np -m --accept=grib2 -r http://sample.com/... (4 Replies)
Discussion started by: ajayram_arya
4 Replies

9. Shell Programming and Scripting

Wget to download multiple source code

Can a modified command be used to download multiple source codes from specific sites and output each into a separate output file?. All the sites are in a text file (attached): wget -qO- http://www.genedx.com/test-catalog/available-tests/edar-gene-sequencing/ | cat > output.txt (4 Replies)
Discussion started by: cmccabe
4 Replies

10. Shell Programming and Scripting

Wget - working in browser but cannot download from wget

Hi, I need to download a zip file from my the below US govt link. https://www.sam.gov/SAMPortal/extractfiledownload?role=WW&version=SAM&filename=SAM_PUBLIC_MONTHLY_20160207.ZIP I only have wget utility installed on the server. When I use the below command, I am getting error 403... (2 Replies)
Discussion started by: Prasannag87
2 Replies

LEARN ABOUT REDHAT

unix2dos

unix2dos(1)						      General Commands Manual						       unix2dos(1)

NAME

       unix2dos - UNIX to DOS text file format converter

SYNOPSYS

       unix2dos [options] [-c convmode] [-o file ...] [-n infile outfile ...]

       Options:

       [-hkqV] [--help] [--keepdate] [--quiet] [--version]

DESCRIPTION

       This manual page documents unix2dos, the program that converts text files in UNIX format to DOS format.

OPTIONS

       The following options are available:

       -h --help
	      Print online help.

       -k --keepdate
	      Keep the date stamp of output file same as input file.

       -q --quiet
	      Quiet mode. Suppress all warning and messages.

       -V --version
	      Prints version information.

       -c --convmode convmode
	      Sets conversion mode. Simulates unix2dos under SunOS.

       -o --oldfile file ...
	      Old file mode. Convert the file and write output to it. The program default to run in this mode. Wildcard names may be used.

       -n --newfile infile outfile ...
	      New  file  mode.	Convert the infile and write output to outfile. File names must be given in pairs and wildcard names should NOT be
	      used or you WILL lost your files.

EXAMPLES

       Get input from stdin and write output to stdout.

	      unix2dos

       Convert and replace a.txt. Convert and replace b.txt.

	      unix2dos a.txt b.txt

	      unix2dos -o a.txt b.txt

       Convert and replace a.txt in ASCII conversion mode.  Convert and replace b.txt in ISO conversion mode.

	      unix2dos a.txt -c iso b.txt

	      unix2dos -c ascii a.txt -c iso b.txt

       Convert and replace a.txt while keeping original date stamp.

	      unix2dos -k a.txt

	      unix2dos -k -o a.txt

       Convert a.txt and write to e.txt.

	      unix2dos -n a.txt e.txt

       Convert a.txt and write to e.txt, keep date stamp of e.txt same as a.txt.

	      unix2dos -k -n a.txt e.txt

       Convert and replace a.txt. Convert b.txt and write to e.txt.

	      unix2dos a.txt -n b.txt e.txt

	      unix2dos -o a.txt -n b.txt e.txt

       Convert c.txt and write to e.txt. Convert and replace a.txt.  Convert and replace b.txt. Convert d.txt and write to f.txt.

	      unix2dos -n c.txt e.txt -o a.txt b.txt -n d.txt f.txt

DIAGNOSTICS

BUGS

       The program does not work properly under MSDOS in stdio processing mode.  If you know why is that so, please tell me.

AUTHOR

       Benjamin Lin - ( blin@socs.uts.edu.au )

MISCELLANY

       Tested environment:

	      Linux 1.2.0 with GNU C 2.5.8

	      SunOS 4.1.3 with GNU C 2.6.3

	      MS-DOS 6.20 with Borland C++ 4.02

       Suggestions and bug reports are welcome.

SEE ALSO

       dos2unix(1)

1995.03.31							   unix2dos v2.2						       unix2dos(1)
All times are GMT -4. The time now is 05:37 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy