FAQ how to download multiple specific files via command line


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting FAQ how to download multiple specific files via command line
Prev   Next
# 2  
Old 10-18-2011
wget url1 url2 url3 url4 ? robots.txt doesn't apply unless you're telling it to go recursively...

Last edited by Corona688; 10-18-2011 at 06:09 PM..
 
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Perl command line option '-n','-p' and multiple files: can it know a file name of a printed line?

I am looking for help in processing of those options: '-n' or '-p' I understand what they do and how to use them. But, I would like to use them with more than one file (and without any shell-loop; loading the 'perl' once.) I did try it and -n works on 2 files. Question is: - is it possible to... (6 Replies)
Discussion started by: alex_5161
6 Replies

2. Shell Programming and Scripting

Bash to tell download where specific files are stored

The bash below will download all the files in download to /home/Desktop/folder. That works great, but within /home/Desktop/folder there are several folders bam, other, and vcf, is there a way to specify by extention in the download file where to download it to? For example, all .pdf and .zip... (2 Replies)
Discussion started by: cmccabe
2 Replies

3. Shell Programming and Scripting

Curl command to download multiple files with a file prefix

I am using the below curl command to download a single file from client server and it is working as expected curl --ftp-ssl -k -u ${USER}:${PASSWD} ftp://${HOST}:${PORT}/path/to/${FILE} --output ${DEST}/${FILE} let say the client has 3 files hellofile.101, hellofile.102, hellofile.103 and I... (3 Replies)
Discussion started by: r@v!7*7@
3 Replies

4. Shell Programming and Scripting

Bash to download specific files and save in two folders

I am trying to download all files from a user authentication, password protected https site, with a particular extension (.bam). The files are ~20GB each and I am not sure if the below is the best way to do it. I am also not sure how to direct the downloaded files to a folder as well as external... (7 Replies)
Discussion started by: cmccabe
7 Replies

5. UNIX for Dummies Questions & Answers

Using awk to multiple and replace in a specific line

Hi Folks, I have the file in which I need to multiply the content of a line and replace the initial content of that line with the obtained answer. For example if this is my input file file1.txt 2.259314750 xxxxxx 1.962774350 xxxxxx 2.916817290 xxxxxx 1.355026900 ... (4 Replies)
Discussion started by: Madiouma Ndiaye
4 Replies

6. UNIX for Dummies Questions & Answers

How to download file protected by image captcha from command line?

Hello, any way to download file from image captcha download protected website? The download link is not static but session based, generated. I can do also via web browser, but i trust rather command line, maybe im wrong (1 Reply)
Discussion started by: postcd
1 Replies

7. Shell Programming and Scripting

Download multiple files uing wget

Need Assistance . Using wget how can i download multiple files from http site. Http doesnt has wild card (*) but FTP has it . Any ideas will be appreciative. wget --timeout=120 --append-output=output.txt --no-directories --cut-dirs=1 -np -m --accept=grib2 -r http://sample.com/... (4 Replies)
Discussion started by: ajayram_arya
4 Replies

8. Shell Programming and Scripting

sed command to replace a line at a specific line number with some other line

my requirement is, consider a file output cat output blah sdjfhjkd jsdfhjksdh sdfs 23423 sdfsdf sdf"sdfsdf"sdfsdf"""""dsf hellow there this doesnt look good et cetc etc etcetera i want to replace a line of line number 4 ("this doesnt look good") with some other line ... (3 Replies)
Discussion started by: vivek d r
3 Replies

9. Solaris

[ASK] Download with command line Solaris

hello all. I use wget to fetch and to do the download. But in solaris I can not use the command line to download because they do not have wget and fetch.please help me for this? Tnx (2 Replies)
Discussion started by: moslemovic
2 Replies

10. OS X (Apple)

Command line tool to join multiple .wmv files?

I need a simple command line executable that allows me to join many wmv files into one output wmv file, preferrably in a simple way like this: wmvjoin file1.wmv file2.wmv .... > outputfile.wmv So what I want is the wmv-equivalent of mpgtx I cannot find it on internet. Thanks. (2 Replies)
Discussion started by: karman
2 Replies
Login or Register to Ask a Question
MERGECAP(1)						  The Wireshark Network Analyzer					       MERGECAP(1)

NAME
mergecap - Merges two or more capture files into one SYNOPSIS
mergecap [ -a ] [ -F <file format> ] [ -h ] [ -s <snaplen> ] [ -T <encapsulation type> ] [ -v ] -w <outfile>|- <infile> ... DESCRIPTION
Mergecap is a program that combines multiple saved capture files into a single output file specified by the -w argument. Mergecap knows how to read libpcap capture files, including those of tcpdump, Wireshark, and other tools that write captures in that format. By default, Mergecap writes the capture file in libpcap format, and writes all of the packets from the input capture files to the output file. Mergecap is able to detect, read and write the same capture files that are supported by Wireshark. The input files don't need a specific filename extension; the file format and an optional gzip compression will be automatically detected. Near the beginning of the DESCRIPTION section of wireshark(1) or http://www.wireshark.org/docs/man-pages/wireshark.html <http://www.wireshark.org/docs/man-pages/wireshark.html> is a detailed description of the way Wireshark handles this, which is the same way Mergecap handles this. Mergecap can write the file in several output formats. The -F flag can be used to specify the format in which to write the capture file, mergecap -F provides a list of the available output formats. Packets from the input files are merged in chronological order based on each frame's timestamp, unless the -a flag is specified. Mergecap assumes that frames within a single capture file are already stored in chronological order. When the -a flag is specified, packets are copied directly from each input file to the output file, independent of each frame's timestamp. The output file frame encapsulation type is set to the type of the input files if all input files have the same type. If not all of the input files have the same frame encapsulation type, the output file type is set to WTAP_ENCAP_PER_PACKET. Note that some capture file formats, most notably libpcap, do not currently support WTAP_ENCAP_PER_PACKET. This combination will cause the output file creation to fail. OPTIONS
-a Causes the frame timestamps to be ignored, writing all packets from the first input file followed by all packets from the second input file. By default, when -a is not specified, the contents of the input files are merged in chronological order based on each frame's timestamp. Note: when merging, mergecap assumes that packets within a capture file are already in chronological order. -F <file format> Sets the file format of the output capture file. Mergecap can write the file in several formats; mergecap -F provides a list of the available output formats. The default is to use the file format of the first input file. -h Prints the version and options and exits. -s <snaplen> Sets the snapshot length to use when writing the data. If the -s flag is used to specify a snapshot length, frames in the input file with more captured data than the specified snapshot length will have only the amount of data specified by the snapshot length written to the output file. This may be useful if the program that is to read the output file cannot handle packets larger than a certain size (for example, the versions of snoop in Solaris 2.5.1 and Solaris 2.6 appear to reject Ethernet frames larger than the standard Ethernet MTU, making them incapable of handling gigabit Ethernet captures if jumbo frames were used). -v Causes mergecap to print a number of messages while it's working. -w <outfile>|- Sets the output filename. If the name is '-', stdout will be used. This setting is mandatory. -T <encapsulation type> Sets the packet encapsulation type of the output capture file. If the -T flag is used to specify a frame encapsulation type, the encapsulation type of the output capture file will be forced to the specified type, rather than being the type appropriate to the encapsulation type of the input capture files. Note that this merely forces the encapsulation type of the output file to be the specified type; the packet headers of the packets will not be translated from the encapsulation type of the input capture file to the specified encapsulation type (for example, it will not translate an Ethernet capture to an FDDI capture if an Ethernet capture is read and '-T fddi' is specified). SEE ALSO
tcpdump(8), pcap(3), wireshark(1), tshark(1), dumpcap(1), editcap(1), text2pcap(1) NOTES
Mergecap is based heavily upon editcap by Richard Sharpe <sharpe[AT]ns.aus.com> and Guy Harris <guy[AT]alum.mit.edu>. Mergecap is part of the Wireshark distribution. The latest version of Wireshark can be found at <http://www.wireshark.org>. HTML versions of the Wireshark project man pages are available at: http://www.wireshark.org/docs/man-pages <http://www.wireshark.org/docs/man-pages>. AUTHORS
Original Author -------- ------ Scott Renfro <scott[AT]renfro.org> Contributors ------------ Bill Guyton <guyton[AT]bguyton.com> 1.2.8 2010-05-05 MERGECAP(1)