Sponsored Content
Top Forums Shell Programming and Scripting Wget - how to ignore files in immediate directory? Post 302906014 by vanessafan99 on Monday 16th of June 2014 04:16:42 PM
Old 06-16-2014
Wget - how to ignore files in immediate directory?

i am trying to recursively save a remote FTP server but exclude the files immediately under a directory directory1

Code:
wget -r -N ftp://user:pass@hostname/directory1

I want to keep these which may have more files under them
Code:
directory1/dir1/file.jpg
directory1/dir2/file.jpg
directory1/dir3/file.jpg

but I want to exclude all the files directory under directory1, as these are the unsorted files

Code:
directory1/file1.jpg
directory1/file2.jpg


How can I exclude all the files immediately under directory1 but keep all other files?

thanks!
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How to ignore '.' files

I'm running Fedora Core 6 as an FTP server on a powerMac G4... I'm trying to create a script to remove files older than 3 days... I'm able to find all data older than 3 days but it finds hidden files such as /home/ftp/goossens/.canna /home/ftp/goossens/.kde... (4 Replies)
Discussion started by: James_UK
4 Replies

2. Solaris

How to ignore incomplete files

On Solaris, suppose there is a directory 'dir'. Log files of size approx 1MB are continuously being deposited here by scp command. I have a script that scans this dir every 5 mins and moves away the log files that have been deposited so far. How do I design my script so that I pick up *only*... (6 Replies)
Discussion started by: sentak
6 Replies

3. UNIX for Advanced & Expert Users

Why is wget copying my directory tree with some files with "@"?

I'm using wget 1.11.4 on Cygwin 1.5.25. I'm trying to recursively download a directory tree, which is the root of a javadoc tree. This is approximately the command line I tried: wget -x -p -r http://<host>/.../apidoc When it finished, it seemed like it downloaded... (0 Replies)
Discussion started by: dkarr
0 Replies

4. Shell Programming and Scripting

wget a directory structure question

Can you tell me how to download the directory tree just starting from "project1/" in this URL? "https://somesite.com/projects/t/project1/" This command does not seem to do what I want as it downloads also files from the upper hierarchy: wget --no-check-certificate --http-user=user... (4 Replies)
Discussion started by: majormark
4 Replies

5. Shell Programming and Scripting

Getting ls to ignore ~ and # files

Is there a way to customize ls to ignore files ending with ~ and #? (those are Emacs backup and auto-save files). I found -B option, which only ignores ~ files (2 Replies)
Discussion started by: yaroslavvb
2 Replies

6. Shell Programming and Scripting

Find: ignore directory completely

Hello, I know find can be prevented from recursing into directories with something like the following... find . -name .svn -prune -a type d But how can I completely prevent directories of a certain name (.svn) from being displayed at all, the top level and the children? I really... (2 Replies)
Discussion started by: nwb123
2 Replies

7. Shell Programming and Scripting

Wget to ignore an IP address

Hello Unix Geeks, I am in a situation to use wget for crawling a site where the site contains 5 IP addresses. Out of 5, 4 are accessible and 1 is having a problem due to firewall problems. In this case, my wget is getting stuck with that X.X.X.X and giving up. How can I ignore this IP and... (4 Replies)
Discussion started by: sathyaonnuix
4 Replies

8. Shell Programming and Scripting

Find command with ignore directory

Dear All, I am using find command find /my_rep/*/RKYPROOF/*/*/WDM/HOME_INT/PWD_DATA -name rk*myguidelines*.pdf -print The problem i am facing here is find /my_rep/*/ the directory after my_rep could be mice001, mice002 and mice001_PO, mice002_PO i want to ignore mice***_PO directory... (3 Replies)
Discussion started by: yadavricky
3 Replies

9. Shell Programming and Scripting

How to change wget download directory?

i have a cron that mirrors a site periodically wget -r -nc --passive-ftp ftp://user:pass@123.456.789.0 i want to download this into a directory called /files but when I do this, it always create a new directory called "123.456.789.0" (the hostname) it puts it into /files/123.456.789.0 but... (3 Replies)
Discussion started by: vanessafan99
3 Replies

10. UNIX for Advanced & Expert Users

AIX find ignore directory

I am using aix. I would like to ignore the /u directory. I tried this but it is not working. find / -type f -type d \( -path /u \) -prune -o -name '*rpm*' 2>/dev/null /u/appx/ls.rpm /u/arch/vim.rpm (4 Replies)
Discussion started by: cokedude
4 Replies
kitcap(5)							File Formats Manual							 kitcap(5)

Name
       kitcap - kit descriptor database for gentape and genra utilities.

Description
       The  file is a database for kit descriptors containing product codes, directories, files, and subsets that make up a product description to
       be used by or to create distribution media.  All fields are separated by colons (:) with a backslash () at the end of  a  line	indicating
       continuation.   Lines  starting	with a number sign (#) are considered comments and are ignored.  Comment fields with a kitcode description
       are delimited by an opening number sign (#) and a closing colon (:).

       The following entry examples are for TK50 and MT9 media types:
       Product-codeTK | Product Description:directory1:directory2:directory3:
	    SPACE:SPACE:SPACE:INSTCTRL:subset1:subset2:subset3:subset4:subset5

       Product-codeMT | Product Description:directory1:directory2:directory3:
	    SPACE:SPACE:SPACE:INSTCTRL:subset1:subset2:subset3:subset4:subset5:
	    %%2:
	    subset6:subset7:subset8:subset9:subset10

       The following parts make up the descriptor for magnetic tape media:

       Product-code
	      This is an arbitrary name made up of letters and/or numbers unique to the product that it describes.  Typical codes include a  prod-
	      uct identifier and a version identifier, as indicated in the previous examples.

       Media-code (TK or MT)
	      The media-code is a 2 letter reference that describes the type of media the files will be written to.  The media code must be either
	      TK for TK50 or MT for 9-track magnetic tape devices.  During run time, the utility probes the device to be written to and determines
	      if  it is a TK50- or MT9-type device.  It then appends either TK or MT to the kitcode given on the command line and searches for the
	      kitcode (product-code/media-code) in the kitcap file.

       Product Description
	      This field is a description of the software product that is being created by the utility and replaces the NAME field in the file	of
	      all the subsets that make up a product.  This is an optional field for magnetic tape media.

       Directories
	      The magnetic tape media production utility has the ability of producing multi-product tapes.  That is, it can take subsets from dif-
	      ferent products that are based in different directories and merge them together to form a third product, which is a  combination	of
	      the  original  products.	Directory entries provide the full path locations of where the subsets that are to be put on media will be
	      stored.  There must be at least one directory entry for each kitcap descriptor.

       SPACE
	      The SPACE file is a place holder for tape records composed of 1k of NULL characters. Three SPACE files are used as  dummy  files	to
	      ensure compatibility with ULTRIX operating system kits.

       INSTCTRL
	      The INSTCTRL image contains setld control information.

       Subsets
	      This  field  provides  a list of subsets or files that are to be either written to the magnetic tape media or verified from the mag-
	      netic tape media.  Each subset listed must be stored in one of the directories listed in that particular kitcap  descriptor.   If  a
	      file or subset is stored in a subdirectory of one of the directories listed in the kitcap descriptor, it is possible to include that
	      sub-path with the subset/filename entry instead of listing the entire path/subpath as another directory  listing.   For  example,  a
	      directory listed in the kitcap descriptor under the rules given in the Directories section is listed as:
	      /KITS/MYPRODUCT/001
	      A particular subset or file that a user would like to include on the media is stored in:
	      /KITS/MYPRODUCT/001/subdirectory/subset
	      Since the subdirectory/subset specification is part of the directory tree, it is not necessary to include the full path subdirectory
	      in the directory listing.  An alternative is to include the subdirectory path with the subset name in the subset list.  For example:
	      MY-PROD-001 | This is a good product:
		   /KITS/MYPRODUCT/001:SPACE:SPACE:SPACE:
		   INSTCTRL:subset1:subset2:subdirectory/subset3:subset4

       Volume identifier (MT9 media only)
	      The volume identifier is optional.  Multi-tape support is available for products that have subsets or files that take up	more  room
	      than is available by a single 9-track magnetic tape.  If the subset list results in an end-of-tape condition, the subset list can be
	      split into any number of multi-volume sets by placing (where n is the volume number of the next tape) anywhere  appropriate  in  the
	      subset  list.   The  subsets listed between the volume identifiers must fit on a single piece of media.  By default, the subset list
	      located directly after the directory list is always considered the first volume.	Therefore, a volume identifier for the first  vol-
	      ume in a multi-volume kit descriptor is not necessary.

       The following example shows a entry for disks:
       Product-codeRA:partition:
	    dd=/:Product_Description:
	    directory1:directory2:directory3:
	    instctrl:subset1:subset2:subset3:subset4:subset5:
	    dd=SUB/DIR:Product_Description:
	    instctrl:directory1:directory2:directory3:
	    subset1:subset2:subset3:subset4:subset5

       The following parts make up the kitcap descriptor for disk media:

       Product-code
	      Same as for magnetic tape.

       Media-code
	      The media code for disks is RA and is appended to the product-code provided by the user at run time, by the utility.

       Disk Partition
	      This field is the partition where you want the software written to on the disk.

       dd=
	      This  field  tells the utility what directory you want the subsets written to on the disk media that is being created.  The contrac-
	      tion can be thought of as the ``destination directory'' for the subsets.	This field is required and allows a hierarchial  structure
	      for  those  who want to put multiple products on the same disk, or want to separate parts of one product into different areas on the
	      disk.

       Typically, a disk is mounted by the utility onto a temporary mount point under This location becomes the disks root directory.  If  a  user
       wants to have only one directory for an entire product, a valid entry would be dd=/.  This entry tells the utility to write all the follow-
       ing subsets under the mount point.

       In the disk kitcap descriptor example given previously, the first five subsets are being written to the mount point, or root directory, for
       the  disk  media  being made.  Then a new directory on the disk media is made, and the next five subsets are written into that directory on
       the disk media.

       It is important to note that the top-level directory of the media disk is always considered the mount point used by the script and is  ref-
       erenced	by dd=/.  Any subdirectories listed as destination directories are created starting from the mount point and must be referenced in
       full.  For instance, in the previous example, if the user wanted to put some other subsets in a subdirectory of DIR,  the  entry  would	be
       dd=SUB/DIR/SUBSUBDIR.  Note that each new destination directory requires a product description.

       Product Description
	      This field is similar to the one defined under the magnetic tape description.  However, in the case of disk media there are 2 impor-
	      tant differences.  The product description is a required field, and all words in the description must be connected with  underscores
	      (_).  The script removes the underscores at run time.  For example, suppose the desired description was as follows:
	      This is a good product
	      The Product Description entry when making disk media would become:
	      This_is_a_good_product

       Directories
	      Same as for magnetic tape.

       instctrl
	      A directory containing the same information as INSTCTRL for magnetic tape.

       Subsets
	      Same as for magnetic tape.

Examples
       TK50 and MT9 (single-volume tape)kitcap description
       MYPRODUCT400 | MYPRODUCT software version 4:
	    :# directory listing :
	    /directory1:/directory2:/directory3:
	    SPACE:SPACE:SPACE:INSTCTRL:
	    :# subset listing :
	    subset1:subset2:subset3:subset4:subset5

       MT9 kitcap description (multi-volume tape)
       MYPRODUCT400 | MYPRODUCT software version 4:
	    /directory1:/directory2:/directory3:
	    SPACE:SPACE:SPACE:INSTCTRL:
	    subset1:subset2:subset3:subset4:subset5:
	    :# Volume 2 :
	    %%2:
	    subset6:subset7:subset8:subset9:subset10

       RA60 kitcap description (single product)
       MYPRODUCT400:c:
	    dd=/:MYPRODUCT_software_version_4:
	    /directory1:/directory2:/directory3:
	    instctrl:subset1:subset2:subset3:subset4:subset5

       RA60 kitcap description (multiple product)
       MYPRODUCT400:c:
	    dd=MYPRODUCT/BASE:
	    MYPRODUCT_software_version_4_base_subsets:
	    /directory1:/directory2:/directory3:
	    instctrl:subset1:subset2:subset3:subset4:subset5:
	    dd=MYPRODUCT/NONBASE:
	    MYPRODUCT_software_version_4_nonbase_subsets:
	    /directory1:/directory2:/directory3:
	    instctrl:subset1:subset2:subset3:subset4:subset5

See Also
       genra(8), gentapes(8)

																	 kitcap(5)
All times are GMT -4. The time now is 11:20 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy