Sponsored Content
Top Forums Shell Programming and Scripting Wget - how to ignore files in immediate directory? Post 302906026 by vanessafan99 on Monday 16th of June 2014 05:22:58 PM
Old 06-16-2014
the directories are constantly changing and it is a cron that runs regularly
so there arent always the same directories
also there would be hundreds of directories
i want to get these directories and everything under them but not the files that are above them
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How to ignore '.' files

I'm running Fedora Core 6 as an FTP server on a powerMac G4... I'm trying to create a script to remove files older than 3 days... I'm able to find all data older than 3 days but it finds hidden files such as /home/ftp/goossens/.canna /home/ftp/goossens/.kde... (4 Replies)
Discussion started by: James_UK
4 Replies

2. Solaris

How to ignore incomplete files

On Solaris, suppose there is a directory 'dir'. Log files of size approx 1MB are continuously being deposited here by scp command. I have a script that scans this dir every 5 mins and moves away the log files that have been deposited so far. How do I design my script so that I pick up *only*... (6 Replies)
Discussion started by: sentak
6 Replies

3. UNIX for Advanced & Expert Users

Why is wget copying my directory tree with some files with "@"?

I'm using wget 1.11.4 on Cygwin 1.5.25. I'm trying to recursively download a directory tree, which is the root of a javadoc tree. This is approximately the command line I tried: wget -x -p -r http://<host>/.../apidoc When it finished, it seemed like it downloaded... (0 Replies)
Discussion started by: dkarr
0 Replies

4. Shell Programming and Scripting

wget a directory structure question

Can you tell me how to download the directory tree just starting from "project1/" in this URL? "https://somesite.com/projects/t/project1/" This command does not seem to do what I want as it downloads also files from the upper hierarchy: wget --no-check-certificate --http-user=user... (4 Replies)
Discussion started by: majormark
4 Replies

5. Shell Programming and Scripting

Getting ls to ignore ~ and # files

Is there a way to customize ls to ignore files ending with ~ and #? (those are Emacs backup and auto-save files). I found -B option, which only ignores ~ files (2 Replies)
Discussion started by: yaroslavvb
2 Replies

6. Shell Programming and Scripting

Find: ignore directory completely

Hello, I know find can be prevented from recursing into directories with something like the following... find . -name .svn -prune -a type d But how can I completely prevent directories of a certain name (.svn) from being displayed at all, the top level and the children? I really... (2 Replies)
Discussion started by: nwb123
2 Replies

7. Shell Programming and Scripting

Wget to ignore an IP address

Hello Unix Geeks, I am in a situation to use wget for crawling a site where the site contains 5 IP addresses. Out of 5, 4 are accessible and 1 is having a problem due to firewall problems. In this case, my wget is getting stuck with that X.X.X.X and giving up. How can I ignore this IP and... (4 Replies)
Discussion started by: sathyaonnuix
4 Replies

8. Shell Programming and Scripting

Find command with ignore directory

Dear All, I am using find command find /my_rep/*/RKYPROOF/*/*/WDM/HOME_INT/PWD_DATA -name rk*myguidelines*.pdf -print The problem i am facing here is find /my_rep/*/ the directory after my_rep could be mice001, mice002 and mice001_PO, mice002_PO i want to ignore mice***_PO directory... (3 Replies)
Discussion started by: yadavricky
3 Replies

9. Shell Programming and Scripting

How to change wget download directory?

i have a cron that mirrors a site periodically wget -r -nc --passive-ftp ftp://user:pass@123.456.789.0 i want to download this into a directory called /files but when I do this, it always create a new directory called "123.456.789.0" (the hostname) it puts it into /files/123.456.789.0 but... (3 Replies)
Discussion started by: vanessafan99
3 Replies

10. UNIX for Advanced & Expert Users

AIX find ignore directory

I am using aix. I would like to ignore the /u directory. I tried this but it is not working. find / -type f -type d \( -path /u \) -prune -o -name '*rpm*' 2>/dev/null /u/appx/ls.rpm /u/arch/vim.rpm (4 Replies)
Discussion started by: cokedude
4 Replies
condor_preen(1) 					      General Commands Manual						   condor_preen(1)

Name
       condor_preen remove - extraneous files from Condor directories

Synopsis
       condor_preen [-mail] [-remove] [-verbose]

Description
       condor_preenexamines  the  directories belonging to Condor, and removes extraneous files and directories which may be left over from Condor
       processes which terminated abnormally either due to internal errors or a system crash. The directories checked are the  LOG  ,	EXECUTE  ,
       and   SPOOL directories as defined in the Condor configuration files. condor_preenis intended to be run as user	root or user  condor peri-
       odically as a backup method to ensure reasonable file system cleanliness in the face of errors. This is done automatically  by  default	by
       the condor_masterdaemon. It may also be explicitly invoked on an as needed basis.

       When  condor_preencleans the  SPOOL directory, it always leaves behind the files specified in the configuration variable  VALID_SPOOL_FILES
       as given by the configuration. For the  LOG directory, the only files removed or reported are those listed within the  configuration  vari-
       able   INVALID_LOG_FILES  list. The reason for this difference is that, in general, the files in the  LOG directory ought to be left alone,
       with few exceptions. An example of exceptions are core files. As there are new log files introduced regularly, it is less effort to specify
       those that ought to be removed than those that are not to be removed.

Options
       -mail

	  Send mail to the user defined in the	PREEN_ADMIN configuration variable, instead of writing to the standard output.

       -remove

	  Remove the offending files and directories rather than reporting on them.

       -verbose

	  List all files found in the Condor directories, even those which are not considered extraneous.

Exit Status
       condor_preenwill exit with a status value of 0 (zero) upon success, and it will exit with the value 1 (one) upon failure.

Author
       Condor Team, University of Wisconsin-Madison

Copyright
       Copyright  (C)  1990-2012  Condor  Team,  Computer  Sciences Department, University of Wisconsin-Madison, Madison, WI. All Rights Reserved.
       Licensed under the Apache License, Version 2.0.

       See the Condor Version 7.8.2 Manualor http://www.condorproject.org/licensefor additional notices. condor-admin@cs.wisc.edu

								  September 2012						   condor_preen(1)
All times are GMT -4. The time now is 01:55 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy