06-16-2014
thanks! that does seem to work but your commands are more advanced than what i know
for example i dont really understand what exactly the awk is doing but it does seem to be getting the directories
the first line saves the files into index.html
and the second one prints outs a lot of ftp:// lines
how do i feed that into the wget?
thanks!
10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
I'm running Fedora Core 6 as an FTP server on a powerMac G4...
I'm trying to create a script to remove files older than 3 days...
I'm able to find all data older than 3 days but it finds hidden files such as
/home/ftp/goossens/.canna
/home/ftp/goossens/.kde... (4 Replies)
Discussion started by: James_UK
4 Replies
2. Solaris
On Solaris, suppose there is a directory 'dir'.
Log files of size approx 1MB are continuously being
deposited here by scp command. I have a script that scans
this dir every 5 mins and moves away the log files that
have been deposited so far.
How do I design my script so that I pick up *only*... (6 Replies)
Discussion started by: sentak
6 Replies
3. UNIX for Advanced & Expert Users
I'm using wget 1.11.4 on Cygwin 1.5.25.
I'm trying to recursively download a directory tree, which is the root of a javadoc tree.
This is approximately the command line I tried:
wget -x -p -r http://<host>/.../apidoc
When it finished, it seemed like it downloaded... (0 Replies)
Discussion started by: dkarr
0 Replies
4. Shell Programming and Scripting
Can you tell me how to download the directory tree just starting from "project1/" in this URL?
"https://somesite.com/projects/t/project1/"
This command does not seem to do what I want as it downloads also files from the upper hierarchy:
wget --no-check-certificate --http-user=user... (4 Replies)
Discussion started by: majormark
4 Replies
5. Shell Programming and Scripting
Is there a way to customize ls to ignore files ending with ~ and #? (those are Emacs backup and auto-save files). I found -B option, which only ignores ~ files (2 Replies)
Discussion started by: yaroslavvb
2 Replies
6. Shell Programming and Scripting
Hello,
I know find can be prevented from recursing into directories with something like the following...
find . -name .svn -prune -a type d
But how can I completely prevent directories of a certain name (.svn) from being displayed at all, the top level and the children?
I really... (2 Replies)
Discussion started by: nwb123
2 Replies
7. Shell Programming and Scripting
Hello Unix Geeks,
I am in a situation to use wget for crawling a site where the site contains 5 IP addresses. Out of 5, 4 are accessible and 1 is having a problem due to firewall problems.
In this case, my wget is getting stuck with that X.X.X.X and giving up. How can I ignore this IP and... (4 Replies)
Discussion started by: sathyaonnuix
4 Replies
8. Shell Programming and Scripting
Dear All,
I am using find command
find /my_rep/*/RKYPROOF/*/*/WDM/HOME_INT/PWD_DATA -name rk*myguidelines*.pdf -print
The problem i am facing here is find /my_rep/*/
the directory after my_rep could be mice001, mice002 and mice001_PO, mice002_PO
i want to ignore mice***_PO directory... (3 Replies)
Discussion started by: yadavricky
3 Replies
9. Shell Programming and Scripting
i have a cron that mirrors a site periodically
wget -r -nc --passive-ftp ftp://user:pass@123.456.789.0
i want to download this into a directory called /files
but when I do this, it always create a new directory called "123.456.789.0" (the hostname)
it puts it into /files/123.456.789.0
but... (3 Replies)
Discussion started by: vanessafan99
3 Replies
10. UNIX for Advanced & Expert Users
I am using aix. I would like to ignore the /u directory. I tried this but it is not working.
find / -type f -type d \( -path /u \) -prune -o -name '*rpm*' 2>/dev/null
/u/appx/ls.rpm
/u/arch/vim.rpm (4 Replies)
Discussion started by: cokedude
4 Replies
PYP(1) General Commands Manual PYP(1)
NAME
pyp - The Pyed Piper: A Modern Python Alternative to awk, sed and Other Unix Text Manipulation Utilities
SYNOPSIS
pyp [options] files ...
DESCRIPTION
pyp, the Pyed Piper, is a command line tool for text manipulation. It is similar to awk and sed in functionality, but its subcommands are
Python based, and thus more familiar to many programmers.
It can operate both on a per-line base and on the complete input stream. Different features can be pipelined in a single command by using
the pipe character familiar from shell commands.
pyp backs up its input for reruns with modified commands, and can save commands as macros. On the downside, the rerun feature makes it
unsuitable for continuous pipe operation.
OPTIONS
These programs follow the usual GNU command line syntax, with long options starting with two dashes (`-'). A summary of options is
included below. For a complete description, use --manual.
-h, --help
Show this help message and exit.
-m, --manual
Prints out extended help.
-l, --macro_list
Lists all available macros.
-s MACRO_SAVE_NAME, --macro_save=MACRO_SAVE_NAME
Saves current command as macro. use "#" for adding
comments EXAMPLE:
pyp -s "great_macro # prints first letter" "p[1]".
-f MACRO_FIND_NAME, --macro_find=MACRO_FIND_NAME
Searches for macros with keyword or user name.
-d MACRO_DELETE_NAME, --macro_delete=MACRO_DELETE_NAME
Deletes specified public macro.
-g, --macro_group
Specify group macros for save and delete; default is user.
-t TEXT_FILE, --text_file=TEXT_FILE
Specify text file to load. For advanced users,
you should typically cat a file into pyp.
-x, --execute
Execute all commands.
-c, --turn_off_color
Prints raw, uncolored output.
-u, --unmodified_config
Prints out generic PypCustom.py config file.
-b BLANK_INPUTS, --blank_inputs=BLANK_INPUTS
Generate this number of blank input lines; useful for
generating numbered lists with variable 'n'.
-n, --no_input
Use with command that generates output with no input;
same as --dummy_input 1.
-k, --keep_false
Print blank lines for lines that test as False.
default is to filter out False lines from the output.
-r, --rerun
Rerun based on automatically cached data from the last run.
Use this after executing "pyp", pasting input into the shell,
and hitting CTRL-D.
SEE ALSO
awk(1), grep(1), sed(1).
AUTHOR
pyp was written by Toby Rosen <tobyrosen@gmail.com>.
This manual page was written by Khalid El Fathi <khalid@elfathi.fr>, for the Debian project (and may be used by others).
March 19, 2012 PYP(1)