Thanks. Sorry, I guess I was a bit vague here. When I wrote
Quote:
shell script which removes all the XML tags including the text inside the tags from some 4 million XML files
I meant the script deletes contents inside the tags like
This means my script removes only the above tags including all the text inside the tags (like "text" and "?xml version="1.0" encoding="iso-8859-1" ?") and keeps the main paragraphs of the files.
---------- Post updated at 05:12 PM ---------- Previous update was at 05:09 PM ----------
Oh great!
You've pointed out one more fault. It is indeed deleting everything. This I can fix myself.
I have a task to move more than 35000 files every two hours, from the same directory to another directory based on a file that has the list of filenames
I tried the following logics
(1)
find . -name \*.dat > list
for i in `cat list` do mv $i test/ done
(2)
cat list|xargs -i mv "{}"... (7 Replies)
Hi all;
I'm having a problem when want to list a large number of files in current directory using find together with the prune option.
First i used this command but it list all the files including those in sub directories:
find . -name "*.dat" | xargs ls -ltr
Then i modified the command... (2 Replies)
Hi,
i have more than 1000 data files(.txt) like this
first file format:
178.83 554.545
179.21 80.392
second file:
178.83 990.909
179.21 90.196
etc.
I want to combine them to the following format:
178.83,554.545,990.909,...
179.21,80.392,90.196,... (7 Replies)
Hellow i have a large number of files that i want to concatenate to one. these files start with the word 'VOICE_' for example
VOICE_0000000000
VOICE_1223o23u0
VOICE_934934927349
I use the following code:
cat /ODS/prepaid/CDR_FLOW/MEDIATION/VOICE_* >> /ODS/prepaid/CDR_FLOW/WORK/VOICE
... (10 Replies)
Hi Forum.
I was trying to search the following scenario on the forum but was not able to.
Let's say that I have a very large file that has some bad data in it (for ex: 0.0015 in the 12th column) and I would like to find the line number and remove that particular line.
What's the easiest... (3 Replies)
Hi. I need to delete a large number of files listed in a txt file. There are over 90000 files in the list. Some of the directory names and some of the file names do have spaces in them.
In the file, each line is a full path to a file:
/path/to/the files/file1
/path/to/some other/files/file 2... (4 Replies)
Hi!
I just want to count number of files in a directory, and write to new text file, with number of files and their name
output should look like this,,
assume that below one is a new file created by script
Number of files in directory = 25
1. a.txt
2. abc.txt
3. asd.dat... (20 Replies)
Want to sftp large number of files ... approx 150 files will come to server every minute. (AIX box)
Also need make sure file has been sftped successfully...
Please let me know :
1. What is the best / faster way to transfer files?
2. should I use batch option -b so that connectivity will be... (3 Replies)
Hi.
I found many scripts in the web of achieving this.
But I like to use this one
find /EDWH-DMT03 -xdev -size +10000 -exec ls -la {} \;|sort -n -k 5 > LARGE.rst
But the problem is, why it still list out files with 89 bytes as the output? Is there anything wrong with the command?
My... (7 Replies)
Discussion started by: aimy
7 Replies
LEARN ABOUT DEBIAN
pegasus-config
PEGASUS-CONFIG(1)PEGASUS-CONFIG(1)NAME
pegasus-config - The authority for where parts of the Pegasus system exists on the filesystem. pegasus-config can be used to find libraries
such as the DAX generators.
SYNOPSIS
pegasus-config [-h] [--help] [-V] [--version] [--noeoln]
[--perl-dump] [--perl-hash] [--python-dump] [--sh-dump]
[--bin] [--conf] [--java] [--perl] [--python]
[--python-externals] [--schema] [--classpath]
[--local-site] [--full-local]
DESCRIPTION
pegasus-config is used to find locations of Pegasus system components. The tool is used internally in Pegasus and by users who need to find
paths for DAX generator libraries and schemas.
OPTIONS -h, --help
Prints help and exits.
-V, --version
Prints Pegasus version information
--perl-dump
Dumps all settings in perl format as separate variables.
--perl-hash
Dumps all settings in perl format as single perl hash.
--python-dump
Dumps all settings in python format.
--sh-dump
Dumps all settings in shell format.
--bin
Print the directory containing Pegasus binaries.
--conf
Print the directory containing configuration files.
--java
Print the directory containing the jars.
--perl
Print the directory to include into your PERL5LIB.
--python
Print the directory to include into your PYTHONLIB.
--python-externals
Print the directory to the external Python libraries.
--schema
Print the directory containing schemas.
--classpath
Builds a classpath containing the Pegasus jars.
--noeoln
Do not produce a end-of-line after output. This is useful when being called from non-shell backticks in scripts. However, order is
important for this option: If you intend to use it, specify it first.
--local-site [d]
Create a site catalog entry for site "local". This is only an XML snippet without root element nor XML headers. The optional argument
"d" points to the mount point to use. If not specified, defaults to the user's $HOME directory.
--full-local [d]
Create a complete site catalog with only site "local". The an XML snippet without root element nor XML headers. The optional argument
"d" points to the mount point to use. If not specified, defaults to the user's $HOME directory.
EXAMPLE
To set the PYTHONPATH variable in your shell for using the Python DAX API:
export PYTHONPATH=`pegasus-config --python`
To set the same path inside Python:
config = subprocess.Popen("pegasus-config --python-dump", stdout=subprocess.PIPE, shell=True).communicate()[0]
exec config
To set the PERL5LIB variable in your shell for using the Perl DAX API:
export PERL5LIB=`pegasus-config --perl`
To set the same path inside Perl:
eval `pegasus-config --perl-dump`;
die("Unable to eval pegasus-config output: $@") if $@;
will set variables a number of lexically local-scoped my variables with prefix "pegasus_" and expand Perl's search path for this script.
Alternatively, you can fail early and collect all Pegasus-related variables into a single global %pegasus variable for convenience:
BEGIN {
eval `pegasus-config --perl-hash`;
die("Unable to eval pegasus-config output: $@") if $@;
}
AUTHOR
Pegasus Team http://pegasus.isi.edu
05/24/2012 PEGASUS-CONFIG(1)