04-22-2015
Hello RudiC,
Well, I managed to trim down the dataset with "sort -u input > output", but this will only remove pure duplicates. But still, running my script on this filtered dataset will take ages :/
I presume 'sed' could help me but I can't figure out what regex I should feed him..
10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
file:
1 xxxxxxx
2 xxx xxx
5 xxx xxx
...
180 xxxxxx
200 xxx
how to remove any lines with the first number range 1-180 (9 Replies)
Discussion started by: bluemoon1
9 Replies
2. UNIX for Dummies Questions & Answers
All,
I have a text file with several entries like below:
personname
personname.domain.com
I know there is a way to use vi to remove only the personname.domain.com line. Can someone help? I believe that it involves /s/g/ something...I just can't remember the exact syntax.
Thanks (2 Replies)
Discussion started by: kjbaumann
2 Replies
3. Shell Programming and Scripting
Hi gurus,
i'm trying to remove a number of lines from a large file using the following command:
sed '1,5000d' oldfile > newfile
Somehow the lines in the old file are not deleted...
Am I doing this wrongly? Any suggestions? :confused:
Thanks! :)
wee (10 Replies)
Discussion started by: lweegp
10 Replies
4. Shell Programming and Scripting
A small question
I have a test.txt file
I have contents as:
a:google
b:yahoo
:
c:facebook
:
d:hotmail
How do I remove the line with :
my output should be
a:google
b:yahoo
c:facebook
d:hotmail (5 Replies)
Discussion started by: aronmelon
5 Replies
5. Shell Programming and Scripting
Hi,
I'm not a expert in shell programming, so i've come here to take help from u gurus.
I'm trying to tailor a csv file that i got to make it work for the LOAD FROM command.
I've a datatable csv of the below format -
--in file format
xx,xx,xx ,xx , , , , ,,xx,
xxxx,, ,, xxx,... (11 Replies)
Discussion started by: dvah
11 Replies
6. Shell Programming and Scripting
Hey Gang-
I have a list of servers. I want to exclude servers that begin with and end with certain characters. Is there an easy command to do this?
Example
wvm1234dev
wvm1234pro
uvm1122dev
uvm1122bku
uvm1344dev
I want to exclude any lines that start with "wvm" OR "uvm" AND end... (7 Replies)
Discussion started by: idiotboy
7 Replies
7. UNIX for Dummies Questions & Answers
Hi,
I have a huge file which has Lacs of lines. File system got full.
I want your guys help to suggest me a solution so that I can remove all lines from that file but not last 50,000 lines. I want solution which can remove lines from existing file so that I can have some space left with. (28 Replies)
Discussion started by: prashant2507198
28 Replies
8. Shell Programming and Scripting
I have a file that contains the following:
Party_Id1;Party_id2;Party_id3;
1;2;3;
0
0
4;5;6;
0
7;8;9;
How can I adjust the file so it looks like this:
Party_Id1;Party_id2;Party_id3;
1;2;3;
4;5;6;
7;8;9;
I Think the '0' is something like a carriage return, I don't know. But how... (2 Replies)
Discussion started by: katled
2 Replies
9. Shell Programming and Scripting
I have two files, a keepout.txt and a database.csv. They're unsorted, but could be sorted.
keepout:
user1
buser3
anuser19
notheruser27
database:
user1,2343,"information about",field,blah,34
user2,4231,"mo info",etc,stuff,43
notheruser27,4344,"hiya",thing,more thing,423... (4 Replies)
Discussion started by: esoffron
4 Replies
10. Shell Programming and Scripting
I have been searching and trying to come up with an awk that will perform the following on a
converted text file (original is a pdf).
1. Since the first two lines are (begin with) text they are removed
2. if $1 is a number then all text is merged (combined) into one line until the next... (3 Replies)
Discussion started by: cmccabe
3 Replies
LEARN ABOUT DEBIAN
drs_tool
drslib() drslib()
NAME
drs_tool - Command-line interface to DRSLIB
SYNPOSIS
DESCRIPTION
USAGE
Usage
Usage: drs_tool [command] [options] [drs-pattern]
command: list list publication-level datasets todo show file operations pending for the next version upgrade make changes to
the selected datasets to upgrade to the next version mapfile make a mapfile of the selected dataset history list all versions of the
selected dataset init initialise CMIP5 product detection data
OPTIONS
--help display this help message and exit
--version
output version information and exit
-h, --help show this help message and exit
-R ROOT, --root=ROOT
Root directory of the DRS tree
-I INCOMING, --incoming=INCOMING
Incoming directory for DRS files. Defaults to <root>/output
-a ACTIVITY, --activity=ACTIVITY
Set DRS attribute activity for dataset discovery
-p PRODUCT, --product=PRODUCT
Set DRS attribute product for dataset discovery
-i INSTITUTE, --institute=INSTITUTE
Set DRS attribute institute for dataset discovery
-m MODEL, --model=MODEL
Set DRS attribute model for dataset discovery
-e EXPERIMENT, --experiment=EXPERIMENT
Set DRS attribute experiment for dataset discovery
-f FREQUENCY, --frequency=FREQUENCY
Set DRS attribute frequency for dataset discovery
-r REALM, --realm=REALM
Set DRS attribute realm for dataset discovery
-v VERSION, --version=VERSION
Force version upgrades to this version
-P FILE, --profile=FILE
Profile the script exectuion into FILE
--detect-product
Automatically detect the DRS product of incoming data
REQUIRES
FILES
drs_tool uses metaconfig to configure and store configuration information. Its configuration may be found then in a metaconfig.conf file,
in /etc/metaconfig/metaconfig.conf on Debian systems, which may be overriden by $HOME/.metaconfig.conf or ./metaconfig.conf
ENVIRONMENT VARIABLES
The variable METACONF_CONF
VERSION
0.3.0a3
BUGS
To query and report bugs in drs_tool, please use reportbug on Debian systems.
SEE ALSO
http://esgf.org/esgf-drslib-site/index.html
AUTHOR
DRSLIB was written by Stephen Pascoe, STFC.SC.UK. This manual page was written by Alastair McKinstry.
drslib()