06-13-2012
Quote:
Originally Posted by
methyl
Any comment about the order of the data in the files and whether there is a one-for-one match between the two files (in which case the paste command might be suitable?
Edit: Posts crossed. I can see that neither file is in any particular order and that your sample does not show a one-for-one match.
It's going to be necessary to sort both files. Does the order of the final output data matter?
There is one-for-one match and its not necessary that the id from input.txt is always found in status.txt, if its found then there is only one match.
No the order doesn't matter here.
7 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
Hello,
I have got one file with more than 120+ million records(35 GB in size). I have to extract some relevant data from file based on some parameter and generate other output file.
What will be the besat and fastest way to extract the ne file.
sample file format :--... (2 Replies)
Discussion started by: learner16s
2 Replies
2. Shell Programming and Scripting
hi,
I'm trying to sort a file which has 3.7 million records an gettign the following error...any help is appreciated...
sort: Write error while merging.
Thanks (6 Replies)
Discussion started by: greenworld
6 Replies
3. What is on Your Mind?
Here is an easy game!
I wrote a number between 0 and 20 (that can include 0 and 20) on a piece of paper. I am staring at it now, imagining the number so you can read my mind ;)
Reply once, and only once, with a number from 0 to 20 and the first person to guess it wins 1,000,000 Bits.
... (24 Replies)
Discussion started by: Neo
24 Replies
4. Shell Programming and Scripting
I have a log file that is about 1.2 million lines long and about 300MB.
we need a way to clean up this file and only keep the last few thousand lines.
if i use tail command we run our of memory as the file is too big.
I do have a key word to match on.
example, we want to keep every line... (8 Replies)
Discussion started by: robsonde
8 Replies
5. UNIX for Dummies Questions & Answers
Hi,
one of the server, log directory was never cleaned up. We have so many files. I want to remove all the files that starts with dfr* but I get error message when I use the *.
rm qfr*
bash: /usr/bin/rm: Arg list too long
I am trying to write this script but not working.
... (4 Replies)
Discussion started by: samnyc
4 Replies
6. UNIX for Dummies Questions & Answers
Hi,
Which way is faster rm -rf /path/ or find / -name -exec rm {} \; and why? (7 Replies)
Discussion started by: cain82
7 Replies
7. UNIX for Dummies Questions & Answers
Hi,
here is my problem:
I've got a file with 6 columns (file1):
a b c d e f
a b c d e f
a b c d e f
a b c d e f
I need to add 1 million columns to this file, each column needs to be a zero.
Here is how the result file (file2) should look like (for the sake of the example, I've only... (7 Replies)
Discussion started by: zajtat
7 Replies
LEARN ABOUT CENTOS
shell-quote
SHELL-QUOTE(1) User Contributed Perl Documentation SHELL-QUOTE(1)
NAME
shell-quote - quote arguments for safe use, unmodified in a shell command
SYNOPSIS
shell-quote [switch]... arg...
DESCRIPTION
shell-quote lets you pass arbitrary strings through the shell so that they won't be changed by the shell. This lets you process commands
or files with embedded white space or shell globbing characters safely. Here are a few examples.
EXAMPLES
ssh preserving args
When running a remote command with ssh, ssh doesn't preserve the separate arguments it receives. It just joins them with spaces and
passes them to "$SHELL -c". This doesn't work as intended:
ssh host touch 'hi there' # fails
It creates 2 files, hi and there. Instead, do this:
cmd=`shell-quote touch 'hi there'`
ssh host "$cmd"
This gives you just 1 file, hi there.
process find output
It's not ordinarily possible to process an arbitrary list of files output by find with a shell script. Anything you put in $IFS to
split up the output could legitimately be in a file's name. Here's how you can do it using shell-quote:
eval set -- `find -type f -print0 | xargs -0 shell-quote --`
debug shell scripts
shell-quote is better than echo for debugging shell scripts.
debug() {
[ -z "$debug" ] || shell-quote "debug:" "$@"
}
With echo you can't tell the difference between "debug 'foo bar'" and "debug foo bar", but with shell-quote you can.
save a command for later
shell-quote can be used to build up a shell command to run later. Say you want the user to be able to give you switches for a command
you're going to run. If you don't want the switches to be re-evaluated by the shell (which is usually a good idea, else there are
things the user can't pass through), you can do something like this:
user_switches=
while [ $# != 0 ]
do
case x$1 in
x--pass-through)
[ $# -gt 1 ] || die "need an argument for $1"
user_switches="$user_switches "`shell-quote -- "$2"`
shift;;
# process other switches
esac
shift
done
# later
eval "shell-quote some-command $user_switches my args"
OPTIONS
--debug
Turn debugging on.
--help
Show the usage message and die.
--version
Show the version number and exit.
AVAILABILITY
The code is licensed under the GNU GPL. Check http://www.argon.org/~roderick/ or CPAN for updated versions.
AUTHOR
Roderick Schertler <roderick@argon.org>
perl v5.16.3 2010-06-11 SHELL-QUOTE(1)