05-31-2014
Ya that was my concern..
Your code will remove all duplicate lines, but what i need is to remove only duplicate in headers like this:
Quote:
20140523121432,0,12,roger
Loc=redmont
From=roger@xxx.com, To=david@yyy.com,,
I thought of using any pattern matching kinda logic but not able to move with it..
Last edited by Gautham; 05-31-2014 at 02:17 AM..
Reason: updated
10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
In other news, I have a colors text file with hundreds of lines, and I want to print only the even numbered lines. for example I have this file looks something like this:
ALLCOLORS.TXT
red red green red
blue red red red
green red red blue
green green green
blue blue blue
red blue blue blue... (1 Reply)
Discussion started by: ajp7701
1 Replies
2. Shell Programming and Scripting
I need some help deleting lines in a file that contain spaces. Im sure awk or sed will work but i dont know much about those commands. Any help is appreciated :D (7 Replies)
Discussion started by: r04dw4rri0r
7 Replies
3. UNIX for Dummies Questions & Answers
Hello, i am new to the forum and know nothing about programing, Linux or Unix :( hope somebody can help me out.
I have a .txt file that i need to delete certain lines from. After searching the forum i noticed that using "sed" was the way to go, so i installed gnuwin32 (i use windows xp... (4 Replies)
Discussion started by: luis3141
4 Replies
4. Shell Programming and Scripting
File
6 dbnawldb010-b office Memphis_Corp_SQL_Diff Memphis-Corp-SQL-Inc-Application-Backup 03/09/11 03:24:04
42 luigi-b IPNRemitDB Memphis_Corp_SQL_Diff Memphis-Corp-SQL-Inc-Application-Backup 03/10/11 00:41:36
6 ebs-sqldev1-b IPNTracking Memphis_Corp_SQL_Diff... (4 Replies)
Discussion started by: ajiwww
4 Replies
5. Shell Programming and Scripting
I have one base file, and multiple target files-- each have uniform line structure so no need to use grep to find things-- can just define sections by line number.
My question is quite simple-- can I use sed to copy a defined block of lines (say lines 5-10) from filename1.txt to overwrite an... (3 Replies)
Discussion started by: czar21
3 Replies
6. Shell Programming and Scripting
Hi everyone,
I have two files (A and B) and want to combine them to one by always taking 10 rows from file A and subsequently 6 lines from file B. This process shall be repeated 40 times (file A = 400 lines; file B = 240 lines).
Does anybody have an idea how to do that using perl, awk or sed?... (6 Replies)
Discussion started by: ink_LE
6 Replies
7. UNIX for Dummies Questions & Answers
I have a diff command that does what I want but when comparing large text/log files, it uses up all the memory I have (sometimes over 8gig of memory)
diff file1.txt file2.txt | grep '^<'| awk '{$1="";print $0}' | sed 's/^ *//'
Is there a better more efficient way to find the lines in one file... (5 Replies)
Discussion started by: raptor25
5 Replies
8. UNIX for Dummies Questions & Answers
Ok hope my vocab is right here, i'm trying to write multiple sets of arguments to another file for example:
I have a script that accepts four arguments and sends them to a another file
$write.sh it then out in
so the file receiver.txt would contain this:
it then out in
what... (2 Replies)
Discussion started by: austing5
2 Replies
9. Shell Programming and Scripting
Hi
I have a file with contents:
NAMES
John
carrey
williams
How can I get all the names and store them in seperate variables(or arrays)
please keep in mind that the no. of such names is not known.Three here is a bogus value
~thanks (4 Replies)
Discussion started by: leghorn
4 Replies
10. Shell Programming and Scripting
apprecieate your help to resove this.
My source file looke like
1001 000 HEADER
1001 001 RAJESH
1001 002 100
1001 002 200
1001 002 500
1001 006 FOOTER
1002 000 HEADER
1002 001 RAMESH
1002 002 100
1002 002 200
1002 002 500
1002 006 FOOTER
my... (8 Replies)
Discussion started by: Ganesh L
8 Replies
LEARN ABOUT CENTOS
shell-quote
SHELL-QUOTE(1) User Contributed Perl Documentation SHELL-QUOTE(1)
NAME
shell-quote - quote arguments for safe use, unmodified in a shell command
SYNOPSIS
shell-quote [switch]... arg...
DESCRIPTION
shell-quote lets you pass arbitrary strings through the shell so that they won't be changed by the shell. This lets you process commands
or files with embedded white space or shell globbing characters safely. Here are a few examples.
EXAMPLES
ssh preserving args
When running a remote command with ssh, ssh doesn't preserve the separate arguments it receives. It just joins them with spaces and
passes them to "$SHELL -c". This doesn't work as intended:
ssh host touch 'hi there' # fails
It creates 2 files, hi and there. Instead, do this:
cmd=`shell-quote touch 'hi there'`
ssh host "$cmd"
This gives you just 1 file, hi there.
process find output
It's not ordinarily possible to process an arbitrary list of files output by find with a shell script. Anything you put in $IFS to
split up the output could legitimately be in a file's name. Here's how you can do it using shell-quote:
eval set -- `find -type f -print0 | xargs -0 shell-quote --`
debug shell scripts
shell-quote is better than echo for debugging shell scripts.
debug() {
[ -z "$debug" ] || shell-quote "debug:" "$@"
}
With echo you can't tell the difference between "debug 'foo bar'" and "debug foo bar", but with shell-quote you can.
save a command for later
shell-quote can be used to build up a shell command to run later. Say you want the user to be able to give you switches for a command
you're going to run. If you don't want the switches to be re-evaluated by the shell (which is usually a good idea, else there are
things the user can't pass through), you can do something like this:
user_switches=
while [ $# != 0 ]
do
case x$1 in
x--pass-through)
[ $# -gt 1 ] || die "need an argument for $1"
user_switches="$user_switches "`shell-quote -- "$2"`
shift;;
# process other switches
esac
shift
done
# later
eval "shell-quote some-command $user_switches my args"
OPTIONS
--debug
Turn debugging on.
--help
Show the usage message and die.
--version
Show the version number and exit.
AVAILABILITY
The code is licensed under the GNU GPL. Check http://www.argon.org/~roderick/ or CPAN for updated versions.
AUTHOR
Roderick Schertler <roderick@argon.org>
perl v5.16.3 2010-06-11 SHELL-QUOTE(1)