07-01-2015
Invoking awk once to make up to 3400 changes to a file is 99.94% likely to run MUCH faster (by a few orders of magnitude) than invoking sed 3400 times to perform the same task. If you would give us clear, explicit descriptions of what the files being processed really look like (instead of showing us the output you get from running a script with unspecified input file formats), we would be happy to try to help you solve your problem more efficiently.
The key to writing efficient, reliable software is to know what the input looks like, what output needs to be produced, and what constraints can be assumed concerning the inputs and outputs.
10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
Need help.
I have a source file that listed sets of numbers/words and what i'm trying to do is, by each line of the source file i want to look for same numbers/words in second file and if it match then write it to third file. Third file should have whole line from source file plus whole line... (7 Replies)
Discussion started by: suresh7730
7 Replies
2. Shell Programming and Scripting
I have a list of DNS servers I need to look up information on. Each of these servers has a master and a slave database. Essentially what I need to do is create two text files for each server. One with the Master view and one with the Slave view. There's 20 servers, in the end I should have 40 text... (4 Replies)
Discussion started by: spartan22
4 Replies
3. Shell Programming and Scripting
Hi all,
I have 2 files:Column1 of first file has to be matched with column 3 of second file
first file contain DATA like this in 2 columns one with gene name second with whether CAD,HT,RA T2Dor any one
column 1 column2
ARFGEF2 CAD
DDEF2 CAD
PSCD3 CAD
PSCD4 CAD
CAMK1... (5 Replies)
Discussion started by: manigrover
5 Replies
4. Shell Programming and Scripting
Hi Friends,
I'm a great fan of this forum... it has helped me tone my skills in shell scripting. I have a challenge here, which I'm sure you guys would help me in achieving...
File A has a list of job ids and I need to compare this with the File B (*.log) and File C (extend *.log) and copy... (6 Replies)
Discussion started by: asnandhakumar
6 Replies
5. Shell Programming and Scripting
Hello experts,
Please help me in achieving this in an easier way possible. I have 2 csv files with following data:
File1
08/23/2012 12:35:47,JOB_5330
08/23/2012 12:35:47,JOB_5330
08/23/2012 12:36:09,JOB_5340
08/23/2012 12:36:14,JOB_5340
08/23/2012 12:36:22,JOB_5350
08/23/2012... (5 Replies)
Discussion started by: asnandhakumar
5 Replies
6. Shell Programming and Scripting
I need to take 2 input files and create 1 output based on matches from each file. I am looking to match field #1 in both files (Userid) and create an output file that will be a combination of fields from
both file1 and file2 if there are any differences in the fields 2,3,4,5,or 6.
Below is an... (5 Replies)
Discussion started by: ambroze
5 Replies
7. Shell Programming and Scripting
Hi,
I need to compare 2 text files with around 60000 rows and 1 column. I need to compare these and write the mismatch data to 3rd file.
File1 - file2 = file3
wc -l file1.txt
58112
wc -l file2.txt
55260
head -5 file1.txt
101214200123
101214700300
101250030067
101214100500... (10 Replies)
Discussion started by: Divya Nochiyil
10 Replies
8. Shell Programming and Scripting
Hi, I am newbie in shell script.
I need your help to solve my problem.
Firstly, I have 2 files of csv and i want to compare of the contents then the output will be written in a new csv file.
File1:
SourceFile,DateTimeOriginal
/home/intannf/foto/IMG_0713.JPG,2015:02:17 11:14:07... (8 Replies)
Discussion started by: refrain
8 Replies
9. UNIX for Dummies Questions & Answers
Hi guys, looking for some help with a way to compare data in two files but with some conditions.
example,
File 1 consists of
site1,10.1.1.1
site2,20.2.2.2
site3,30.3.3.3
File 2 contains
site1,l0.1.1.1
site2,50.1.1.1
site3,30.3.3.3
site4,40.1.1.1
I want to be able to match the... (1 Reply)
Discussion started by: mutley2202
1 Replies
10. Shell Programming and Scripting
hi all,
trying this using shell/bash with sed/awk/grep
I have two files, one containing one column, the other containing multiple columns (comma delimited).
file1.txt
abc12345
def12345
ghi54321
...
file2.txt
abc1,text1,texta
abc,text2,textb
def123,text3,textc
gh,text4,textd... (6 Replies)
Discussion started by: shogun1970
6 Replies
LEARN ABOUT CENTOS
edinplace
edinplace(1) Mail Avenger 0.8.3 edinplace(1)
NAME
edinplace - edit a file in place
SYNOPSIS
edinplace [--error=code] [[--file=file] command [arg ...]]
DESCRIPTION
edinplace runs command with its input from file (or standard input by default), and then replaces the contents of file with the output of
command. To the extent possible, edinplace attempts to exit with the same status as command.
If edinplace is run on standard input (no --file option), it must inherit a file descriptor 0 that is open for both reading and writing.
When processing standard input, if edinplace does not encounter a fatal error, it rewinds its standard input to offset 0 before exiting.
Thus, a script can first run edinplace command, then run another filter command such as grep, and the resulting output will be the output
of grep on command's output.
If no command is specified, edinplace just rewinds its standard input to file offset 0. In this case, it is an error to supply the --file
option. Of course, rewinding only works when standard input is a real file (as opposed to a pipe or device).
There are two options:
--error=code (-x code)
Ordinarily, edinplace attempts to exit with the same status as command. However, if edinplace encounters some fatal error (such as
being unable to execute command), it will exit with status code. The default value is 1. The range of valid exit codes is 1-255,
inclusive.
--file=file (-f file)
Specifies that file should be edited. Otherwise, edinplace will edit its standard input (which must be opened for both reading and
writing).
--skipfrom
Skip the first line of the file if it starts "From ". If edinplace is run without a command, positions the file offset at the start of
the second line of the file. If edinplace is run with a command, then the first line of the file is neither fed to the command, nor
overwritten. This option is useful for running edinplace over mail files, which sometimes start with a "From " line specifying the
envelope sender of the message. Since "From " is not part of the message header, just a Unix convention, some programs are confused by
the presence of that line. Note that if you specify a command, then edinplace resets the file offset to 0 upon exiting, even if the
--skipfrom option was present.
EXAMPLES
The following command prepends the string "ORIGINAL: " to the beginning of each line in text file message:
edinplace -f message sed -e 's/^/ORIGINAL: /'
The following command runs the spamassassin mail filter program on a mail message stored in file message, replacing the contents of message
with spamassassin's annotated output, and exiting with code 100 if spamassassin thinks the message is spam. If edinplace encounters any
fatal errors, it will exit with code 111.
edinplace -x 111 -f message spamassassin -e 100
(spamassassin reads a mail message on standard input and outputs an annotated copy of the message including information about whether or
not the message is likely to be spam and why. The -e option to spamassassin specifies what exit status spamassassin should use if the
message appears to be spam; edinplace will use the same exit code as the program it has run.)
To run spamassassin on incoming mail before accepting the mail from the remote client, place the following line in an appropriate Mail
Avenger rcpt file as the last command executed:
bodytest edinplace -x 111 spamassassin -e 100
SEE ALSO
avenger(1)
The Mail Avenger home page: <http://www.mailavenger.org/>.
BUGS
edinplace does not make a copy of the file being edited, but rather overwrites the file as it is being processed. At any point where
command has produced more output than it has consumed input from the file, edinplace buffers the difference in memory. Thus, a command
that outputs large amounts of data before reading the input file can run edinplace out of memory. (A program that outputs data as it reads
even a very large file should be fine, however.)
If command crashes or malfunctions for any reason, you will likely lose the input file, since edinplace will view this as a program that
simply outputs the empty file.
AUTHOR
David Mazieres
Mail Avenger 0.8.3 2012-04-05 edinplace(1)