Sponsored Content
Top Forums Shell Programming and Scripting Compare output of UNIX command and match data to text file Post 302948715 by Don Cragun on Wednesday 1st of July 2015 03:30:35 PM
Old 07-01-2015
Invoking awk once to make up to 3400 changes to a file is 99.94% likely to run MUCH faster (by a few orders of magnitude) than invoking sed 3400 times to perform the same task. If you would give us clear, explicit descriptions of what the files being processed really look like (instead of showing us the output you get from running a script with unspecified input file formats), we would be happy to try to help you solve your problem more efficiently.

The key to writing efficient, reliable software is to know what the input looks like, what output needs to be produced, and what constraints can be assumed concerning the inputs and outputs.
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Compare two text file and output the same to third file

Need help. I have a source file that listed sets of numbers/words and what i'm trying to do is, by each line of the source file i want to look for same numbers/words in second file and if it match then write it to third file. Third file should have whole line from source file plus whole line... (7 Replies)
Discussion started by: suresh7730
7 Replies

2. Shell Programming and Scripting

Redirect output to a different text file depending source of data

I have a list of DNS servers I need to look up information on. Each of these servers has a master and a slave database. Essentially what I need to do is create two text files for each server. One with the Master view and one with the Slave view. There's 20 servers, in the end I should have 40 text... (4 Replies)
Discussion started by: spartan22
4 Replies

3. Shell Programming and Scripting

Request to check: compare two files , match same entries, write data before it

Hi all, I have 2 files:Column1 of first file has to be matched with column 3 of second file first file contain DATA like this in 2 columns one with gene name second with whether CAD,HT,RA T2Dor any one column 1 column2 ARFGEF2 CAD DDEF2 CAD PSCD3 CAD PSCD4 CAD CAMK1... (5 Replies)
Discussion started by: manigrover
5 Replies

4. Shell Programming and Scripting

Match list of strings in File A and compare with File B, C and write to a output file in CSV format

Hi Friends, I'm a great fan of this forum... it has helped me tone my skills in shell scripting. I have a challenge here, which I'm sure you guys would help me in achieving... File A has a list of job ids and I need to compare this with the File B (*.log) and File C (extend *.log) and copy... (6 Replies)
Discussion started by: asnandhakumar
6 Replies

5. Shell Programming and Scripting

Compare 2 files and match column data and align data from 3 column

Hello experts, Please help me in achieving this in an easier way possible. I have 2 csv files with following data: File1 08/23/2012 12:35:47,JOB_5330 08/23/2012 12:35:47,JOB_5330 08/23/2012 12:36:09,JOB_5340 08/23/2012 12:36:14,JOB_5340 08/23/2012 12:36:22,JOB_5350 08/23/2012... (5 Replies)
Discussion started by: asnandhakumar
5 Replies

6. Shell Programming and Scripting

awk help: Match data fields from 2 files & output results from both into 1 file

I need to take 2 input files and create 1 output based on matches from each file. I am looking to match field #1 in both files (Userid) and create an output file that will be a combination of fields from both file1 and file2 if there are any differences in the fields 2,3,4,5,or 6. Below is an... (5 Replies)
Discussion started by: ambroze
5 Replies

7. Shell Programming and Scripting

Compare 2 text file with 1 column in each file and write mismatch data to 3rd file

Hi, I need to compare 2 text files with around 60000 rows and 1 column. I need to compare these and write the mismatch data to 3rd file. File1 - file2 = file3 wc -l file1.txt 58112 wc -l file2.txt 55260 head -5 file1.txt 101214200123 101214700300 101250030067 101214100500... (10 Replies)
Discussion started by: Divya Nochiyil
10 Replies

8. Shell Programming and Scripting

Compare 2 files of csv file and match column data and create a new csv file of them

Hi, I am newbie in shell script. I need your help to solve my problem. Firstly, I have 2 files of csv and i want to compare of the contents then the output will be written in a new csv file. File1: SourceFile,DateTimeOriginal /home/intannf/foto/IMG_0713.JPG,2015:02:17 11:14:07... (8 Replies)
Discussion started by: refrain
8 Replies

9. UNIX for Dummies Questions & Answers

Compare data - Match first column and compare second

Hi guys, looking for some help with a way to compare data in two files but with some conditions. example, File 1 consists of site1,10.1.1.1 site2,20.2.2.2 site3,30.3.3.3 File 2 contains site1,l0.1.1.1 site2,50.1.1.1 site3,30.3.3.3 site4,40.1.1.1 I want to be able to match the... (1 Reply)
Discussion started by: mutley2202
1 Replies

10. Shell Programming and Scripting

Match text to lines in a file, iterate backwards until text or text substring matches, print to file

hi all, trying this using shell/bash with sed/awk/grep I have two files, one containing one column, the other containing multiple columns (comma delimited). file1.txt abc12345 def12345 ghi54321 ... file2.txt abc1,text1,texta abc,text2,textb def123,text3,textc gh,text4,textd... (6 Replies)
Discussion started by: shogun1970
6 Replies
edinplace(1)							Mail Avenger 0.8.3						      edinplace(1)

NAME
edinplace - edit a file in place SYNOPSIS
edinplace [--error=code] [[--file=file] command [arg ...]] DESCRIPTION
edinplace runs command with its input from file (or standard input by default), and then replaces the contents of file with the output of command. To the extent possible, edinplace attempts to exit with the same status as command. If edinplace is run on standard input (no --file option), it must inherit a file descriptor 0 that is open for both reading and writing. When processing standard input, if edinplace does not encounter a fatal error, it rewinds its standard input to offset 0 before exiting. Thus, a script can first run edinplace command, then run another filter command such as grep, and the resulting output will be the output of grep on command's output. If no command is specified, edinplace just rewinds its standard input to file offset 0. In this case, it is an error to supply the --file option. Of course, rewinding only works when standard input is a real file (as opposed to a pipe or device). There are two options: --error=code (-x code) Ordinarily, edinplace attempts to exit with the same status as command. However, if edinplace encounters some fatal error (such as being unable to execute command), it will exit with status code. The default value is 1. The range of valid exit codes is 1-255, inclusive. --file=file (-f file) Specifies that file should be edited. Otherwise, edinplace will edit its standard input (which must be opened for both reading and writing). --skipfrom Skip the first line of the file if it starts "From ". If edinplace is run without a command, positions the file offset at the start of the second line of the file. If edinplace is run with a command, then the first line of the file is neither fed to the command, nor overwritten. This option is useful for running edinplace over mail files, which sometimes start with a "From " line specifying the envelope sender of the message. Since "From " is not part of the message header, just a Unix convention, some programs are confused by the presence of that line. Note that if you specify a command, then edinplace resets the file offset to 0 upon exiting, even if the --skipfrom option was present. EXAMPLES
The following command prepends the string "ORIGINAL: " to the beginning of each line in text file message: edinplace -f message sed -e 's/^/ORIGINAL: /' The following command runs the spamassassin mail filter program on a mail message stored in file message, replacing the contents of message with spamassassin's annotated output, and exiting with code 100 if spamassassin thinks the message is spam. If edinplace encounters any fatal errors, it will exit with code 111. edinplace -x 111 -f message spamassassin -e 100 (spamassassin reads a mail message on standard input and outputs an annotated copy of the message including information about whether or not the message is likely to be spam and why. The -e option to spamassassin specifies what exit status spamassassin should use if the message appears to be spam; edinplace will use the same exit code as the program it has run.) To run spamassassin on incoming mail before accepting the mail from the remote client, place the following line in an appropriate Mail Avenger rcpt file as the last command executed: bodytest edinplace -x 111 spamassassin -e 100 SEE ALSO
avenger(1) The Mail Avenger home page: <http://www.mailavenger.org/>. BUGS
edinplace does not make a copy of the file being edited, but rather overwrites the file as it is being processed. At any point where command has produced more output than it has consumed input from the file, edinplace buffers the difference in memory. Thus, a command that outputs large amounts of data before reading the input file can run edinplace out of memory. (A program that outputs data as it reads even a very large file should be fine, however.) If command crashes or malfunctions for any reason, you will likely lose the input file, since edinplace will view this as a program that simply outputs the empty file. AUTHOR
David Mazieres Mail Avenger 0.8.3 2012-04-05 edinplace(1)
All times are GMT -4. The time now is 10:35 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy