06-18-2012
Removing Lines Shared by Multiple Files
Hey everyone, I have a question about comparing two files. I have two lists of files. The first list, todo.csv, lists a series of compounds my supervisor wants me to perform calculations on. The second list, done.csv, lists a series of compounds that I have already performed calculations on. However, the two lists overlap, and I am interested in reducing redundancy. So my question; is there any way to remove all lines from todo.csv that also occur in done.csv? I suspect awk could be used to do the trick, but I am not really sure how. Any help would be greatly appreciated.
10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
I have several huge files wich contains oracle table creation scripts as follows:
I would need to remove the pattern colored in red above. Any sed/awk/pearl code will be of much help.
Thanks (2 Replies)
Discussion started by: sabyasm
2 Replies
2. Shell Programming and Scripting
I'm sure this will be an easy question for you experts out there, but I have been searching the forum and working on this for a couple hours now and can't get it right.
I have a very messy data file that I am trying to tidy up - one of the issues is some records are split into multiple lines: ... (4 Replies)
Discussion started by: tink
4 Replies
3. UNIX for Dummies Questions & Answers
Hi,
I'm trying to remove multiple lines of text based off a series of different words and output it to a new file
The document contains a ton of data but i want to delete any line that has the following
mx1.rr.biz.com or ns2.ri.biz.com
i tried using grep -v filename "mx1.rr.biz.com" >... (3 Replies)
Discussion started by: spartan22
3 Replies
4. Shell Programming and Scripting
Hi Guru's , I have a whole bunch of files in /var/tmp that i need to strip any blank lines from, so ive written the following script to identify the lines (which works perfectly).. but i wanted to know, how can I actually strip the identified lines from the actual source files ??
my... (11 Replies)
Discussion started by: hcclnoodles
11 Replies
5. Shell Programming and Scripting
I have 9,000 + html files.
I am using the following to remove the content from a certain line up
for i in `ls`
do
sed '1,569d' $i > $i.bak
done
This will remove the unwanted formatting keeping the content I need which changes in each HTML file.
the problem I have now is that the... (2 Replies)
Discussion started by: deaconf19
2 Replies
6. Shell Programming and Scripting
I'm hitting a brick wall, I have huge ldif files that I'm trying to sanitize and can do it all with SED except one thing.
I have a publicKey attribute in binary that can be one line or multiple lines.
I'm trying to remove publicKey: and stop at sn (the next attribute). Even with word Wrap... (3 Replies)
Discussion started by: LCol
3 Replies
7. Shell Programming and Scripting
I would like to remove carriage returns/line feeds in a text file, but in a specific cadence:
Read first line (Header Line 1), remove cr/lf at the end (replace it with a space ideally);
Read the next line (Line of Text 2), leave the cr/lf intact;
Read the next line, remove the cr/lf;
Read... (14 Replies)
Discussion started by: tomr2012
14 Replies
8. Shell Programming and Scripting
GM,
I have an issue at work, which requires a simple solution. But, after multiple attempts, I have not been able to hit on the code needed.
I am assuming that sed, awk or even perl could do what I need.
I have an application that adds extra blank page feeds, for multiple reports, when... (7 Replies)
Discussion started by: jxfish2
7 Replies
9. Shell Programming and Scripting
Hello Gurus,
I have a multiple pipe separated files which have records going over multiple Lines. End of line separator is \n and records going over multiple lines have <CR> as separator. below is example from one file.
1|ABC DEF|100|10
2|PQ
RS
T|200|20
3| UVWXYZ|300|30
4| GHIJKL|400|40... (7 Replies)
Discussion started by: dJHa
7 Replies
10. Shell Programming and Scripting
I have a file similar to the below. I am selecting only the paragraphs with @inlineifset.
I am using the following command
sed '/@inlineifset/,/^ *$/!d;
s/@inlineifset{mrg, @btpar{@//' $flnm >> $ofln
This produces
@section Correlations between
seismograms,,,,}}
... (5 Replies)
Discussion started by: Danette
5 Replies
LEARN ABOUT DEBIAN
obd2csv
obd2csv(1) General Commands Manual obd2csv(1)
NAME
obd2csv - Convert obdgpslogger(1) logs to csv files
SYNOPSIS
obd2csv [ options ]
DESCRIPTION
Convert obdgpslogger(1) logs to csv files
OPTIONS
-o|--out <output filename>
Output to this .csv file
-d|--db <database>
Work from logs stored in this database file
-s|--start <time>
Only dump rows more recent than this
-e|--end <time>
Only dump rows older than this
-z|--gzip
gzip compress output using zlib [if available]
-v|--version
Print out version number and exit.
-h|--help
Print out help and exit.
NOT OPTIONS
These options aren't intended for end-users, they're for the GUI.
-p|--progress
Print out progress. It will occasionally print a number in the range [0..100], indicating progress percentage.
SEE ALSO
obdgpslogger(1), obd2kml(1), obd2gpx(1), obdsim(1), obdgui(1), obdlogrepair(1)
AUTHORS
Gary "Chunky Ks" Briggs <chunky@icculus.org>
obd2csv(1)