10-14-2008
Removing end of line to merge multiple lines
I'm sure this will be an easy question for you experts out there, but I have been searching the forum and working on this for a couple hours now and can't get it right.
I have a very messy data file that I am trying to tidy up - one of the issues is some records are split into multiple lines:
999999000 "Name" "this is text for line one
line two
line three"
And I've been trying all sorts of version of sed to get it to look like this:
999999000 "Name" "this is text for line one line two line three"
and yes, I have tried things like sed 's/$/ /' file1 > file2... the problem is not every line has an issue, so I'm trying to figure out how to only remove line feeds for problematic lines, not all lines
the problem lines will begin with alpha characters not numeric, so I've been trying to do something with that but to no avail
thanks
Last edited by tink; 10-14-2008 at 12:17 PM..
10 More Discussions You Might Find Interesting
1. UNIX for Dummies Questions & Answers
Hello everybody,
I'm trying to merge a lot of files, but I want to include the filename to the end of each line. I've tried to use cat, but I got stuck.
My files are for example:
file01.001
123456 aaa ddd ee
458741 eee fff ee
file02.003
478596 uuu ddd ee
145269 ttt fff ee
... (4 Replies)
Discussion started by: ernesto561
4 Replies
2. UNIX for Dummies Questions & Answers
How do you remove trailing empty lines at the end of a text file? Thanks! (3 Replies)
Discussion started by: evelibertine
3 Replies
3. Shell Programming and Scripting
I would like to remove carriage returns/line feeds in a text file, but in a specific cadence:
Read first line (Header Line 1), remove cr/lf at the end (replace it with a space ideally);
Read the next line (Line of Text 2), leave the cr/lf intact;
Read the next line, remove the cr/lf;
Read... (14 Replies)
Discussion started by: tomr2012
14 Replies
4. UNIX for Dummies Questions & Answers
I'm trying to remove all of the empty lines at the end of a Tab delimited file. They have no data just tabs.
I've tried may things, here are a couple:
sed /^\t.\t/d File1 > File2
sed /^\t{44}/d File1 > File2
What am I missing? (9 Replies)
Discussion started by: SirHenry1
9 Replies
5. Shell Programming and Scripting
example:
comment Now_TB.table column errac is for error messages
1 - first
2 - second
3 -third ;
in this example I need to be able to grab the comment as first word and ; as the last word and it might span a few lines. I need it to be put all in one line without line breaks so I can... (4 Replies)
Discussion started by: wambli
4 Replies
6. Shell Programming and Scripting
Hi guys,
So i have a input file with several sequences aligned (fasta)
>NC_005930 241 bp
MNMINIFIINNIFDQFIPVKLSIFSLTSVGSIIA
LSWVWINTKTHWAISRSNTP-SLLLNSL
WTLLITNL-NEKTNPWAPWLFSLFLLCFSFNIMSLI-PYTF-SQ
TSHLSFTFGLSLPIWIMVNIAGFKNNWKKKISHLLPQGTPIYLVPVMII
IETISLFIQPLTLGFRLGANLLAGHLLIFLCSCTIWE... (6 Replies)
Discussion started by: andreia
6 Replies
7. Shell Programming and Scripting
hello everyone,
im new here, and also programming with awk, sed and grep commands on linux.
In my text i have many lines with this config:
1 1 4 3 1 1 2 5
2 2 1 1 1 3 1 2
1 3 1 1 1 2 2 2
5 2 4 1
3 2 1 1 4 1 2 1
1 1 3 2 1 1 5 4
1 3 1 1... (3 Replies)
Discussion started by: satir
3 Replies
8. Shell Programming and Scripting
GM,
I have an issue at work, which requires a simple solution. But, after multiple attempts, I have not been able to hit on the code needed.
I am assuming that sed, awk or even perl could do what I need.
I have an application that adds extra blank page feeds, for multiple reports, when... (7 Replies)
Discussion started by: jxfish2
7 Replies
9. Shell Programming and Scripting
Hi all, I'm relatively new to scripting, I can do pretty basic things. I have a daily log file that looks like:
timestamp=2017-06-28-01.01.35.080576;
event status=0;
userid=user1;
authid=user1;
application id=10.10.10.10.11111.12345678901;
application name=GUI;
... (29 Replies)
Discussion started by: dwdnet
29 Replies
10. Shell Programming and Scripting
I have a file similar to the below. I am selecting only the paragraphs with @inlineifset.
I am using the following command
sed '/@inlineifset/,/^ *$/!d;
s/@inlineifset{mrg, @btpar{@//' $flnm >> $ofln
This produces
@section Correlations between
seismograms,,,,}}
... (5 Replies)
Discussion started by: Danette
5 Replies
JOIN(1) General Commands Manual JOIN(1)
NAME
join - relational database operator
SYNOPSIS
join [ options ] file1 file2
DESCRIPTION
Join forms, on the standard output, a join of the two relations specified by the lines of file1 and file2. If file1 is `-', the standard
input is used.
File1 and file2 must be sorted in increasing ASCII collating sequence on the fields on which they are to be joined, normally the first in
each line.
There is one line in the output for each pair of lines in file1 and file2 that have identical join fields. The output line normally con-
sists of the common field, then the rest of the line from file1, then the rest of the line from file2.
Fields are normally separated by blank, tab or newline. In this case, multiple separators count as one, and leading separators are dis-
carded.
These options are recognized:
-an In addition to the normal output, produce a line for each unpairable line in file n, where n is 1 or 2.
-e s Replace empty output fields by string s.
-jn m Join on the mth field of file n. If n is missing, use the mth field in each file.
-o list
Each output line comprises the fields specifed in list, each element of which has the form n.m, where n is a file number and m is a
field number.
-tc Use character c as a separator (tab character). Every appearance of c in a line is significant.
SEE ALSO
sort(1), comm(1), awk(1)
BUGS
With default field separation, the collating sequence is that of sort -b; with -t, the sequence is that of a plain sort.
The conventions of join, sort, comm, uniq, look and awk(1) are wildly incongruous.
JOIN(1)