Sponsored Content
Top Forums Shell Programming and Scripting awk - need to remove unwanted newlines on match Post 302344871 by Bubnoff on Monday 17th of August 2009 11:28:06 PM
Old 08-18-2009
awk - need to remove unwanted newlines on match

Context:
I need to remove unwanted newlines from a data file listing books and associated data. Here is a sample listing ( line numbers included ):
Code:
1 360762| Skip-beat! 14 /| 9781421517544| nb        | 2008.| Nakamura, Yoshiki.| NAKAMUR | Kyoko Mogami followed 
2 her true love Sho to Tokyo to support him while he made it big as an idol. But he's casting her out now that he's famous. 
Kyoko won't suffer in silence--she's going to get her sweet revenge by beating Sho in show biz.
3 361018| Angel numbers 101 : the meaning of 111, 123, 444, and other number sequences /| 1401920012| b         | 2008.| 
Virtue, Doreen, 1958-| 133.3359 VIRTUE |


I am using the following, found in these forums, for removing unwanted newlines:

Code:
awk 'NR==1{s=$0;next} /^[a-zA-Z]|^;/{s=s$0;next} {print s;s=$0} END{if(s)print s}' $RAW_DATA > $UNSPLIT

However, it is inexact and leaves some lines with punctuation and dates unresolved.

It needs to:
Find lines in which the first field DOES NOT contain precisely 6 digits and append them to the line above.

Thanks ~

Bub
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Remove improperly placed newlines

Hello, there. I have a file that's a horrible, horrible mess. (Basically, it's an export from a firewall config.) The people who generated the file didn't think that putting a newline in the middle of a hostname would ever be a problem. It is. Here's an example of the stuff in the file: ... (2 Replies)
Discussion started by: mikesimone
2 Replies

2. Shell Programming and Scripting

perl regexp: no match across newlines

Hi. Here's a tricky one (at least to me): I have a file named theFile.txt (UTF-8) that contains the following: a b cWhen I execute perl -pe 's|a.*c|d|sg' theFile.txtin bash 3.2 on MAC OS X 10.6, I get no match, i.e. the result is a b cagain. Any clues why? (2 Replies)
Discussion started by: BatManWSL
2 Replies

3. Shell Programming and Scripting

sed remove newlines and spaces

Hi all, i am getting count from oracle 11g by spooling it to a file. Now there are some newline characters and blank spaces i need to remove these. pl provide me a awk/sed solution. the spooled file is attached. i tried this.. but not getting req o/p (6 Replies)
Discussion started by: rishav
6 Replies

4. Shell Programming and Scripting

How to remove unwanted strings?

Hi Guys, Can someone give me a hand on how I can remove unwanted strings like "<Number>" and "</Number>" and retain only the numbers from the input file below. INPUT FILE: <Number>10050000</Number> <Number>1001340001</Number> <Number>1001750002</Number> <Number>100750003</Number>... (8 Replies)
Discussion started by: pinpe
8 Replies

5. Shell Programming and Scripting

Awk-sed help : to remove first and last line with pattern match:

awk , sed Experts, I want to remove first and last line after pattern match "vg" : I am trying : # sed '1d;$d' works fine , but where the last line is not having vg entry it is deleting one line of data. - So it should check for the pattern vg if present , then it should delete the line ,... (5 Replies)
Discussion started by: rveri
5 Replies

6. UNIX for Dummies Questions & Answers

Remove newlines

Hi buddy's my file are like this: s.no,name,band,sal 1,"suneel",,10 2,"bargav sand",,20 30," ebdug gil",,4 but i want s.no,name,band,sal 1,"suneel",,10 2,"bargav sand",,20 30,"ebdug gil",,4 any command or Shell script for this. please help me it's urgent to implement (33 Replies)
Discussion started by: Suneelbabu.etl
33 Replies

7. UNIX for Dummies Questions & Answers

Using find with awk to remove newlines

I want to list all html files present in a directory tree, the remove the newline and get one string with a space between files find /home/chrisd/Desktop/seg/geohtml/ -name '*.html' | awk BEGIN{FS=\r} '{print}' ---------- Post updated at 06:47 PM ---------- Previous update was at 06:25 PM... (5 Replies)
Discussion started by: kristinu
5 Replies

8. Shell Programming and Scripting

Awk; pattern match, remove and re write

the following pattern match works correctly for me awk '/name="Fruits"/{f=1;next} /"name=Vegetables"/{f=0} f' filename This works well for me. Id like to temporarily move the match out of the file ( > newfile) and be able to stick it back in the same place at a later time. Is this... (7 Replies)
Discussion started by: TY718
7 Replies

9. Shell Programming and Scripting

awk to remove field and match strings to add text

In file1 field $18 is removed.... column header is "Otherinfo", then each line in file1 is used to search file2 for a match. When a match is found the last four strings in file2 are copied to file1. Maybe: cut -f1-17 file1 and then match each line to file2 file1 Chr Start End ... (6 Replies)
Discussion started by: cmccabe
6 Replies

10. Shell Programming and Scripting

Using awk to remove lines from file that match text

I am trying to remove each line in which $2 is FP or RFP. I believe the below will remove one instance but not both. Thank you :). file 12 123 FP 11 10 RFP awk awk -F'\t' ' $2 != "FP"' file desired output 12 11 (6 Replies)
Discussion started by: cmccabe
6 Replies
VLMGR(1)						      Quick Database Manager							  VLMGR(1)

NAME
vlmgr - administration utility for QDBM Villa SYNOPSIS
vlmgr create [-cz|-cy|-cx] name vlmgr put [-kx|-ki] [-vx|-vi|-vf] [-keep|-cat|-dup] name key val vlmgr out [-l] [-kx|-ki] name key vlmgr get [-nl] [-l] [-kx|-ki] [-ox] [-n] name key vlmgr list [-nl] [-k|-v] [-kx|-ki] [-ox] [-top key] [-bot key] [-gt] [-lt] [-max num] [-desc] name vlmgr optimize name vlmgr inform [-nl] name vlmgr remove name vlmgr repair [-ki] name vlmgr exportdb [-ki] name file vlmgr importdb [-ki] name file vlmgr version DESCRIPTION
This manual page documents briefly the vlmgr commands. vlmgr is a utility for debugging Villa and its applications. It features editing and checking of a database. It can be used for database applications with shell scripts. This command is used in the above format. name specifies a database name. key specifies the key of a record. val specifies the value of a record. OPTIONS
A summary of options is included below. For a complete description, see the file:///usr/share/doc/qdbm-doc/spex.html#villacli. -cz compress leaves in the database with ZLIB. -cy compress leaves in the database with LZO. -cx compress leaves in the database with BZIP2. -l all records corresponding the key are dealt. -kx treat key as a binary expression of hexadecimal notation. -ki treat key as an integer expression of decimal notation. -vx treat val as a binary expression of hexadecimal notation. -vi treat val as an integer expression of decimal notation. -vf read the value from a file specified with val. -keep specify the storing mode for `VL_DKEEP'. -cat specify the storing mode for `VL_DCAT'. -dup specify the storing mode for `VL_DDUP'. -nl open the database without file locking. -top key specify the top key of listing. -bot key specify the bottom key of listing. -gt do not include the top key of listing. -lt do not include the bottom key of listing. -max num specify the max number of listing. -desc list in descending order. -ox treat the output as a binary expression of hexadecimal notation. -n do not output the tailing newline. -k output keys only. -v output values only. SEE ALSO
qdbm(3) vltsv(1), villa(3), vlopen(3). AUTHOR
QDBM was written by Mikio Hirabayashi <mikio@fallabs.com>. This manual page was written by Fumitoshi UKAI <ukai@debian.or.jp>, for the Debian project (but may be used by others). Man Page 2005-05-23 VLMGR(1)
All times are GMT -4. The time now is 10:11 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy