Sponsored Content
Top Forums Shell Programming and Scripting Delimited records splitted into different lines Post 302972580 by Scrutinizer on Sunday 8th of May 2016 05:11:09 AM
Old 05-08-2016
The standard way of tackling this would be something like this:
Code:
awk -F\| 'NF<15{while(getline s>0) $0=$0 s}1'  file

Which should work with your sample..

However as others have pointed out, unless you provide more information it will be difficult to tell if this would be a solution to your problem..


---
Note that this will not work if the newlines appear in the very last field.

Also note that the csv format allows for newlines within quoted fields, so the sample you posted seems to within specification, so by joining the lines you are effectively changing the content by removing the newlines..

If you want to process the file you do not need remove the newlines in order to process the file. For example to print field 10 (without the enclosing double quotes), you could do something like this:
Code:
$> awk -F\| 'NF<15{while(getline s>0) $0=$0 RS s}{gsub(/^"|"$/,x,$10); print $10}' file
5.1.0 e This is a permanent error. Please verify the address(es) and try again.
<TEST>:
123.123.12 does not like recipient.
Remote host said: 123 Address rejected test@test
Giving up on 1200.
--- 
$>


Last edited by Scrutinizer; 05-08-2016 at 08:35 AM..
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Delete Duplicate records from a tilde delimited file

Hi All, I want to delete duplicate records from a tilde delimited file. Criteria is considering the first 2 fields, the combination of which has to be unique, below is a sample of records in the input file 1620000010338~2446694087~0~20061130220000~A00BCC1CT... (5 Replies)
Discussion started by: irshadm
5 Replies

2. UNIX for Dummies Questions & Answers

Extract records by column value - file non-delimited

the data in my file is has no delimiters. it looks like this: H52082320024740010PH333200612290000930 0.0020080131 D5208232002474000120070306200703060580T1502 TT 1.00 H52082320029180003PH333200702150001 30 100.0020080205 D5208232002918000120070726200707260580T1502 ... (3 Replies)
Discussion started by: jclanc8
3 Replies

3. Shell Programming and Scripting

how to Insert values in multiple lines(records) within a pipe delimited text file in specific cols

this is Korn shell unix. The scenario is I have a pipe delimited text file which needs to be customized. say for example,I have a pipe delimited text file with 15 columns(| delimited) and 200 rows. currently the 11th and 12th column has null values for all the records(there are other null columns... (4 Replies)
Discussion started by: vasan2815
4 Replies

4. Shell Programming and Scripting

Print records which do not have expected number of fields in a comma delimited file

Hi, I have a comma (,) delimited file, in which few fields are enclosed with in double quotes " ". I have to print the records in the file which donot have expected number of field with the line number. File1 ==== name,desgnation,doj,project #header#... (7 Replies)
Discussion started by: machomaddy
7 Replies

5. Shell Programming and Scripting

Create new lines using a delimited string.

Hi I have a text file called 'fileA' which contains the follwoing line examples 01:rec1:25,50,75,100 02:rec2:30,60 03:rec3:20,40 I would like to create a new file where each of the comma separated values appears on a new line but prefixed with the first two fields e.g. 01:rec1:25... (3 Replies)
Discussion started by: mackmb
3 Replies

6. UNIX for Dummies Questions & Answers

Removing empty lines at the end of a Tab-delimited file

I'm trying to remove all of the empty lines at the end of a Tab delimited file. They have no data just tabs. I've tried may things, here are a couple: sed /^\t.\t/d File1 > File2 sed /^\t{44}/d File1 > File2 What am I missing? (9 Replies)
Discussion started by: SirHenry1
9 Replies

7. UNIX for Advanced & Expert Users

Wanted best way to validate delimited file records

actually i post about this issue before but many folkz miss-understood with my quesion, We are checking for the delimited file records validation Delimited file will have data like this: Aaaa|sdfhxfgh|sdgjhxfgjh|sdgjsdg|sgdjsg| Aaaa|sdfhxfgh|sdgjhxfgjh|sdgjsdg|sgdjsg|... (3 Replies)
Discussion started by: Seshendranath
3 Replies

8. Shell Programming and Scripting

Script to match strings that sometimes are splitted in 2 lines

Hello to all, I have an hexdump -C format as below: 31 54 47 55 48 4c 52 31 5f 52 31 32 31 31 32 ff 44 00 00 0E 01 32 14 56 42 17 47 48 0f ff ff ff 44 00 00 01 32 14 56 00 23 83 95 2f 42 17 47 48 00 0f ff ff 00 15 00 0a 48 00 01 5a 00 02 17 00 00 2f 00 00 30 00 00 31 00 00 ff 34 ff 44 00... (23 Replies)
Discussion started by: Ophiuchus
23 Replies

9. UNIX for Dummies Questions & Answers

How to convert a comma delimited string to records or lines of text?

Hi, I am not sure if I've posted this question before. Anyway, I previously asked about converting lines of text into a comma delimited string. Now I am needing to do the other way around ... :( :o Can anyone advise how is this possible? Example as below: Converting records/lines to... (2 Replies)
Discussion started by: newbie_01
2 Replies

10. UNIX for Beginners Questions & Answers

Need help on an old post - How to convert a comma delimited string to records or lines of text?

Hi, Apologies in advance to the moderator if I am posting this the wrong way. I've searched and found the solution to an old post but as it is a very old post, I don't see an option to update it with additional question. The question I have is in relation to the following post: How to... (6 Replies)
Discussion started by: newbie_01
6 Replies
URI::Find::Delimited(3pm)				User Contributed Perl Documentation				 URI::Find::Delimited(3pm)

NAME
URI::Find::Delimited - Find URIs which may be wrapped in enclosing delimiters. DESCRIPTION
Works like URI::Find, but is prepared for URIs in your text to be wrapped in a pair of delimiters and optionally have a title. This will be useful for processing text that already has some minimal markup in it, like bulletin board posts or wiki text. SYNOPSIS
my $finder = URI::Find::Delimited->new; my $text = "This is a [http://the.earth.li/ titled link]."; $finder->find($text); print $text; METHODS
new my $finder = URI::Find::Delimited->new( callback => &callback, delimiter_re => [ '[', ']' ], ignore_quoted => 1 # defaults to 0 ); All arguments are optional; defaults are provided (see below). Creates a new URI::Find::Delimited object. This object works similarly to a URI::Find object, but as well as just looking for URIs it is also aware of the concept of a wrapped, titled URI. These look something like [http://foo.com/ the foo website] where: * "[" is the opening delimiter * "]" is the closing delimiter * "http://foo.com/" is the URI * "the foo website" is the title * the URI and title are separated by spaces and/or tabs The URI::Find::Delimited object will extract each of these parts separately and pass them to your callback. callback "callback" is a function which is called on each URI found. It is passed five arguments: the opening delimiter (if found), the closing delimiter (if found), the URI, the title (if found), and any whitespace found between the URI and title. The return value of the callback will replace the original URI in the text. If you do not supply your own callback, the object will create a default one which will put your URIs in 'a href' tags using the URI for the target and the title for the link text. If no title is provided for a URI then the URI itself will be used as the title. If the delimiters aren't balanced (eg if the opening one is present but no closing one is found) then the URI is treated as not being wrapped. Note: the default callback will not remove the delimiters from the text. It should be simple enough to write your own callback to remove them, based on the one in the source, if that's what you want. In fact there's an example in this distribution, in "t/delimited.t". delimiter_re The "delimiter_re" parameter is optional. If you do supply it then it should be a ref to an array containing two regexes. It defaults to using single square brackets as the delimiters. Don't use capturing groupings "( )" in your delimiters or things will break. Use non-capturing "(?: )" instead. ignore_quoted If the "ignore_quoted" parameter is supplied and set to a true value, then any URIs immediately preceded with a double-quote char- acter will not be matched, ie your callback will not be executed for them and they'll be treated just as normal text. This is kinda lame but it's in here because I need to be able to ignore things like <img src="http://foo.com/bar.gif"> A better implementation may happen at some point. SEE ALSO
URI::Find. AUTHOR
Kake Pugh (kake@earth.li). COPYRIGHT
Copyright (C) 2003 Kake Pugh. All Rights Reserved. This module is free software; you can redistribute it and/or modify it under the same terms as Perl itself. CREDITS
Tim Bagot helped me stop faffing over the name, by pointing out that RFC 2396 Appendix E uses "delimited". Dave Hinton helped me fix the regex to make it work for delimited URIs with no title. Nick Cleaton helped me make "ignore_quoted" work. Some of the code was taken from URI::Find. perl v5.8.8 2008-03-01 URI::Find::Delimited(3pm)
All times are GMT -4. The time now is 07:42 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy