Sponsored Content
Top Forums Shell Programming and Scripting Conditional replacement of a delimiter Post 302515314 by samahs on Tuesday 19th of April 2011 07:24:20 PM
Old 04-19-2011
Conditional replacement of a delimiter

Hello,

I'm new to this forum but this seems like the place to ask this question.

I have a pipe delimited data file with the fields except for the header being encased in double quotes. I found out that some of the fields have an trash pipe within the data itself. I'd like to conditionally remove any pipe that is not a field delimiter.

Here is some sample data
Code:
Email|DateCreated|FirstName|LastName|PostCode|Address_line1|Address_line2|City
"sample@email.com"|"2007/11/12 17:57:04"|"Keith A| Glass"|""|"20110"|""|""|""
"sample@verizo|n.net"|"2007/11/12 18:18:56"|"ESPN"|"Mi||er"|"06/25/1955"|""|""|"NY"
"sample@ao|.com"|"2007/11/12 18:20:25"|"Mary"|"Shelley"|"05/16/1845"|"James St.||Apt. 1"|""|"CT"
"samp|e@diagnostics.com"|"2007/11/12 18:20:28"|"Jam|es"|"Sample"|""|"110 Perkins Street||Apt. 6"|""|""

As you can see the header has the pipe delimited, and those are fine. But those pipes that are within the field are shifting the data when I load into the database.

Thank you for your help.
Samah

Last edited by Scott; 04-19-2011 at 10:09 PM.. Reason: Added code tags
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Replacement of Delimiter

Dear all, i have a proble. in my input file i have records with delimiter like aa-------bb------cc--vghjav---ef----kjd dj--------ih------yy--ujdjkkl---dd----jid now i want to replace the delimiter "-" with "~" i have used a command i.e cat FILENAME | tr "-" "~" >> Newfile this command... (3 Replies)
Discussion started by: panknil
3 Replies

2. Shell Programming and Scripting

Substring based on delimiter, finding last delimiter

Hi, I have a string like ABC.123.XYZ-A1-B2-P1-C4. I want to delimit the string based on "-" and then get result as only two strings. One with string till last hyphen and other with value after last hyphen... For this case, it would be something like first string as "ABC.123.XYZ-A1-B2-P1" and... (6 Replies)
Discussion started by: gupt_ash
6 Replies

3. Shell Programming and Scripting

HELP Need in SED/PERL conditional line replacement

Hi , I need some help on perl/sed conditional replacement The situation is like below . I have a file contents like below . AAA|BBB|CCC|DDD AAA|BCF|CCC|HHH AAA|BVF|JJJ|KKK Here in the above file . I know my second column value (taking "|" as my delimited ) Basically I have to... (3 Replies)
Discussion started by: robin.r888
3 Replies

4. Shell Programming and Scripting

Conditional tab replacement sed/awk

Hi I am struggling to find a solutions to this problem: I have a directory full of files and I wish to: read each line of each file and if any one line in those files is longer than 72 characters I want to replace any tab characters with a space character. Ive been... (3 Replies)
Discussion started by: benackland
3 Replies

5. Shell Programming and Scripting

conditional replacement

Hi all, I need a bash, sed, awk script or one liner to do the following task: This is the format of a text file: 2010-06-11 20:01 902656 HOP-W-100412-1.doc 2010-11-05 18:01 364447 NEX-W-101104-1 2010-07-06 10:01 64512 Cerintele 2010-07-06 10:01 599420 content 2010-07-19 14:01 1785344... (7 Replies)
Discussion started by: supervazi
7 Replies

6. Shell Programming and Scripting

selective replacement of delimiter

I have a file with two fields seperated by comma data looks like below with the header The o/p should look like this Basically, the req is to replace only the first occuring comma with pipe can we do this with any commands (2 Replies)
Discussion started by: dsravan
2 Replies

7. Shell Programming and Scripting

Shell script to put delimiter for a no delimiter variable length text file

Hi, I have a No Delimiter variable length text file with following schema - Column Name Data length Firstname 5 Lastname 5 age 3 phoneno1 10 phoneno2 10 phoneno3 10 sample data - ... (16 Replies)
Discussion started by: Gaurav Martha
16 Replies

8. Shell Programming and Scripting

Perl Code to change file delimiter (passed as argument) to bar delimiter

Hi, Extremely new to Perl scripting, but need a quick fix without using TEXT::CSV I need to read in a file, pass any delimiter as an argument, and convert it to bar delimited on the output. In addition, enclose fields within double quotes in case of any embedded delimiters. Any help would... (2 Replies)
Discussion started by: JPB1977
2 Replies

9. Shell Programming and Scripting

Conditional replacement of columns in a text file

Hello scriping expert friends, I have 2 requirements on replacing fields of text files: I have lot of data with contents like below: Requirement-1: The digit after 0 should always be changed to 1 (3 Replies)
Discussion started by: magnus29
3 Replies

10. Shell Programming and Scripting

Conditional replacement in CSV files

Hello, I have many CSV files with variable number of rows and columns. Sample of few problematic CSV files. ,,Price,Price,Price,Price,Price,Price,Price,Price,Price,Qty Date,Sl,AAA,BBB,CCC,DDD,EEE,FFF,GGG,HHH,PriQueue,%busy 30/07/2014,1,AAA,BBB,CCC,DDD,EEE,FFF,GGG,HHH,NA,0... (8 Replies)
Discussion started by: reddyr
8 Replies
Regexp::Common::delimited(3pm)				User Contributed Perl Documentation			    Regexp::Common::delimited(3pm)

NAME
Regexp::Common::delimited -- provides a regex for delimited strings SYNOPSIS
use Regexp::Common qw /delimited/; while (<>) { /$RE{delimited}{-delim=>'"'}/ and print 'a " delimited string'; /$RE{delimited}{-delim=>'/'}/ and print 'a / delimited string'; } DESCRIPTION
Please consult the manual of Regexp::Common for a general description of the works of this interface. Do not use this module directly, but load it via Regexp::Common. $RE{delimited}{-delim}{-esc} Returns a pattern that matches a single-character-delimited substring, with optional internal escaping of the delimiter. When "-delim=S" is specified, each character in the sequence S is a possible delimiter. There is no default delimiter, so this flag must always be specified. If "-esc=S" is specified, each character in the sequence S is the delimiter for the corresponding character in the "-delim=S" list. The default escape is backslash. For example: $RE{delimited}{-delim=>'"'} # match "a " delimited string" $RE{delimited}{-delim=>'"'}{-esc=>'"'} # match "a "" delimited string" $RE{delimited}{-delim=>'/'} # match /a / delimited string/ $RE{delimited}{-delim=>q{'"}} # match "string" or 'string' Under "-keep" (See Regexp::Common): $1 captures the entire match $2 captures the opening delimiter (provided only one delimiter was specified) $3 captures delimited portion of the string (provided only one delimiter was specified) $4 captures the closing delimiter (provided only one delimiter was specified) $RE{quoted}{-esc} A synonym for $RE{delimited}{q{-delim='"`}{...}} SEE ALSO
Regexp::Common for a general description of how to use this interface. AUTHOR
Damian Conway (damian@conway.org) MAINTAINANCE
This package is maintained by Abigail (regexp-common@abigail.be). BUGS AND IRRITATIONS
Bound to be plenty. For a start, there are many common regexes missing. Send them in to regexp-common@abigail.be. LICENSE and COPYRIGHT This software is Copyright (c) 2001 - 2009, Damian Conway and Abigail. This module is free software, and maybe used under any of the following licenses: 1) The Perl Artistic License. See the file COPYRIGHT.AL. 2) The Perl Artistic License 2.0. See the file COPYRIGHT.AL2. 3) The BSD Licence. See the file COPYRIGHT.BSD. 4) The MIT Licence. See the file COPYRIGHT.MIT. perl v5.14.2 2010-02-23 Regexp::Common::delimited(3pm)
All times are GMT -4. The time now is 07:21 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy