Sponsored Content
Top Forums Shell Programming and Scripting Conditional replacement of a delimiter Post 302515314 by samahs on Tuesday 19th of April 2011 07:24:20 PM
Old 04-19-2011
Conditional replacement of a delimiter

Hello,

I'm new to this forum but this seems like the place to ask this question.

I have a pipe delimited data file with the fields except for the header being encased in double quotes. I found out that some of the fields have an trash pipe within the data itself. I'd like to conditionally remove any pipe that is not a field delimiter.

Here is some sample data
Code:
Email|DateCreated|FirstName|LastName|PostCode|Address_line1|Address_line2|City
"sample@email.com"|"2007/11/12 17:57:04"|"Keith A| Glass"|""|"20110"|""|""|""
"sample@verizo|n.net"|"2007/11/12 18:18:56"|"ESPN"|"Mi||er"|"06/25/1955"|""|""|"NY"
"sample@ao|.com"|"2007/11/12 18:20:25"|"Mary"|"Shelley"|"05/16/1845"|"James St.||Apt. 1"|""|"CT"
"samp|e@diagnostics.com"|"2007/11/12 18:20:28"|"Jam|es"|"Sample"|""|"110 Perkins Street||Apt. 6"|""|""

As you can see the header has the pipe delimited, and those are fine. But those pipes that are within the field are shifting the data when I load into the database.

Thank you for your help.
Samah

Last edited by Scott; 04-19-2011 at 10:09 PM.. Reason: Added code tags
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Replacement of Delimiter

Dear all, i have a proble. in my input file i have records with delimiter like aa-------bb------cc--vghjav---ef----kjd dj--------ih------yy--ujdjkkl---dd----jid now i want to replace the delimiter "-" with "~" i have used a command i.e cat FILENAME | tr "-" "~" >> Newfile this command... (3 Replies)
Discussion started by: panknil
3 Replies

2. Shell Programming and Scripting

Substring based on delimiter, finding last delimiter

Hi, I have a string like ABC.123.XYZ-A1-B2-P1-C4. I want to delimit the string based on "-" and then get result as only two strings. One with string till last hyphen and other with value after last hyphen... For this case, it would be something like first string as "ABC.123.XYZ-A1-B2-P1" and... (6 Replies)
Discussion started by: gupt_ash
6 Replies

3. Shell Programming and Scripting

HELP Need in SED/PERL conditional line replacement

Hi , I need some help on perl/sed conditional replacement The situation is like below . I have a file contents like below . AAA|BBB|CCC|DDD AAA|BCF|CCC|HHH AAA|BVF|JJJ|KKK Here in the above file . I know my second column value (taking "|" as my delimited ) Basically I have to... (3 Replies)
Discussion started by: robin.r888
3 Replies

4. Shell Programming and Scripting

Conditional tab replacement sed/awk

Hi I am struggling to find a solutions to this problem: I have a directory full of files and I wish to: read each line of each file and if any one line in those files is longer than 72 characters I want to replace any tab characters with a space character. Ive been... (3 Replies)
Discussion started by: benackland
3 Replies

5. Shell Programming and Scripting

conditional replacement

Hi all, I need a bash, sed, awk script or one liner to do the following task: This is the format of a text file: 2010-06-11 20:01 902656 HOP-W-100412-1.doc 2010-11-05 18:01 364447 NEX-W-101104-1 2010-07-06 10:01 64512 Cerintele 2010-07-06 10:01 599420 content 2010-07-19 14:01 1785344... (7 Replies)
Discussion started by: supervazi
7 Replies

6. Shell Programming and Scripting

selective replacement of delimiter

I have a file with two fields seperated by comma data looks like below with the header The o/p should look like this Basically, the req is to replace only the first occuring comma with pipe can we do this with any commands (2 Replies)
Discussion started by: dsravan
2 Replies

7. Shell Programming and Scripting

Shell script to put delimiter for a no delimiter variable length text file

Hi, I have a No Delimiter variable length text file with following schema - Column Name Data length Firstname 5 Lastname 5 age 3 phoneno1 10 phoneno2 10 phoneno3 10 sample data - ... (16 Replies)
Discussion started by: Gaurav Martha
16 Replies

8. Shell Programming and Scripting

Perl Code to change file delimiter (passed as argument) to bar delimiter

Hi, Extremely new to Perl scripting, but need a quick fix without using TEXT::CSV I need to read in a file, pass any delimiter as an argument, and convert it to bar delimited on the output. In addition, enclose fields within double quotes in case of any embedded delimiters. Any help would... (2 Replies)
Discussion started by: JPB1977
2 Replies

9. Shell Programming and Scripting

Conditional replacement of columns in a text file

Hello scriping expert friends, I have 2 requirements on replacing fields of text files: I have lot of data with contents like below: Requirement-1: The digit after 0 should always be changed to 1 (3 Replies)
Discussion started by: magnus29
3 Replies

10. Shell Programming and Scripting

Conditional replacement in CSV files

Hello, I have many CSV files with variable number of rows and columns. Sample of few problematic CSV files. ,,Price,Price,Price,Price,Price,Price,Price,Price,Price,Qty Date,Sl,AAA,BBB,CCC,DDD,EEE,FFF,GGG,HHH,PriQueue,%busy 30/07/2014,1,AAA,BBB,CCC,DDD,EEE,FFF,GGG,HHH,NA,0... (8 Replies)
Discussion started by: reddyr
8 Replies
STRSEP(3)						   BSD Library Functions Manual 						 STRSEP(3)

NAME
strsep, stresep -- separate strings LIBRARY
Standard C Library (libc, -lc) SYNOPSIS
#include <string.h> char * strsep(char **stringp, const char *delim); char * stresep(char **stringp, const char *delim, int escape); DESCRIPTION
The strsep() function locates, in the nul-terminated string referenced by *stringp, the first occurrence of any character in the string delim (or the terminating '' character) and replaces it with a ''. The location of the next character after the delimiter character (or NULL, if the end of the string was reached) is stored in *stringp. The original value of *stringp is returned. An ``empty'' field, i.e., one caused by two adjacent delimiter characters, can be detected by comparing the location referenced by the pointer returned by strsep() to ''. If *stringp is initially NULL, strsep() returns NULL. The stresep() function also takes an escape character that allows quoting the delim- iter character so that it can be part of the source string. EXAMPLES
The following uses strsep() to parse a string, containing tokens delimited by white space, into an argument vector: char **ap, *argv[10], *inputstring; for (ap = argv; ap < &argv[9] && (*ap = strsep(&inputstring, " ")) != NULL;) { if (**ap != '') ap++; } HISTORY
The strsep() function is intended as a replacement for the strtok() function. While the strtok() function should be preferred for portabil- ity reasons (it conforms to ANSI X3.159-1989 (``ANSI C89'')) it is unable to handle empty fields, i.e., detect fields delimited by two adja- cent delimiter characters, or to be used for more than a single string at a time. The strsep() function first appeared in 4.4BSD. BSD
August 12, 2006 BSD
All times are GMT -4. The time now is 02:44 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy