Sponsored Content
Top Forums Shell Programming and Scripting Conditional replacement of a delimiter Post 302515314 by samahs on Tuesday 19th of April 2011 07:24:20 PM
Old 04-19-2011
Conditional replacement of a delimiter

Hello,

I'm new to this forum but this seems like the place to ask this question.

I have a pipe delimited data file with the fields except for the header being encased in double quotes. I found out that some of the fields have an trash pipe within the data itself. I'd like to conditionally remove any pipe that is not a field delimiter.

Here is some sample data
Code:
Email|DateCreated|FirstName|LastName|PostCode|Address_line1|Address_line2|City
"sample@email.com"|"2007/11/12 17:57:04"|"Keith A| Glass"|""|"20110"|""|""|""
"sample@verizo|n.net"|"2007/11/12 18:18:56"|"ESPN"|"Mi||er"|"06/25/1955"|""|""|"NY"
"sample@ao|.com"|"2007/11/12 18:20:25"|"Mary"|"Shelley"|"05/16/1845"|"James St.||Apt. 1"|""|"CT"
"samp|e@diagnostics.com"|"2007/11/12 18:20:28"|"Jam|es"|"Sample"|""|"110 Perkins Street||Apt. 6"|""|""

As you can see the header has the pipe delimited, and those are fine. But those pipes that are within the field are shifting the data when I load into the database.

Thank you for your help.
Samah

Last edited by Scott; 04-19-2011 at 10:09 PM.. Reason: Added code tags
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Replacement of Delimiter

Dear all, i have a proble. in my input file i have records with delimiter like aa-------bb------cc--vghjav---ef----kjd dj--------ih------yy--ujdjkkl---dd----jid now i want to replace the delimiter "-" with "~" i have used a command i.e cat FILENAME | tr "-" "~" >> Newfile this command... (3 Replies)
Discussion started by: panknil
3 Replies

2. Shell Programming and Scripting

Substring based on delimiter, finding last delimiter

Hi, I have a string like ABC.123.XYZ-A1-B2-P1-C4. I want to delimit the string based on "-" and then get result as only two strings. One with string till last hyphen and other with value after last hyphen... For this case, it would be something like first string as "ABC.123.XYZ-A1-B2-P1" and... (6 Replies)
Discussion started by: gupt_ash
6 Replies

3. Shell Programming and Scripting

HELP Need in SED/PERL conditional line replacement

Hi , I need some help on perl/sed conditional replacement The situation is like below . I have a file contents like below . AAA|BBB|CCC|DDD AAA|BCF|CCC|HHH AAA|BVF|JJJ|KKK Here in the above file . I know my second column value (taking "|" as my delimited ) Basically I have to... (3 Replies)
Discussion started by: robin.r888
3 Replies

4. Shell Programming and Scripting

Conditional tab replacement sed/awk

Hi I am struggling to find a solutions to this problem: I have a directory full of files and I wish to: read each line of each file and if any one line in those files is longer than 72 characters I want to replace any tab characters with a space character. Ive been... (3 Replies)
Discussion started by: benackland
3 Replies

5. Shell Programming and Scripting

conditional replacement

Hi all, I need a bash, sed, awk script or one liner to do the following task: This is the format of a text file: 2010-06-11 20:01 902656 HOP-W-100412-1.doc 2010-11-05 18:01 364447 NEX-W-101104-1 2010-07-06 10:01 64512 Cerintele 2010-07-06 10:01 599420 content 2010-07-19 14:01 1785344... (7 Replies)
Discussion started by: supervazi
7 Replies

6. Shell Programming and Scripting

selective replacement of delimiter

I have a file with two fields seperated by comma data looks like below with the header The o/p should look like this Basically, the req is to replace only the first occuring comma with pipe can we do this with any commands (2 Replies)
Discussion started by: dsravan
2 Replies

7. Shell Programming and Scripting

Shell script to put delimiter for a no delimiter variable length text file

Hi, I have a No Delimiter variable length text file with following schema - Column Name Data length Firstname 5 Lastname 5 age 3 phoneno1 10 phoneno2 10 phoneno3 10 sample data - ... (16 Replies)
Discussion started by: Gaurav Martha
16 Replies

8. Shell Programming and Scripting

Perl Code to change file delimiter (passed as argument) to bar delimiter

Hi, Extremely new to Perl scripting, but need a quick fix without using TEXT::CSV I need to read in a file, pass any delimiter as an argument, and convert it to bar delimited on the output. In addition, enclose fields within double quotes in case of any embedded delimiters. Any help would... (2 Replies)
Discussion started by: JPB1977
2 Replies

9. Shell Programming and Scripting

Conditional replacement of columns in a text file

Hello scriping expert friends, I have 2 requirements on replacing fields of text files: I have lot of data with contents like below: Requirement-1: The digit after 0 should always be changed to 1 (3 Replies)
Discussion started by: magnus29
3 Replies

10. Shell Programming and Scripting

Conditional replacement in CSV files

Hello, I have many CSV files with variable number of rows and columns. Sample of few problematic CSV files. ,,Price,Price,Price,Price,Price,Price,Price,Price,Price,Qty Date,Sl,AAA,BBB,CCC,DDD,EEE,FFF,GGG,HHH,PriQueue,%busy 30/07/2014,1,AAA,BBB,CCC,DDD,EEE,FFF,GGG,HHH,NA,0... (8 Replies)
Discussion started by: reddyr
8 Replies
io_pipe(3)						     Library Functions Manual							io_pipe(3)

NAME
io_pipe - create a Unix pipe SYNTAX
#include <io.h> int io_pipe(int64 pfd[2]); DESCRIPTION
io_pipe creates a new UNIX ``pipe.'' The pipe can receive data and provide data; any bytes written to the pipe can then be read from the pipe in the same order. A pipe is typically stored in an 8192-byte memory buffer; the exact number depends on the UNIX kernel. Bytes are written to the end of the buffer and read from the beginning of the buffer. Once a byte has been read, it is eliminated from the buffer, making space for another byte to be written; readers cannot ``rewind'' a pipe to read old data. Once 8192 bytes have been written to the buffer, the pipe will not be ready for further writing until some of the bytes have been read. Once all the bytes written have been read, the pipe will not be ready for further reading until more bytes are written. io_pipe sets d[0] to the number of a new descriptor reading from the pipe, and sets d[1] to the number of a new descriptor writing to the pipe. It then returns 1 to indicate success. If something goes wrong, io_pipe returns 0, setting errno to indicate the error; in this case it frees any memory that it allocated for the new pipe, and it leaves d alone. SEE ALSO
io_readfile(3), io_createfile(3), io_socketpair(3) io_pipe(3)
All times are GMT -4. The time now is 11:39 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy