How to remove the delimiter from the column value within a file?


 
Thread Tools Search this Thread
Top Forums UNIX for Advanced & Expert Users How to remove the delimiter from the column value within a file?
# 8  
Old 09-27-2016
It is the cat -v that makes ^@ ambiguous. If you want to change null bytes in a file to pipe symbols, use tr; not cat and sed:
Code:
tr '\0' '|' < raw_file.dat > parsed_raw_file.dat

But, of course, if your data files contain other null bytes (such as in numeric values stored as type int or float instead of in strings), you will need something that actually parses the format of your .dat files and only changes the null bytes that are field delimiters in your data to pipe symbols.
# 9  
Old 09-27-2016
Hi,
The tr command is THE solution is this case : it can translate easily any character like NUL , which is \0 for unix.
See :
( od -c is just here to show where the NUL is )

Code:
$ od -c n.tmp
0000000   a   a   a   a   \  \0   b   b   b  \n
0000012

$ tr '\0' '|' < n.tmp > n.txt

$ od -c n.txt
0000000   a   a   a   a   \   |   b   b   b  \n
0000012

# 10  
Old 10-04-2016
Hi,
You can just use tr to replace null chars with anything you want , e.g. |
See:
Code:
$ od -c 1.tmp
0000000   w   e       h   a   v   e       n   u   l   l      \0  \0  \0
0000020       c   h   a   r   s  \n
0000027

$ tr \\000 \| < 1.tmp
we have null ||| chars

Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Replace delimiter for a particular column in a pipe delimited file

I have an input file as below Emp1|FirstName|MiddleName|LastName|Address|Pincode|PhoneNumber 1234|FirstName1|MiddleName2|LastName3| Add1 || ADD2|123|000000000 Output : 1234|FirstName1|MiddleName2|LastName3| Add1 ,, ADD2|123|000000000 OR 1234,FirstName1,MiddleName2,LastName3, Add1 ||... (2 Replies)
Discussion started by: styris
2 Replies

2. Shell Programming and Scripting

Perl Code to change file delimiter (passed as argument) to bar delimiter

Hi, Extremely new to Perl scripting, but need a quick fix without using TEXT::CSV I need to read in a file, pass any delimiter as an argument, and convert it to bar delimited on the output. In addition, enclose fields within double quotes in case of any embedded delimiters. Any help would... (2 Replies)
Discussion started by: JPB1977
2 Replies

3. Shell Programming and Scripting

Remove the values from a certain column without deleting the Column name in a .CSV file

(14 Replies)
Discussion started by: dhruuv369
14 Replies

4. Shell Programming and Scripting

Shell script to put delimiter for a no delimiter variable length text file

Hi, I have a No Delimiter variable length text file with following schema - Column Name Data length Firstname 5 Lastname 5 age 3 phoneno1 10 phoneno2 10 phoneno3 10 sample data - ... (16 Replies)
Discussion started by: Gaurav Martha
16 Replies

5. Shell Programming and Scripting

How to avoid Delimiter occuring in column values in .csv file

Hello Gurus, I need to create a file from a .csv file extracting specific columns only. File structure is Column1,Column2,Column3,Column4 abcd,1234,"asdf, tew,123",123456 efgh,234,asdf,654321 My output file should have abcd,123456 efgh,654321 Can you pls help me with the code. ... (10 Replies)
Discussion started by: ritesh.bhawsar
10 Replies

6. Shell Programming and Scripting

How to remove delimiter from specific column?

I have 5 column in sample txt file where in i have to create report based upon 1,3 and 5 th column.. I have : in first and third coulmn. But I want to retain the colon of fifth coulmn and remove the colon of first column.. 5th column contains String message (for example,... (7 Replies)
Discussion started by: Shirisha
7 Replies

7. Shell Programming and Scripting

rearrange the column names with comma as column delimiter

Hi, I am new to shell scripting, i have requirement can any one help me out in this regrads, in directory i have file like invoice1.txt, invoice2.txt in each file i have fixed number of columns, 62 in number but they are randomly arranged.like for first file invoice1.txt can have columns... (5 Replies)
Discussion started by: madhav62
5 Replies

8. Shell Programming and Scripting

Remove first column from file

Hi, This is how data in test.txt file | |abc|zxcv|xy12| | |cvs|zzvc|a23p| How can remove first column. abc|zxcv|xy12| cvs|zzvc|a23p| Thanks srimitta (8 Replies)
Discussion started by: srimitta
8 Replies

9. Shell Programming and Scripting

How to read the first column in a flat file with ~ as delimiter

I have one flat file like below id1~col~batch1 id2~col2~batch2 id3~col3~batch3 I need to read the first column one by one and I need to write one db2 query based on that column1 Like for (i=0;i<=10;i++) do insert into table column (con_id) values (select column from table where... (4 Replies)
Discussion started by: siri_886
4 Replies
Login or Register to Ask a Question