Visit Our UNIX and Linux User Community


Editing long records with characters that need to be escaped.


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Editing long records with characters that need to be escaped.
# 1  
Old 09-01-2009
Editing long records with characters that need to be escaped.

Hi all,

I'm new in unix scripting and I've a problem with a script... Smilie

I need to read a file, add some fields in the records, and write them in another file, but even when I simply read and write the records, the shell interprets some caracters and the result is that the records get splitted on multiples lines, loosing that characters.

For example:

for cur_line in `cat "$fil"`
do
echo "$cur_line"
done;

In each record I can find any type of special characters: like " ' & ; , and so on...

Maybe someone can help me... Smilie I hope I've been clear enough, although my english...
Thank a lot.
# 2  
Old 09-01-2009
Macs,

can u just put that sample file so that i can know where it's going wrong.
# 3  
Old 09-01-2009
This could be an example of a record I have to work with:

Code:
2009-07-20 17:18:03.0,"15.134.13.45.13","XXXXXX","15.134.13.45.13","xxxxxxx.xxxxxxx.IT","PIN6","SQLPLUS","PFZ5","SELECT","SIP_JDSIS3_T","IFRI","='3435354454334'","select PfrfID_ID0, substr(IFRI,1,15) IFRI2, substr(ORIGINARY_IFRI,1,15) ""TO_CHAR(CAST((( TIMESTAMP '1970-01-01 00:00:00 GMT' + numtodsinterval(ROUND(CREATED_T), 'SECOND') ) at TIME ZONE 'MET') AS DATE), 'DD/MM/YYYY HH24:MI.SS') CREATED,  TO_CHAR(CAST((( TIMESTAMP '1970-01-01 00:00:00 GMT' + numtodsinterval(ROUND(CREATION_DATE), 'SECOND') ) at TIME ZONE 'MET') AS DATE), 'DD/MM/YYYY HH24:MI.SS') CREAT_DATE,  TO_CHAR(CAST((( TIMESTAMP '1970-01-01 00:00:00 GMT' + numtodsinterval(ROUND(MOD_T), 'SECOND') ) at TIME ZONE 'MET') AS DATE), 'DD/MM/YYYY HH24:MI.SS') from SIP_IFR03_T where IORI='3653453437941'","1"

I read it from a csv file in UNIX format.
The result of the simple script example I posted before is that this informations are printed in multiple lines... and some caracters are disappearing, so I have no possibility to work with it...

I really have no idea......

Thank you!

---------- Post updated at 03:00 PM ---------- Previous update was at 02:56 PM ----------

My intention was to use the pattern "," as separator for the field and add an empty field where it needs it, with awk.

But Double Quote are disappearing...And the single record is splitted in little parts...

mmm...

Last edited by vgersh99; 09-02-2009 at 02:47 PM.. Reason: code tags, PLEASE!
# 4  
Old 09-02-2009
No ideas?
Smilie
# 5  
Old 09-02-2009
Quote:
2009-07-20 17:18:03.0,"15.134.13.45.13","XXXXXX","15.134.13.45.13","xxxxxxx.xxxxxxx.IT","PIN6","SQLPLUS","PFZ5" ,"SELECT","SIP_JDSIS3_T","IFRI","='3435354454334' ","select PfrfID_ID0, substr(IFRI,1,15) IFRI2, substr(ORIGINARY_IFRI,1,15) ""TO_CHAR(CAST((( TIMESTAMP '1970-01-01 00:00:00 GMT' + numtodsinterval(ROUND(CREATED_T), 'SECOND') ) at TIME ZONE 'MET') AS DATE), 'DD/MM/YYYY HH24:MI.SS') CREATED, TO_CHAR(CAST((( TIMESTAMP '1970-01-01 00:00:00 GMT' + numtodsinterval(ROUND(CREATION_DATE), 'SECOND') ) at TIME ZONE 'MET') AS DATE), 'DD/MM/YYYY HH24:MI.SS') CREAT_DATE, TO_CHAR(CAST((( TIMESTAMP '1970-01-01 00:00:00 GMT' + numtodsinterval(ROUND(MOD_T), 'SECOND') ) at TIME ZONE 'MET') AS DATE), 'DD/MM/YYYY HH24:MI.SS') from SIP_IFR03_T where IORI='3653453437941' ","1"

Your problem is directly related to the "^L" character. This is the ascii character for "Form feed, page eject".
Char: FF
Oct: 14
Dec: 12
Hex: c
Control-Key: ^L
# 6  
Old 09-03-2009
Thank you for the info, I'll search something about it.

Smilie

Previous Thread | Next Thread
Test Your Knowledge in Computers #222
Difficulty: Easy
NetMarketShare reported that the combined versions of Microsoft Windows lead the worldwide market with close to a 93% market share at the end of August 2011, followed by OS X at 6%, and Linux with 1%.
True or False?

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Editing long text file

Good morning all, I have a machine running IRIX and I need to edit a text file on the terminal that is literally thousands of lines. Does anyone know the most efficient way to edit portions of files like these? Obviously simply using the vi command isn't going to work since I get a too many lines... (1 Reply)
Discussion started by: James C
1 Replies

2. Windows & DOS: Issues & Discussions

How to copy a file with long records ,i.e spanning to 2 or 3 lines from UNIX to excel?

Hi Experts, I have a Unix csv file which has long records ie the record length is more than 80 so it goes to the next line.So when its in unix though it spans to two or three lines it still counts it as one record. But what is happening is for the records that are long when i copy it into excel i... (0 Replies)
Discussion started by: 100bees
0 Replies

3. Shell Programming and Scripting

Help in Editing Records of a file

Hi Friends , Need help here , I Have a file which has as a number of rows (records) in it . 00012919 7836049 S 00012920 7836049 S 00012921 3828157 Y 00012922 3828157 Y 00012923 3828157 S T005290070331000012923 I Have another file which has few more... (3 Replies)
Discussion started by: robert89
3 Replies

4. Solaris

Reason why some commands need escaped-characters and other not

Hi, this is my first post and hope to make some contribution soon. I'm still learning the basics of UNIX and Linux and BASH. Thus my need to understand the subject at hand. I don't have a problem with technical detail, so hit me :) I have a script where two commands use the contents of a... (2 Replies)
Discussion started by: doublefrangelic
2 Replies

5. Programming

How i can read a long integer from standar input and a string with as many characters as specified..

how i can read a long integer from standar input and a string with as many characters as specified in the number? i thing that i must use the read command ofcourse.... (6 Replies)
Discussion started by: aintour
6 Replies

6. Shell Programming and Scripting

Trim leading zeros to make field 6 characters long

Hi all- I've got a file that will have multiple columns. In one column there will be a string that is 10 digits in length, but I need to trim the first four zeros to make it 6 characters? example: 0000001234 0000123456 0000234566 0000000321 output: 001234 123456 234566 000321 (5 Replies)
Discussion started by: Cailet
5 Replies

7. Shell Programming and Scripting

Breaking long lines into (characters, newline, space) groups

Hello, I am currently trying to edit an ldif file. The ldif specification states that a newline followed by a space indicates the subsequent line is a continuation of the line. So, in order to search and replace properly and edit the file, I open the file in textwrangler, search for "\r " and... (14 Replies)
Discussion started by: rowie718
14 Replies

8. Shell Programming and Scripting

count characters in specific records

I have a text file which represents a http flow like this: HTTP/1.1 200 OK Date: Fri, 23 Jan 2009 17:16:24 GMT Server: Apache Last-Modified: Fri, 23 Jan 2009 17:08:03 GMT Accept-Ranges: bytes Cache-Control: max-age=540 Expires: Fri, 23 Jan 2009 17:21:31 GMT Vary: Accept-Encoding ... (1 Reply)
Discussion started by: littleboyblu
1 Replies

9. Shell Programming and Scripting

problems with sed and bash. Escaped characters ?

Hi, I'm writing a long script for bash (on RHEL 5.0) to execute many commands. So, my idea is to create a function to deal with error checking and logging (see ceckoutput() below). This works with all commands except for sed. I think it may be a problems with escaped characters. So I did the... (4 Replies)
Discussion started by: macL
4 Replies

10. Shell Programming and Scripting

How to deal with long records (> 8k)??

Still working on manipulating SQL statements. Some are very long (10-20k). I'm using a shell script to read the SQL statements that are stored in a DB2 table and writing the records out to a file on Unix. The records appear to be getting truncated at 8k. Is there any way for me to avoid the... (1 Reply)
Discussion started by: bradtri2
1 Replies

Featured Tech Videos