UNIX for Beginners Questions & Answers

Hi All

I am facing some issues while processing a file.
The file has \n as End Of Line. However for some records '\n' is also present as part of string. Hence I am not able to identify proper records from file.
File is # delimited.
Example:
2 rows spread across 4 lines.

Code:

line1: COL1#COL2#[NEWLINE]
line2:LINE WITH "\n" in it.[NEWLINE]
line3:Another line[NEWLINE]
line4:COL1#COL2#COL3[NEWLINE]

When I am reading above file, I am getting 4 rows. However it should be 2 only.
Now I want to process the file so that output looks like
2 rows spread across 2 lines only

Code:

COL1#COL2#LINE WITH "\n" in it.Another line[NEWLINE]
COL1#COL2#COL3[NEWLINE]

Just FYI: COL1,COL2,COL3 are generic and it can be anything. Also, I know in which column new lines may come.

TIA
Nitin

If COL1, COL2,... are anything how can you determine that they make one line?

There must be some thought process you have developed to tell when a \n character is not really the end of the line. Please specify it. Like what defines a column? columns==one word?
columns are delimited by quotes?

Hi Jim

Thanks for quick reply.
I am exporting data from RDBMS table into flat file with FIELDS TERMINATED BY '#' LINES TERMINATED BY '\n'.
However in the file there are few strings which have \n as the value. So a string ideally should be in a single line however it may spread to multiple lines with \n as value inside string.
Eg:

Code:

THIS IS SINGLE LINE STRING

Code:

THIS IS
MULTI-LINE
STRING SEPARATED BY
NEWLINE identifier.

Now the problem is when I m reading data in HIVE, I cannot read this string completely because it encounters \n and truncates string at that point. Remaining string is treated as another line.

is there I way I can handle this situation by processing file. I don't want to export data again from RDBMS to file.

Or in other words how do we process file, when string has delimiter as the genuine value.

Thanks
Nitin