Shell Programming and Scripting

That works great on that example, it doesn't quite work on my actual file (which I can't actually post unfortunately, without subbing the data out). If we can break down the parts I'd be grateful, e.g.

Code:

/\"/&&$0!~/,/

I take the && to be AND, the $0 to refer to a whole line and !~ to be 'does not match'. So is this line identifying lines that start with a quote and do not end in ',' ? i.e. skipping lines 1 & 2?

If that's the case, unfortunately I can't use that as my actual 'field three' is the content of an article, and individual lines could conceivably end in a comma (and could even end in '",' id someone is being quoted).

Perhaps another way around this is to find an equivalent of xml's CDATA tags - a way to wrap the 'field three' text in a way that the contents are just ignored, and are allowed to pass JSON syntax rules. I don't know if that equivalent exists.

Back to the syntax:

Code:

{++p}{printf (p%2?$0" ":$0"\n")}

are these two separate statements, executed in sequence? it seems p is a var being incremented from I assume 0. The next statement is doing a mod (p%2) that I guess is deciding when to concatenate the lines, but that's where the syntax loses me.

If anyone can clarify this I'd be grateful. Thanks again for posting, yinyuemi, I do appreciate it!

Please try this way, hope it helps:

Code:

awk 'gsub("\"","\"",$0)%2{++p}{printf (p%2?$0" ":$0"\n")}' file
"one":"field one",
"two":"field two",
"three":" field three line 1  field three line 2  field three line 3 "

That's nearly it, except I need the new lines to be replaced with the explicit text '\n' in field three (so that later on, they can be replaced with <BR>). Also, do you know how I could combine that statement with sed in order to output to the original file?

In investigating the print command I found that this:

Code:

awk '{printf ($0)}' testfile.xml

nearly gives me what I want:

Code:

"one":"field one","two":"field two","three":"field three line 1 field three line 2 field three line 3"

except I need:

Code:

"one":"field one","two":"field two","three":"field three line 1 \nfield three line 2 \nfield three line 3"

N.B. I know I suggested before that I was after:

Code:

"one":"field one",
"two":"field two",
"three":"field three line 1 \nfield three line 2 \nfield three line 3"

but in fact it doesn't matter if the new line occurs at the end of a complete line (lines 1 & 2), that is OK in terms of JSON. It's when the new line occurs within a set of "" that it chokes

Code:

awk 'gsub("\"","\"",$0)%2{++p}NF{if(/\"/) {printf (p%2?$0" ":$0" \n")} else {printf (p%2?$0"\\n":$0"\n")}}' file
"one":"field one",
"two":"field two",
"three":" field three line 1 \nfield three line 2 \nfield three line 3 \n"

I think that's it! Thank you so much! It's a complex line and I'm going to put the time in to understand it. One last question is how I could combine it with sed so I can have the line in a bash script writing to the same file. I tried:

Code:

sed -i '' "`awk 'gsub("\"","\"",$0)%2{++p}NF{if(/\"/) {printf (p%2?$0" ":$0" \n")} else {printf (p%2?$0"\\n":$0"\n")}}' $filename `/" $filename

but that's not it...

Anyway, thanks again. As I said originally, I am really up against it timewise, and you have helped very much.

Hi Singerfc,

I also have no idea about it, but you can try what I suggest you before by generating a temp file, then "cat" it to replace the orignal file.

Best,

Y

no problem, yeah I'm trying that with the bash $RANDOM var for the temp file, as the operation I'm coding for could be happening often, and possibly at the same time.

thanks again, if I keep learning maybe I can help you one day

---------- Post updated 03-23-11 at 12:26 AM ---------- Previous update was 03-22-11 at 11:40 PM ----------

argh, as always there is one last detail: I need to delimit quotes (") in that third field, as in:

"field":"he said \"hello\""

as always, help appreciated