Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Awk/Sed - appending within file

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Awk/Sed - appending within file
# 1  
Old 03-21-2011
Awk/Sed - appending within file
hello all,

First time post here. I have searched a bit but could not find an exact answer. I have about a week's experience of Sed and Awk, and am having fun, but am a little stuck.

I am reformatting an xml file into json format. I have got this far:
Code:
{"clients":[
"articles":[
"File_URL":"http://url.com/a/123",
"pub_date":"2011-03-11",
"publication":"Irish Times",
"publication_type":"D",
"headline":"Something happening somewhere",
"page":"9",
"circulation":"0105742.0000",
"area":"83330.00",
"author":"John Doe",
"last_edited":"2011-03-11 05:33:25",
"client":"Company Ltd",
"client":"MegaCorp plc",
"client":"Dept of Departments",

...and so on. What I want is to append the client names up to line 1, then close it off with '],' to follow the correct json structure. I have the following awk script:
Code:
BEGIN {
    clientlist = ""
    FS=":"
}
/\"client\"/ { clientlist = clientlist $2 }
END { print clientlist }

which prints exactly what I need to standard output, but I don't know how to append it to line 1 within the file. I also don't know whether I would be better off using sed for this part of the process (I have a bash script with a number of sed statements that process the file up to this point).

I am enjoying learning sed and awk very much but unfortunately time is not on my side.

Any help much appreciated.
Last edited by Franklin52; 03-23-2011 at 11:11 AM.. Reason: Please use code tags
# 2  
Old 03-21-2011
how about this?
Code:
awk 'BEGIN {
clientlist = ""
FS=":"}
{v=v?v RS $0:$0}
/client/ { clientlist = clientlist $2 }
END { print clientlist"]" RS v }
'
file

This User Gave Thanks to yinyuemi For This Post:
singerfc
# 3  
Old 03-21-2011
hi yinyuemi,

Thanks for the help. That script does write to line 1 so is a big help, however it bumps everything else down. I'm looking to append it to the existing line 1. Can I ask what your added code:
Code:
{v=v?v RS $0:$0}

and
Code:
RS v

is doing?

Also, how would I put that script into a bash script, where the source/target file is referred to as $1? That's where I currently have all my sed statements, but it seemed to choke at the awk part, though it worked from the command line.

Thanks, very much
Last edited by Franklin52; 03-23-2011 at 11:11 AM.. Reason: Please use code tags
# 4  
Old 03-21-2011
Do you mean this:
Code:
awk 'BEGIN {
clientlist = ""
FS=":"}
NR==1{v=$1}NR>1{x=x?x RS $0:$0}
/client/ { clientlist = clientlist $2 }
END { print v FS clientlist"]" RS x }
' file
{clients:[Company Ltd,MegaCorp plc,Dept of Departments,]
articles:[
File_URL:http://url.com/a/123,
pub_date:2011-03-11,
publication:Irish Times,
publication_type:D,
headline:Something happening somewhere,
page:9,
circulation:0105742.0000,
area:83330.00,
author:John Doe,
last_edited:2011-03-11 05:33:25,
client:Company Ltd,
client:MegaCorp plc,
client:Dept of Departments,

---------- Post updated at 07:53 PM ---------- Previous update was at 07:50 PM ----------

Code:
v is a variable, RS is the Record Separator Variable
 {v=v?v RS $0:$0} == {if(v) {v=v RS $0} else {v= $0}}


Quote:
Originally Posted by singerfc
hi yinyuemi,

Thanks for the help. That script does write to line 1 so is a big help, however it bumps everything else down. I'm looking to append it to the existing line 1. Can I ask what your added code:

{v=v?v RS $0:$0}

and

RS v

is doing?

Thanks, very much
# 5  
Old 03-21-2011
Yinyuemi,

That's it! It was printing to stout, so I added '> outputfile.txt' at the end, and I got a new file with the text appended correctly. Is there something I can do to append the results to the input file? I tried '>' to the input filename, but I lost most of the content.

Thanks again!
# 6  
Old 03-21-2011
you can do like this:
Code:
awk 'CODE' inputfile >temp; cat temp >inputfile; rm temp

This User Gave Thanks to yinyuemi For This Post:
singerfc
# 7  
Old 03-21-2011
ah OK, I wasn't sure if it could be done in one step.

Thanks for all your help, much appreciated.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Reading and appending a row from file1 to file2 using awk or sed

Hi, I wanted to add each row of file2.txt to entire length of file1.txt given the sample data below and save it as new file. Any idea how to efficiently do it. Thank you for any help. input file file1.txt file2.txt 140 30 200006 141 32 140 32 200006 142 33 140 35 200006 142... (5 Replies)
Discussion started by: ida1215
5 Replies

2. UNIX for Dummies Questions & Answers

Appending columns at the end of output using awk/sed

Hi , I have the below ouput, =====gopi===== assasassaa adsadsadsdsada asdsadsadasdsa sadasdsadsd =====kannan=== asdasdasd sadasddsaasd adasdd =====hbk=== asasasssa .... .. I want the output like as below, not able paste here correctly. (2 Replies)
Discussion started by: eeegopikannan
2 Replies

3. Shell Programming and Scripting

Appending string (charachters inside the line) to a fixed width file using awk or sed

Source File: abcdefghijklmnop01qrstuvwxyz abcdefghijklmnop02qrstuvwxyz abcdefghijklmnop03qrstuvwxyz abcdefghijklmnop04qrstuvwxyz abcdefghijklmnop05qrstuvwxyz Whatever characters are in 17-18 on each line of the file, it should be concatenated to the same line at the character number... (6 Replies)
Discussion started by: tamahomekarasu
6 Replies

4. Shell Programming and Scripting

awk;sed appending line to previous line....

I know this has been asked before but I just can't parse the syntax as explained. I have a set of files that has user information spread out over two lines that I wish to merge into one: User1NameLast User1NameFirst User1Address E-Mail:User1email User2NameLast User2NameFirst User2Address... (11 Replies)
Discussion started by: walkerwheeler
11 Replies

5. Shell Programming and Scripting

appending a file using sed in ksh

i am trying to append a 5 line SGML file(file1) with a 500,000 line SGML file (file2). file1 is a template, so i wish to preserve. i only want to add lines 5 to the end of file2. i have: cp file1 temp1 sed -n '5,$p' file2 >> temp1 when i check the tail of temp1, i consistantly find the... (3 Replies)
Discussion started by: smac
3 Replies

6. UNIX for Dummies Questions & Answers

SED or AWK: Appending a line Identifier that changes with paragraph?

Have another question that has been eluding me all day. I have data file I'm trying to reformat so that each line is appended with an ID code, but the ID code needs to update as it searches through the file. I.e. ----Begin Original Datafile----- Condition = XXX Header Line 1 Header... (1 Reply)
Discussion started by: selkirk
1 Replies

7. Shell Programming and Scripting

appending and sed

Hello, I want to add string #REAL at the end of all lines that contain real numbers. How to do this using sed ? (1 Reply)
Discussion started by: scotty_123
1 Replies

8. Shell Programming and Scripting

appending to sed output of one file into the middle of file

hi, i have a file file1 file2 ----------- ----------------- aa bbb ccc 111 1111 1111 ddd eee fff 222 3333 4444 ggg hhh... (5 Replies)
Discussion started by: go4desperado
5 Replies

9. Shell Programming and Scripting

Appending string to xml file using "sed"

Hi folks, Following a section in opmn.xml file: </process-type> <process-type id="OC4J_RiGHTv_IRD1" module-id="OC4J"> <environment> <variable id="LD_LIBRARY_PATH" value="/home/ias/v10.1.2/lib" append="true"/> <variable id="SHLIB_PATH"... (2 Replies)
Discussion started by: nir_s
2 Replies

10. UNIX for Dummies Questions & Answers

SED Question -- on appending to a file

:confused: I have a script that Cats a flat database file which contains 12 columns into sed. I want to add a 13th column which includes " ,2005-08-29 " * The date needs to be the current date. This 13th column would be appended to the end of each line. Does anyone have a clue... (5 Replies)
Discussion started by: Redg
5 Replies
Login or Register to Ask a Question