Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Parsing line with name:value pairs in shell script

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Parsing line with name:value pairs in shell script
# 1  
Old 04-03-2013
Parsing line with name:value pairs in shell script
HI ,
I have the following type of lines in a file and need to create a csv file that can be bcp'ed into a db

The problem that I have is the delimited of the <name :value> is a space but some of the values in the pairs have space . eg msg_src_time:03/05/13 10:40:17.919

Need sugesstions on parsing the file

sample input

INPUT
Code:
msg_status:FILL comments: usr_id:73 system_time:03/05/13 10:40:17.920 msg_src_time:03/05/13 10:40:17.919 flags: 0 flags2: 0 comment_index: 0
msg_status:FILL comments:dollar usr_id:73 system_time:03/05/13 10:40:51.352 msg_src_time:03/05/13 10:40:51.352 flags: 0 flags2: 0 comment_index: 0

Output
Code:
FILL ,  , 73 , 03/05/13 10:40:17.920 , :03/05/13 10:40:17.919 , 0 , 0 , 0
FILL , dollar , 73 , 03/05/13 10:40:51.352 , :03/05/13 10:40:51.352 , 0 , 0 , 0

My intial search in this forum found these but this will fail as my delimited is a space and 2 name value pairs have space in them

Code:
echo "Type:G,Instance:instance1,FunctionalID:funcid,Env:dev,AppName:appname" | sed 's/:/=/g' | awk 'BEGIN{FS=","} { print $2 print $3 print $4 print $5 }

Last edited by tasmac; 04-03-2013 at 04:21 PM.. Reason: code tags
# 2  
Old 04-03-2013
How about:
Code:
awk -F'[[:alnum:]]*[_[:alpha:]]+[[:alnum:]]*:' '{$1=$1; sub(OFS,x)}1' OFS=" , " file

output:
Code:
FILL  ,   , 73  , 03/05/13 10:40:17.920  , 03/05/13 10:40:17.919  ,  0  ,  0  ,  0
FILL  , dollar  , 73  , 03/05/13 10:40:51.352  , 03/05/13 10:40:51.352  ,  0  ,  0  ,  0

This User Gave Thanks to Scrutinizer For This Post:
tasmac
# 3  
Old 04-03-2013
I get the following error when i run the above line
Code:
>awk -F'[[:alnum:]]*[_[:alpha:]]+[[:alnum:]]*:' '{$1=$1; sub(OFS,x)}1' OFS=" , " in.trade
awk: syntax error near line 1
awk: illegal statement near line 1
awk: syntax error near line 1
awk: bailing out near line 1

# 4  
Old 04-03-2013
Code:
awk -F"[a-z_]+([0-9])?:" '{for(i = 1; i <= NF; i++) printf("%s", $i == "" ? $i : $i "" (i == NF ? "\n" : ","))}' file

# 5  
Old 04-03-2013
I get a similar error . I am using bash and other awk commands work

Code:
$>awk -F"[a-z_]+([0-9])?:" '{for(i = 1; i <= NF; i++) printf("%s",  $i == "" ? $i : $i "" (i == NF ? "\n" : ","))}' in.trade
awk: syntax error near line 1
awk: illegal statement near line 1

# 6  
Old 04-03-2013
What is your OS?
This User Gave Thanks to shamrock For This Post:
tasmac
# 7  
Old 04-03-2013
Code:
$>uname -a
SunOS gatxdev2 5.10 Generic_147441-19 i86pc i386 i86pc
$>awk 1 /dev/null
awk: syntax error near line 1
awk: bailing out near line 1
$>

Looks like I am using the old awk

according to Unix shell - View topic - How to find the version of awk?
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

File w/ many line pairs--how do I order based on 1st lines only?

I have a file in which the data is stored in pairs of lines. The first line (beginining with ">") is a header, the second line is a sequence. I would like to sort the file by species name. Desired output for the example file: I can use sort -t'_' -k2 to alphabetize headers in the... (1 Reply)
Discussion started by: pathunkathunk
1 Replies

2. Shell Programming and Scripting

Parsing Command Line Arguments In C shell script

]I have a string like "/abc/cmind/def/pq/IC.2.4.6_main.64b/lnx86" and this string is given by user. But in this string instead of 64b user may passed 32 b an i need to parse this string and check wether its is 32b or 64 b and according to it i want to set appropriate flags. How will i do this... (11 Replies)
Discussion started by: codecatcher
11 Replies

3. UNIX for Dummies Questions & Answers

Remove Duplicate Two Line Pairs?

So I have a bunch of files that look like this >gi|33332323 MMKCRGVIMVVEKVMKRDGRIVPFDESRIRWAVQ--- >gi|45235353 MMKCR----VEKMRDVFFDESIRWAVQ They go on...sequences are much longer but all in two line (fasta) format. I want to remove duplicate pairs of ID(GI) number and sequence. I tried... (12 Replies)
Discussion started by: bakere19
12 Replies

4. Shell Programming and Scripting

Parsing a command line parameter in script

I have a simple script that builds a complex program call which passes a number of parameters to the program. I'm trying to enhance the script to include the value of the command line parameter in the name of a file being created. The problem I'm having is that the parameter may include a forward... (11 Replies)
Discussion started by: pbmax626
11 Replies

5. Shell Programming and Scripting

parsing using shell script

I have a file parameters.txt which contains 151524 151525 I have another file OID.csv which contains NE Version Object Type ID SDK param name Object OID test1 Start: 4.2 End: 4.2 pan 151524 speed ... (5 Replies)
Discussion started by: LavanyaP
5 Replies

6. Shell Programming and Scripting

Searching a delimited Key value pairs in shell script

Hello, I have property file with key value pairs separated by pipe , I am trying to write a script which reads the property file and search and print value of specific key. I tried with Sed, I am successfull. The file is as follows ... (4 Replies)
Discussion started by: ANK
4 Replies

7. Shell Programming and Scripting

Matching Pairs with AWK or Shell

Hello, I have a file Pairs.txt composed of 3 columns. For simplicity, this is the file: A B C B D W X Y Z Z W K B A C Y J Q F M P Y X Z I am trying to reduce the file such that I only keep the first occurrence of any "pair". In the example above, I would only want to keep one... (6 Replies)
Discussion started by: InfoSeeker
6 Replies

8. Shell Programming and Scripting

Parsing /proc/net/dev into key:value pairs (self-answered)

Hi all, I need some help with using sed/awk/other linux tools to meet the following goal: I'm trying to take the output of /proc/net/dev: Inter-| Receive | Transmit face |bytes packets errs drop fifo frame compressed multicast|bytes ... (0 Replies)
Discussion started by: felbane
0 Replies

9. Shell Programming and Scripting

Parsing a line in Shell Script

I have record line somthing like below with first line showing char spacing not real record line 1 | 2 | 3rd Field--------------|-4th field--| This is charcatersapcing of line DF20000000000000000130.7890000000750 I shoudl get two line from above line 1st line should 1 | 2 | 3rd... (3 Replies)
Discussion started by: unishiva
3 Replies

10. UNIX for Dummies Questions & Answers

Parsing text from one line with shell scripts

Hi Gurus! I wonder if anyone can help me, I'm sure you guys can. I have a text file which contains a lot of data on the one line as follows: $ What I need to do is pull all of those id values out (eg 2549425) and write them to a list in a text file. Any help would be greatly... (3 Replies)
Discussion started by: th3g0bl1n
3 Replies
Login or Register to Ask a Question