Login or Register to Ask a Question and Join Our Community


UNIX for Beginners Questions & Answers

Json field grap via shell script/awk

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums UNIX for Beginners Questions & Answers Json field grap via shell script/awk
# 1  
Old 02-23-2017
Json field grap via shell script/awk
i have a json data that looks like this:

Code:
{
  "ip": "16.66.35.10",
  "hostname": "No Hostname",
  "city": "Stepney",
  "region": "England",
  "country": "GB",
  "loc": "51.57,-0.0333",
  "org": "AS6871 British Telecommunications PLC",
  "postal": "E1"
}

im looking for a way to assign each value from the above to a variable without having to make multiple external calls to any particular tool like awk (although i love awk).

if awk can be used for this, i dont mind it.

but here's what im currently doing:


Code:
IpInfo=$(cat jsonfile | tr -d '\n')

ip=$(echo "${IpInfo}" | awk -F"," '{print $1}')
hname=$(echo "${IpInfo}" | awk -F"," '{print $2}')
city=$(echo "${IpInfo}" | awk -F"," '{print $3}')
region=$(echo "${IpInfo}" | awk -F"," '{print $4}')
country=$(echo "${IpInfo}" | awk -F"," '{print $5}')
location=$(echo "${IpInfo}" | awk -F"," '{print $6}')
organiz=$(echo "${IpInfo}" | awk -F"," '{print $7}')
postal=$(echo "${IpInfo}" | awk -F"," '{print $8}')

as you can see, this is very bad as im making 8 calls to awk.

how can i achieve what im trying to do?
# 2  
Old 02-23-2017
In bash / ksh93 / zsh try something like this :
Code:
while IFS=\" read x key x val x; do
  [ -n "$key" ] && read "$key" <<< "$val"
done < jsonfile




--
With regular POSIX shell, you could try something like this:
Code:
while IFS=\" read x key x val x
do
  if [ -n "$key" ]; then
    read "$key" << EOF
      $val
EOF
  fi
done < jsonfile

Last edited by Scrutinizer; 02-23-2017 at 03:51 PM..
This User Gave Thanks to Scrutinizer For This Post:
SkySmart
# 3  
Old 02-23-2017
Try
Code:
eval $(sed 's/^ *\|[{}",]*//g; /^ *$/{d;n}; s/: /="/; s/$/"/ /=/!{d;n} ' file)
echo $ip
16.66.35.10
echo $postal
E1

BUT - be aware that using eval is generally deprecated as it may become dangerous - it will execute any mischievous command that may be contained in the text file. The final test for the equals sign is there to remove anything that is NOT a variable assignment, but doesn't necessarily detect every harmful thing and thus utmost care should be taken!

With a recent bash that provides "process substitution", try
Code:
. <(sed 's/^ *\|[{}",]*//g; /^ *$/{d;n}; s/: /="/; s/$/"/; /=/!{d;n}' file)

Last edited by RudiC; 02-23-2017 at 03:50 PM..
This User Gave Thanks to RudiC For This Post:
SkySmart
# 4  
Old 02-23-2017
can i do something like this:

Code:
while IFS=\" read x key x val x; do
   [ -n "$key" ] && read "$key" <<< "$val"
done <<< "${IpInfo}"

the json information wont always be in a file. matter of fact, it's expected to be in a variable.


the above modification gives the following errors:

Code:
Syntax error: redirection unexpected

# 5  
Old 02-23-2017
Here is another approach by creating an output file that can be sourced later:-
Code:
awk -F: '
        /:/ {
                gsub( /[",]/, X, $1 )
                gsub( /[",]|^ /, X, $2 )
                A[$1] = $2
        }
        END {
                for ( k in A )
                        print "export " k "=" "\"" A[k] "\"" > "s_jsonfile"
        }
' jsonfile

source ./s_jsonfile

print "$ip"

This User Gave Thanks to Yoda For This Post:
SkySmart
# 6  
Old 02-23-2017
Quote:
Originally Posted by SkySmart
can i do something like this:

Code:
while IFS=\" read x key x val x; do
  [ -n "$key" ] && read "$key" <<< "$val"
done <<< "${IpInfo}"

the json information wont always be in a file. matter of fact, it's expected to be in a variable.


the above modification gives the following errors:

Code:
Syntax error: redirection unexpected

Sure that should work in bash / ksh93 /zsh:

Code:
IpInfo='{                                                                                        
  "ip": "16.66.35.10",
  "hostname": "No Hostname",
  "city": "Stepney",
  "region": "England",
  "country": "GB",
  "loc": "51.57,-0.0333",
  "org": "AS6871 British Telecommunications PLC",
  "postal": "E1"
}'

while IFS=\" read x key x val x;  do
    [ -n "$key" ] && read "$key" <<< "$val"
done <<< "$IpInfo"

echo "$ip"
16.66.35.10

Bear in mind, this will fail if keys or values may contain escaped double quotes within the double quotes for example.
Last edited by Scrutinizer; 02-23-2017 at 05:14 PM.. Reason: Corrected IPInfo -> IpInfo
This User Gave Thanks to Scrutinizer For This Post:
SkySmart
# 7  
Old 02-23-2017
Try
Code:
TMP="${IpInfo//[,\{\}]/}"
TMP=${TMP//  \"/}
. <(echo "${TMP//\": /=}")
echo $ip
16.66.35.10
echo $city
Stepney

This User Gave Thanks to RudiC For This Post:
SkySmart
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

How to convert any shell command output to JSON format?

Hi All, I am new to shell scripting, Need your help in creating a shell script which converts any unix command output to JSON format output. example: sample df -h command ouput : Filesystem size used avail capacity Mounted /dev/dsk/c1t0d0s0 8.1G 4.0G 4.0G 50% /... (13 Replies)
Discussion started by: balu1234
13 Replies

2. UNIX for Beginners Questions & Answers

Convert String to an Array using shell scripting in JSON file.

This is the sample json I have pasted here. I want all the IP address strings to be converted into an array. For example "10.38.32.202" has to be converted to everywhere in the JSON. There are multiple IPs in a JSON I am pasting one sample object from the JSON. But the IPs already in an Array... (11 Replies)
Discussion started by: vinshas1
11 Replies

3. Shell Programming and Scripting

JSON structure to table form in awk, bash

Hello guys, I want to parse a JSON file in order to get the data in a table form. My JSON file is like this: { "document":{ "page": }, { "column": } ] }, { ... (6 Replies)
Discussion started by: Gescad
6 Replies

4. Shell Programming and Scripting

Remove first meta key from json records using shell

Hi All, I need to get rid of initial meta key from json files with enclosed parenthesis from start and end of the lines which has total 4000 lines. here is the sample Json records : {"start": true, "meta": {"name": "xyz", "creation": "2017-07-14T16:20:06.000+02:00"}} I need to remove... (7 Replies)
Discussion started by: Cloud_Ninja
7 Replies

5. Shell Programming and Scripting

Save awk record field in bourne shell variable

Hello, I am trying to write a shell script that maintains the health of the passwd file. The goal is to check for duplicate usernames, UID's etc. I am able to find and sort out the UID and login names via awk (which I would like to use), but I can't figure out how to save the record field into a... (1 Reply)
Discussion started by: Learn4Life
1 Replies

6. Shell Programming and Scripting

AWK How to replace a field using 2 shell variables?

Hello everybody: I want to replace any field $2 of any file line (f.i. test.txt) matching $1 with a shell variable. $ cat test.txt F 0 B A H -12.33 Now I'm going to ask the value of variable B: $ SEARCHVAR=B $ OLDVAL=$(awk -v SEARCHVAR="$SEARCHVAR"... (4 Replies)
Discussion started by: basalt
4 Replies

7. Shell Programming and Scripting

awk script to split field data

Hi Experts, I have a Input.txt document which contains data fields seperated by tabs. There are 4 fields totally Named UNIQUE, ORDER, CONTACT and WINS. The UNIQUE field contains unique ID and the CONTACT field contains data seperated by comma in some records. I am looking to write an awk script... (12 Replies)
Discussion started by: forumthreads
12 Replies

8. Shell Programming and Scripting

field seperator question (awk script)

Is there a way I could use different a different field seperator for different parts of the body? kinda like {FS = ":"} FILENAME == "products"{ price = $3 if(numprods < $1-100) numprods = $1-100 } {FS = "/"}{} FILENAME == "associates"{ associateid... (5 Replies)
Discussion started by: angermanaged
5 Replies

9. UNIX for Dummies Questions & Answers

Grap data/string from lines between words

Hi all, im newbie in unix, i have a case like this file name : RegisterSubscriber.log file value : <errId>0x0509000000000003</errId><HARs><ok/><affectEntity>510890905290059</affectEntity></HLRes></HRI> I want to grep the line which contain 0x0509000000000003, and i want to grep... (2 Replies)
Discussion started by: andrisetia
2 Replies

10. Shell Programming and Scripting

Split a field in awk script

Hi all, I have a field in the line, let's say argument $6, which is in the format 00.00 If i want to split the field to get rid of the "." in between of the amount, how can i do that i awk script? I have it like this split($6,a,".") but it will get rid of the last 2 digits after the... (4 Replies)
Discussion started by: CamTu
4 Replies
Login or Register to Ask a Question