Login or Register to Ask a Question and Join Our Community


UNIX for Beginners Questions & Answers

How to convert any shell command output to JSON format?

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums UNIX for Beginners Questions & Answers How to convert any shell command output to JSON format?
# 1  
Old 08-13-2019
How to convert any shell command output to JSON format?
Hi All,

I am new to shell scripting, Need your help in creating a shell script which converts any unix command output to JSON format output.
example:
sample df -h command ouput :

Code:
Filesystem   size   used   avail   capacity   Mounted
/dev/dsk/c1t0d0s0   8.1G   4.0G   4.0G   50%   /
/devices   0K   0K   0K   0%   /devices
ctfs   0K   0K   0K   0%   /system/contract
proc   0K   0K   0K   0%   /proc
mnttab   0K   0K   0K   0%   /etc/mnttab
swap   3.1G   836K   3.1G   1%   /etc/svc/volatile
objfs   0K   0K   0K   0%   /system/object
sharefs   0K   0K   0K   0%   /etc/dfs/sharetab
/usr/lib/libc/libc_hwcap1.so.1   8.1G   4.0G   4.0G   50%   /lib/libc.so.1
fd   0K   0K   0K   0%   /dev/fd
swap   3.1G   84K   3.1G   1%   /tmp
swap   3.1G   32K   3.1G   1%   /var/run
/dev/dsk/c1t0d0s7   11G   11M   11G   1%   /export/home
/vol/dev/dsk/c0t0d0/sol_10_113_x86   2.1G   2.1G   0K   100%   /cdrom/sol_10_113_x86

Desired output in JSON format:

Code:
{
"Filesystem": "/dev/dsk/c1t0d0s0",
"size": "8.1G",
"used": "4.0G",
"avail": "4.0G"
"capacity": "50%",
"Mounted": "/"
}

# 2  
Old 08-13-2019
Like everyone here, you are expected to show your own work, provide the name of your OS, the shell version you are using, etc.

And you are expected to search the forums before posting your question.

Welcome! And please follow forum guidelines.

Thanks.

What code have you written and tried so far?
# 3  
Old 08-14-2019
Hi balu1234,

Welcom to the forum!

I'll echo Neo's comments here, what research have you done and what have you tried?

I'd also add that if you are looking for a converter that will convert any unix command line output to JSON, that is a big ask in no small measure due to the complexity and variety of outputs available in the unix cli.

If you only would like to have this output converted on an occaisional basis, then there are a number of online converters - I usually use this one. This provides a number of output options, I have added the keyed output as an example.
Code:
[
 {
   "Filesystem": "/dev/dsk/c1t0d0s0",
   "size": "8.1G",
   "used": "4.0G",
   "avail": "4.0G",
   "capacity": "50%",
   "Mounted": "/"
 },
 {
   "Filesystem": "/devices",
   "size": "0K",
   "used": "0K",
   "avail": "0K",
   "capacity": "0%",
   "Mounted": "/devices"
 },
 {
   "Filesystem": "ctfs",
   "size": "0K",
   "used": "0K",
   "avail": "0K",
   "capacity": "0%",
   "Mounted": "/system/contract"
 },
 {
   "Filesystem": "proc",
   "size": "0K",
   "used": "0K",
   "avail": "0K",
   "capacity": "0%",
   "Mounted": "/proc"
 },
 {
   "Filesystem": "mnttab",
   "size": "0K",
   "used": "0K",
   "avail": "0K",
   "capacity": "0%",
   "Mounted": "/etc/mnttab"
 },
 {
   "Filesystem": "swap",
   "size": "3.1G",
   "used": "836K",
   "avail": "3.1G",
   "capacity": "1%",
   "Mounted": "/etc/svc/volatile"
 },
 {
   "Filesystem": "objfs",
   "size": "0K",
   "used": "0K",
   "avail": "0K",
   "capacity": "0%",
   "Mounted": "/system/object"
 },
 {
   "Filesystem": "sharefs",
   "size": "0K",
   "used": "0K",
   "avail": "0K",
   "capacity": "0%",
   "Mounted": "/etc/dfs/sharetab"
 },
 {
   "Filesystem": "/usr/lib/libc/libc_hwcap1.so.1",
   "size": "8.1G",
   "used": "4.0G",
   "avail": "4.0G",
   "capacity": "50%",
   "Mounted": "/lib/libc.so.1"
 },
 {
   "Filesystem": "fd",
   "size": "0K",
   "used": "0K",
   "avail": "0K",
   "capacity": "0%",
   "Mounted": "/dev/fd"
 },
 {
   "Filesystem": "swap",
   "size": "3.1G",
   "used": "84K",
   "avail": "3.1G",
   "capacity": "1%",
   "Mounted": "/tmp"
 },
 {
   "Filesystem": "swap",
   "size": "3.1G",
   "used": "32K",
   "avail": "3.1G",
   "capacity": "1%",
   "Mounted": "/var/run"
 },
 {
   "Filesystem": "/dev/dsk/c1t0d0s7",
   "size": "11G",
   "used": "11M",
   "avail": "11G",
   "capacity": "1%",
   "Mounted": "/export/home"
 },
 {
   "Filesystem": "/vol/dev/dsk/c0t0d0/sol_10_113_x86",
   "size": "2.1G",
   "used": "2.1G",
   "avail": "0K",
   "capacity": "100%",
   "Mounted": "/cdrom/sol_10_113_x86"
 }
]

Regards

Gull04
This User Gave Thanks to gull04 For This Post:
Neo
# 4  
Old 08-14-2019
In light of Neo's comments, I'll only give you some pointers.

First, install jq. It is a tool for creating, validating and editing JSON files and can be used to extract data from a JSON file, too. If you are using Linux it will probably be available as a package in your distributions repository. If not, go to their (jq) web site and download for your system.

Second, search for CSV to JSON using jq with your favourite search engine. There is a good example (with explanations) on converting pipe-delimited values to JSON using jq on Stack Exchange.

You will probably have to tailor your solution to each command, as gull04 says (in so many words).

When you have an idea of what to do post your attempt here and I'm sure you will have lots of advice on how to fine-tune it.

Andrew
This User Gave Thanks to apmcd47 For This Post:
Neo
# 5  
Old 08-14-2019
Hi Andrew,

Thanks for your response.

CentOS Linux 7 is my operating system.

i installed jq pkg on my server.

i tried below command:

Code:
# jq -s -R '[[ split("\n")[] | select(length > 0) | split("  +";"") | {(.[0]): .[1]}] | add]' /tmp/df.out

i redirect my output to /tmp/df.out file.

but i am getting below output, but not the output which i desired.

Code:
[
  {
    " Filesystem Size Used Avail Use% Mounted ": null,
    " /dev/mapper/centos-root 37G 15G 22G 41% / ": null,
    " devtmpfs 1.9G 0 1.9G 0% /dev ": null,
    " tmpfs 1.9G 0 1.9G 0% /dev/shm ": null,
    " tmpfs 1.9G 9.8M 1.9G 1% /run ": null,
    " tmpfs 1.9G 0 1.9G 0% /sys/fs/cgroup ": null,
    " /dev/sda1 1014M 235M 780M 24% /boot ": null,
    " tmpfs 379M 4.0K 379M 1% /run/user/42 ": null,
    " tmpfs 379M 36K 379M 1% /run/user/1000 ": null,
    " /dev/sr0 4.3G 4.3G 0 100% /run/media/abhinav/CentOS ": null,
    " tmpfs 379M 0 379M 0% /run/user/0 ": null
  }
]

# 6  
Old 08-15-2019
The trick of course if to create an array of JSON objects which are comma separated key, value pairs.

I don't think a tool like jq is designed to "think for you, as to how to create the array of JSON objects from raw data".

That's up to you to design, specific to this single application first.

Personally, I think you are jumping to tools like jq too quickly before you fully understand / design your algorithm to take your raw output and create an array of JSON objects.

If it was me, I would write this in PHP and it would only take a few lines of PHP code to take your input and create the desired JSON file.

Either way, bash or PHP or Javascript, it a simply matter:
  • Taking the output of the command as text as input into your code.
  • Parsing this data.
  • Creating a JSON object from each line of parsed data (the values), keeping the header line as the key to the JSON objects..
  • Add the JSON object above to an another array.
  • Repeat for the next line of data in your output

I have done this 100s of time (using PHP) and there is generally no short-cut to writing your algorithm to take raw text as data and parsing it into a JSON object and then creating an array of JSON objects.

However, generally speaking I create an array of arrays and then convert the final array to JSON; but it really depends on the format of the final JSON object and if it needs to be stringified to be transmitted over the net.

Also, I would caution you not to try to create "one parser to rule them all" as the output of other commands will like need a different parser.

Then when you have done this for a number of different commands, you can consider if you want to build "the mother of all UNIX commands to JSON parser".

Reference:

PHP: json_encode - Manual
This User Gave Thanks to Neo For This Post:
vgersh99
# 7  
Old 08-15-2019
Did you find any examples on the web? This is the article I was alluding to in my previous post. Also the output of the df command you are using needs to be modified to turn it into something more akin to a CSV file. I would suggest:
Code:
df -h | tr -s ' ' ',' > /tmp/df.out

as I'm not sure that jq's split function can be used to split on multiple spaces as a single delimiter.

As you have had a go, I'll post a sample based on the above link.
Code:
$ df -h . | tr -s ' ' ',' | jq -nR '[ 
( input | split(",") ) as $keys | 
( inputs | split(",") ) as $vals | 
[ [$keys, $vals] | 
transpose[] | 
{key:.[0],value:.[1]} ] | 
from_entries ]'
[
  {
    "Filesystem": "/dev/sda1",
    "Size": "150G",
    "Used": "100G",
    "Avail": "50G",
    "Use%": "60%",
    "Mounted": "/home"
  }
]
$

It's not a trivial piece of code, and when I found it I pulled it apart, line by line, until I had an understanding of what is going on.

@NEO, I don't generally do much with JSON, and find JQ is a good tool for pretty-printing, or extracting data from a JSON file. If I wanted to manipulate JSON data, I would probably use python, just as you use PHP.

Andrew
This User Gave Thanks to apmcd47 For This Post:
Neo
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Printing df -h output in json format

Hi All, i am trying to print the df -h ouput in json format. using below script. #!/usr/bin/env bash df -h > /tmp/sdf nawk '{print " "$1" "$2" "$3" "$4" "$5" "$6" "}' /tmp/sdf > /tmp/sdf1 nawk 'NR==1 { for (i=1; i<=NF; i++) { f = i }}{ print $(f), $(f), $(f), $(f), ... (2 Replies)
Discussion started by: sravani25
2 Replies

2. UNIX for Beginners Questions & Answers

Convert String to an Array using shell scripting in JSON file.

This is the sample json I have pasted here. I want all the IP address strings to be converted into an array. For example "10.38.32.202" has to be converted to everywhere in the JSON. There are multiple IPs in a JSON I am pasting one sample object from the JSON. But the IPs already in an Array... (11 Replies)
Discussion started by: vinshas1
11 Replies

3. UNIX for Beginners Questions & Answers

Convert rows to column and print output in required format

Hi All, i am trying to print the solaris 11 packages in below required format, But i am unable to do that. Current ouput : root@abc# pkginfo -l | egrep '(BASEDIR|NAME|VERSION)' | awk '{print}' NAME: QLogic 570x/571x Gigabit Ethernet Driver VERSION: 11.11,REV=2009.11.11 ... (7 Replies)
Discussion started by: balu1234
7 Replies

4. Shell Programming and Scripting

JSON Output format

Dear friends, I'm getting below API result and i would like to format them with Shell scripting. Input "id": 9, "description": "short desc", "name": "test", "name_with_namespace": "ABCD-PDFF-PLATFORM-TEST-V1 / test", "path": "test", "path_with_namespace":... (7 Replies)
Discussion started by: baluchen
7 Replies

5. Shell Programming and Scripting

Loop to convert text output in the HTML format

Hello Everyone, I have a sample file raw.txt as shown below : Drive Bays Bay Name : SD-2C Number of Standby Power Supplies : 4 Number of Drive Enclosures : 12 Summary Status of Contained Modules All... (6 Replies)
Discussion started by: rahul2662
6 Replies

6. Shell Programming and Scripting

Convert json to xml

Hello everyone, I have created a workflow that will pull down data via a RESTful API in JSON, then my code parses and modifies said data. The problem I have is that these APIs I am working with only accept XML to PUT/POST data, and I am looking for an easy way to convert my JSON file to XML. ... (2 Replies)
Discussion started by: Zaphod_B
2 Replies

7. Shell Programming and Scripting

Multiple records need to convert UNIXtime to human readable datatime and all output in one format

Hello Experts, Below is the record i have: sample data attached I want this record of each row to be in single line and there are multiple rowise unixtime mentioned e.g 11996327 , This needs to be converted to Human readdable data and time from multiple rows Can you help me , it will be... (10 Replies)
Discussion started by: manishK
10 Replies

8. Shell Programming and Scripting

Convert TZ output to a different format

Friends, I am trying to convert my local server timezone EST to UTC and for which I used the TZ command, see below $ date Thu Dec 6 10:14:14 EST 2012 $ $ TZ=UTC date -d '10:14 EST' Thu Dec 6 15:14:00 UTC 2012 Now I would like to have the same output in 'yyyymmdd hh:mm' format. ... (4 Replies)
Discussion started by: vivek_damodaran
4 Replies

9. Shell Programming and Scripting

convert the output in table format

Hi All, I have a output like below values val1=test.com val2=10.26.208.11 val3=en1 val4=test-priv1.com val5=192.168.3.4 val6=en2 val7=test-priv2.com val8=192.168.4.4 val9=en3 val10=test-vip.com val11=10.26.208.9 val12=$val3 I want to convet this output values into below... (1 Reply)
Discussion started by: kamauv234
1 Replies

10. Shell Programming and Scripting

shell script to format command output

Hello team, I am running below command which is giving following output. bash-3.00$ ps -eo pid,pcpu,args | sort +1n | grep -i java 12 0.0 grep -i java 8804 0.0 /opt/app/ccr/home/ccr/WebSphere/AppServer/java/bin/sparcv9/java -XX:+UnlockDiag 9241 0.0... (7 Replies)
Discussion started by: coolguyamy
7 Replies
Login or Register to Ask a Question