Building JSON command with bash script


Login or Register for Dates, Times and to Reply

 
Thread Tools Search this Thread
# 29  
With some more mucking around, I now have a version without eval that also escapes the reserved JSON characters:

Code:
#!/bin/bash
mk_json_object(){
   local items rest

   printf -v items '  "%s": "%s"' $(json_quote "$1" "$2")
   shift 2
   [ $# -gt 1 ] && printf -v rest ',\n  "%s": "%s"' $(json_quote "$@")
   printf "{\n%s\n}\n" "$items$rest"
}

mk_json_lst(){
   local list rest
   printf -v list '  "%s"' $(json_quote "$1")
   shift
   [ $# -gt 0 ] && printf -v rest ',\n  "%s"' $(json_quote "$@")
   printf "[\n%s\n]\n" "$list$rest"
}

json_quote() {
    while [ $# -gt 0 ]
    do
        val=${1//\\/\\\\}
        val=${val//\"/\\\"}
        val=${val//$'\n'/\\\n}
        val=${val//$'\t'/\\\t}
        val=${val//$'\r'/\\\r}
        val=${val//$'\f'/\\\f}
        val=${val//$'\b'/\\\b}
        printf "%s" "$val"
        shift
        [ $# -gt 0 ] && printf " "
    done
}

mk_json_object_one_val() {
        local key=$1
        while [ $# -gt 1 ]
        do
            shift
            args=( "${args[@]}" "$key" "$1" )
        done
        mk_json_object "${args[@]}"
}

mk_json_object_one_val '$(uname)' "key
1" '"test"' more$'\r'
mk_json_lst test$'\b' key-2

output:
Code:
{
  "$(uname)": "key\n1",
  "$(uname)": "\"test\"",
  "$(uname)": "more\r"
}
[
  "test\b",
  "key-2"
]

Note there are some extra space and newline characters added to the output in mk_json_object() and mk_json_lst() that you many want to remove if you a building variables to send as paramters to another program.

--- Post updated at 09:50 AM ---

Quote:
Originally Posted by psysc0rpi0n
Now, according to the examples from bitcoin core:
Code:
> bitcoin-cli sendmany "" "{\"1D1ZrZNe3JUo7ZycKEYQQiQAWd9y54F4XX\":0.01,\"1353tsE8YMTA4EuV7dgUXGjNFf9KpVvKHz\":0.02}" 1 "" "[\"1D1ZrZNe3JUo7ZycKEYQQiQAWd9y54F4XX\",\"1353tsE8YMTA4EuV7dgUXGjNFf9KpVvKHz\"]"

So the functions outputs are not matching exactly the docs example. The thing is that the docs example wouldn't also pass in a json checker tool like jsonlint. This said,I think the backslashes escaping the inner double quotes of each address can be removed.

Now, there is a question that might be asked at this point. Will these backslashes be handled by bitcoin core and removed later to pass the command to the RPC server? Are they really needed? I would take a chance in saying they are not needed.
These escaped quotes are simply to protect them from the shell the actual argument values that will react the bitcoin-cli process will be

Code:
1:sendmany
2:
3:{"1D1ZrZNe3JUo7ZycKEYQQiQAWd9y54F4XX":0.01,"1353tsE8YMTA4EuV7dgUXGjNFf9KpVvKHz":0.02}
4:1
5:
6:["1D1ZrZNe3JUo7ZycKEYQQiQAWd9y54F4XX","1353tsE8YMTA4EuV7dgUXGjNFf9KpVvKHz"]


Last edited by Chubler_XL; 09-25-2019 at 09:00 PM..
This User Gave Thanks to Chubler_XL For This Post:
# 30  
But I still need to understand something else. But it's something related with the way bitcoin core processes the command.

Stomp functions don't add any backslashes and the script don't return any script related errors. The error returned is from bitcoin core which complains about insufficient funds. However, when I tried to run the commands above manually without the backslashes, bitcoin core complained about json parsing error. So, why running the command by hand with no backslashes, produces errors and running the command without backslashes using the script, don't produce any errors (related with the script code)????

Tomorrow I need to look into this new issue!

ok, enough for today. Time to sleep.

Thanks to both of you.

Last edited by psysc0rpi0n; 09-25-2019 at 09:09 PM..
# 31  
Quote:
Originally Posted by psysc0rpi0n
But I still need to understand something else. But it's something related with the way bitcoin core processes the command.

Stomp functions don't add any backslashes and the script don't return any script related errors. The error returned is from bitcoin core which complains about insufficient funds. However, when I tried to run the commands above manually without the backslashes, bitcoin core complained about json parsing error. So, why running the command by hand with no backslashes, produces errors and running the command without backslashes using the script, don't produce any errors (related with the script code)????

Tomorrow I need to look into this new issue!

ok, enough for today. Time to sleep.

Thanks to both of you.
The reason for this is that quote removal only occurs on the original input words, not the result of expansions. To demonstrate:

Code:
$ echo "Hello, world"
Hello, world
$ VAR='echo "Hello, world"'
$ $VAR
"Hello, world"

It's also worth noting that quote matching is done before any expansion, for example:
Code:
$ QT='"'
$ echo "$QT"
"
$ echo $QT
"

If you save this code a pparam(or make it a function in your script) you can use it to debug what the shell is doing to your parameters before they are passed on to scripts/functions:

Code:
for((i=1;i<=${#};i++))
do
   echo Param $i: ${!i}
done

Code:
$ VAR='"one" "two"'
$ ./pparam $VAR
Param 1: "one"
Param 2: "two"
$ ./pparam "$VAR"
Param 1: "one" "two"
$ ./pparam "one" "two"
Param 1: one
Param 2: two

These 3 Users Gave Thanks to Chubler_XL For This Post:
# 32  
Quote:
Attempt 3 - readded backslashes because it's clear that they are needed.
Code:
bitcoin-cli -testnet sendmany "" "{\"n1yswZYByRv3okGDHs1oDaevq8gsDaYZzC\":"0.079319635", \"2NFafKRugHFdYEib7xVfkT6bvtkUKK8oShZ\":"0.079319635"}" 6 "Payments" "[\"n1yswZYByRv3okGDHs1oDaevq8gsDaYZzC\", \"2NFafKRugHFdYEib7xVfkT6bvtkUKK8oShZ\"]" true 6 CONSERVATIVE

Result:
Code:
error code: -3
error message:
Invalid amount

I think that is also wrong quoting, because the amounts are not escaped(prepended with slashes). But this causes no errors, because numbers should not be quoted in json. Maybe it will cause an error if you fix the quoting. :-)
# 33  
Quote:
Originally Posted by stomp
Code:
bitcoin-cli -testnet sendmany "" "{\"n1yswZYByRv3okGDHs1oDaevq8gsDaYZzC\":"0.079319635", \"2NFafKRugHFdYEib7xVfkT6bvtkUKK8oShZ\":"0.079319635"}" 6 "Payments" "[\"n1yswZYByRv3okGDHs1oDaevq8gsDaYZzC\", \"2NFafKRugHFdYEib7xVfkT6bvtkUKK8oShZ\"]" true 6 CONSERVATIVE

Result:
Code:
error code: -3
error message:
Invalid amount

I think that is also wrong quoting, because the amounts are not escaped(prepended with slashes). But this causes no errors, because numbers should not be quoted in json. Maybe it will cause an error if you fix the quoting. :-)
In bitcoin core docs, says that the amount can be strings too.

Later after work, I'll look into this.
Thanks
Psy
# 34  
Quote:
Originally Posted by Chubler_XL

...

These escaped quotes are simply to protect them from the shell the actual argument values that will react the bitcoin-cli process will be
Ok, I'm going to try your examples. This is not trivial for a starter. I've just been reading but not practicing much, so this is not clear at first sight!

--- Post updated at 10:02 PM ---

Quote:
Originally Posted by Chubler_XL
The reason for this is that quote removal only occurs on the original input words, not the result of expansions. To demonstrate:

Code:
$ echo "Hello, world"
Hello, world
$ VAR='echo "Hello, world"'
$ $VAR
"Hello, world"

It's also worth noting that quote matching is done before any expansion, for example:
Code:
$ QT='"'
$ echo "$QT"
"
$ echo $QT
"

If you save this code a pparam(or make it a function in your script) you can use it to debug what the shell is doing to your parameters before they are passed on to scripts/functions:

Code:
for((i=1;i<=${#};i++))
do
   echo Param $i: ${!i}
done

Code:
$ VAR='"one" "two"'
$ ./pparam $VAR
Param 1: "one"
Param 2: "two"
$ ./pparam "$VAR"
Param 1: "one" "two"
$ ./pparam "one" "two"
Param 1: one
Param 2: two

I don't fully understand what you mean by:
Quote:
that quote removal only occurs on the original input words, not the result of expansions.
and
Quote:
It's also worth noting that quote matching is done before any expansion, for example:
.

I need to read about expansions!
# 35  
I finally reached my goal...

Some changes in the script in the meantime.

I share the link in github:
TugaRevoBTC/TugaRevoTestnet.sh at master . PsySc0rpi0n/TugaRevoBTC . GitHub

Edited;
But this is not over... Heheh
I'll be doing some changes and I'll be doing tutorials and asking questions here.

Last edited by psysc0rpi0n; 09-27-2019 at 06:00 AM..
Login or Register for Dates, Times and to Reply

Previous Thread | Next Thread
Thread Tools Search this Thread
Search this Thread:
Advanced Search

Test Your Knowledge in Computers #420
Difficulty: Medium
JavaScript supports much of the structured programming syntax from C (e.g., if statements, while loops, switch statements, do while loops, etc.).
True or False?

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

How to convert any shell command output to JSON format?

Hi All, I am new to shell scripting, Need your help in creating a shell script which converts any unix command output to JSON format output. example: sample df -h command ouput : Filesystem size used avail capacity Mounted /dev/dsk/c1t0d0s0 8.1G 4.0G 4.0G 50% /... (9 Replies)
Discussion started by: balu1234
9 Replies

2. Shell Programming and Scripting

JSON structure to table form in awk, bash

Hello guys, I want to parse a JSON file in order to get the data in a table form. My JSON file is like this: { "document":{ "page": }, { "column": } ] }, { ... (6 Replies)
Discussion started by: Gescad
6 Replies

3. UNIX for Beginners Questions & Answers

Json field grap via shell script/awk

i have a json data that looks like this: { "ip": "16.66.35.10", "hostname": "No Hostname", "city": "Stepney", "region": "England", "country": "GB", "loc": "51.57,-0.0333", "org": "AS6871 British Telecommunications PLC", "postal": "E1" } im looking for a way to assign... (9 Replies)
Discussion started by: SkySmart
9 Replies

4. Shell Programming and Scripting

Parsing and Editing a json file with bash script

I am trying to automate editing of a json file using bash script. The file I initially receive is { "appMap": { "URL1": { "name": "a" }, "URL2": { "name": "b" }, "URL3": { "name": "c" }, } WHat I would like to do is replace... (5 Replies)
Discussion started by: Junaid Subhani
5 Replies

5. Shell Programming and Scripting

UNIX or Perl script to convert JSON to CSV

Is there a Unix or Perl script that converts JSON files to CSV or tab delimited format? We are running AIX 6.1. Thanks in advance! (1 Reply)
Discussion started by: warpmail
1 Replies

6. Shell Programming and Scripting

Bash script - cygwin (powershell?) pull from GitHub API Parse JSON

All, Have a weird issue where i need to generate a report from GitHub monthly detailing user accounts and the last time they logged in. I'm using a windows box to do this (work issued) and would like to know if anyone has any experience scripting for GitAPI using windows / cygwin / powershell?... (9 Replies)
Discussion started by: ChocoTaco
9 Replies

7. Shell Programming and Scripting

Bash script with awk command ERROR

Hello im new here... Im trying to read file and create folders from words in it but i get this for loop error awk : line 3 : syntax error at or near for my code is.. #!/bin/bash begin for (( i=1;i<=5;i++)); do awk -v i=$i $0 { print $i } mkdir $i done {print $i} end {} i have... (7 Replies)
Discussion started by: boxstep
7 Replies

8. Shell Programming and Scripting

How to define a variable in a BASH script by using a JSON file online?

Hello, I would like to modify an existing script of mine that uses a manually defined "MCVERSION" variable and make it define that variable instead based on this JSON file stored online: https://s3.amazonaws.com/Minecraft.Download/versions/versions.json Within that JSON, I 'm looking for... (4 Replies)
Discussion started by: nbsparks
4 Replies

9. Shell Programming and Scripting

BASH SCRIPT of LS command

I need help in writing a BASH SCRIPT of ls command. for example: $ ./do_ls.sh files f1.txt f2.jpeg f3.doc $ ./do_ls.sh dirs folder1 folder2 folder3 My attempt: #!/bin/bash # if test $# -d file then echo $dirs else (3 Replies)
Discussion started by: above8k
3 Replies

10. Shell Programming and Scripting

Building command line parameters of arbitrary length

I couldn't find an existing thread that addressed this question, so hopefully this isn't redundant with anything previously posted. Here goes: I am writing a C-Shell script that runs a program that takes an arbitrary number of parameters: myprog -a file1 \ -b file2 \ -c file3 ... \ -n... (2 Replies)
Discussion started by: cmcnorgan
2 Replies

Featured Tech Videos