Large Variable content size caveats?


Login or Register to Reply

 
Thread Tools Search this Thread
# 8  
Old 1 Week Ago
Quote:
Originally Posted by stomp
basically it boils down to jq. Without it I never would have considered doing this task with bash. All JSON-Handling is done with jq. Bash is the code around it.
Thanks for explaining.

Maybe move away from jq and use something more mainstream for processing JSON?

What are you planning to do with the JSON object after you download and process it?

Push it into a mysql db? Push it to Firebase? Save to a flat file on your box? Push it to another server?
# 9  
Old 1 Week Ago
Code:
 What are you planning to do with the JSON object after you download and process it?

JSON is the result data format for the PowerDNS-API.

I'm using the API to query/add/modify/delete DNS-Records - not to store the retrieved data.

The Script is a command line tool, that's used for various purposes(Create/Delete Records for Let's Encrypt Certificate Validation, automatic configuration of MX-Record Changes, Setup of E-Mail-Autoconfiguration,...)

Quote:
Maybe move away from jq and use something more mainstream for processing JSON?
I have no plans to do so, since it's working fine.

Last edited by stomp; 1 Week Ago at 05:47 AM..
# 10  
Old 1 Week Ago
Yes, I understanding working with JSON. I do it nearly daily (read, modify, update, write, between client and server)

FWIW,

If I was going to use the same API to modify and update JSON as you are doing, I would write a quick app with either PHP (if I did not need any UI), or I would develop with with Vue.js if I needed a web UI.

PHP processes JSON much better than bash and can easily update as well with very simple tools to pull and push files from the net built in, obviously.

Vue.js with extensions like Axios and Vuex is so rich in features for processing JSON across the net that comparing it to bash would be like comparing the Starship Enterprise to a flat bottom wooden boat (at worse) or at best (and being generous) an antique car.

Anyway, I realize a lot of people like to use these old tech tools from decades past and build an infrastructure around it to make it work; but honestly when working with JSON across the net as you are doing, there are much better tools than command line jq, curl, wget and bash, I promise Smilie

Anyway, I think I understand the reason you are using bash, curl and jq.... you are comfortable with those tools and that's cool too Smilie . I used to use those tools (excessively) between 15 and 5 years ago, so I understand and might use them again if I was forced to. However, I have noticed a seismic shift in JSON processing, including the use of Firebase and JSON-based NOSQL repos.

PS: At least use Postman to analyze your API calls. Postman is one of the single most productivity enhancing tools out where when working with JSON APIs.
Login or Register to Reply

|
Thread Tools Search this Thread
Search this Thread:
Advanced Search

More UNIX and Linux Forum Topics You Might Find Helpful
Can't input large file to variable newbie2010 Shell Programming and Scripting 3 06-25-2018 06:49 PM
How to get script to create a new file that lists folder content sorted by size? Braveheart UNIX for Beginners Questions & Answers 4 11-27-2016 09:20 AM
Best Method For Query Content In Large JSON Files metallica1973 Programming 0 04-15-2016 05:41 PM
Help with Splitting a Large XML file based on size AND tags Aviktheory11 Shell Programming and Scripting 7 07-03-2014 03:24 AM
Need help in finishing a bash script for listing subfolder by size in a large folder ultimo Shell Programming and Scripting 2 12-04-2013 05:27 AM
Empty directory, large size and performance bdx Red Hat 5 01-20-2012 12:21 PM
Need help with configuring large packet size on Solaris 7 / e6500 sharique Solaris 9 04-07-2011 03:19 AM
Need help with configuring large packet size on Solaris 7 / e6500 sharique UNIX for Advanced & Expert Users 0 04-06-2011 05:37 PM
Columns comparision of two large size files and printing the difference krao Shell Programming and Scripting 6 10-08-2010 12:47 AM
Creating large number of files of specific size swatideswal Shell Programming and Scripting 3 06-24-2009 08:32 AM
Content Management System for uploading large files z1dane Web Development 3 03-20-2009 12:52 AM
Split a large file with patterns and size sudhamacs Shell Programming and Scripting 5 06-25-2008 03:49 PM
Content of Content of a variable! jaduks Shell Programming and Scripting 2 08-27-2007 12:40 AM
Editing a large size file Rock UNIX for Dummies Questions & Answers 2 12-04-2006 08:32 PM
prevent file size is too large ust UNIX for Dummies Questions & Answers 2 03-10-2005 01:04 PM