![\"parse](\"https:\/\/www.ostechnix.com\/wp-content\/uploads\/2019\/03\/json-720x340.png\")
<\/p>\n<\/p>\n
JSON<\/strong>\u00a0is a lightweight and language independent data storage format, easy to integrate with most programming languages and also easy to understand by humans, of course when properly formatted. The word JSON stands for\u00a0J<\/strong>avaS<\/strong>cript\u00a0O<\/strong>bject\u00a0N<\/strong>otation, though it starts with JavaScript, and primarily used to exchange data between server and browser, but now being used in many fields including embedded systems. Here we\u2019re going to parse and pretty print JSON with command line tools on Linux. It\u2019s extremely useful for handling large JSON data in a shell scripts, or manipulating JSON data in a shell script.<\/p>\n The JSON data is structured to be somewhat more human readable. However in most cases, JSON data is stored in a single line, even without a line ending character.<\/p>\n Obviously that\u2019s not very convenient for reading and editing manually.<\/p>\n That\u2019s when pretty print is useful. The name is quite self explanatory, re-formatting the JSON text to be more legible by humans. This is known as\u00a0JSON pretty printing<\/strong>.<\/p>\n JSON data could be parsed with command line text processors like\u00a0awk<\/strong>,\u00a0sed<\/strong>\u00a0and\u00a0gerp<\/strong>. In fact JSON.awk is an awk script to do that. However there are some dedicated tools for the same purpose.<\/p>\n jq<\/strong>\u00a0or\u00a0jshon<\/strong>, JSON parser for shell, both of them are quite useful.<\/p>\n<\/li>\n Shell scripts like\u00a0JSON.sh<\/strong>\u00a0or\u00a0jsonv.sh<\/strong>\u00a0to parse JSON in bash, zsh or dash shell.<\/p>\n<\/li>\n JSON.awk<\/strong>, JSON parser awk script.<\/p>\n<\/li>\n Python modules like\u00a0json.tool<\/strong>.<\/p>\n<\/li>\n underscore-cli<\/strong>, Node.js and javascript based.<\/p>\n<\/li>\n<\/ol>\n In this tutorial I\u2019m focusing only on\u00a0jq<\/strong>, which is quite powerful JSON parser for shells with advanced filtering and scripting capability.<\/p>\n JSON data could be in one and nearly illegible for humans, so to make it somewhat readable, JSON pretty printing is here.<\/p>\n Example:<\/strong>\u00a0A data from\u00a0jsonip.com<\/strong>, to get external IP address in JSON format, use\u00a0curl<\/strong>\u00a0or\u00a0wget<\/strong>\u00a0tools like below.<\/p>\n The actual data looks like this:<\/p>\n Now pretty print it with jq:<\/p>\n This should look like below, after filtering the result with jq.<\/p>\n The Same thing could be done with python\u00a0json.tool<\/strong>\u00a0module. Here is an example:<\/p>\n This Python based solution should be fine for most users, but it\u2019s not that useful where Python is not pre-installed or could not be installed, like on embedded systems.<\/p>\n However the json.tool python module has a distinct advantage, it\u2019s cross platform. So, you can use it seamlessly on Windows, Linux or mac OS.<\/p>\n Suggested read:<\/strong><\/p>\n First, you need to install jq, it\u2019s already picked up by most GNU\/Linux distributions, install it with their respective package installer commands.<\/p>\n On Arch Linux:<\/p>\n On Debian, Ubuntu, Linux Mint:<\/p>\n On Fedora:<\/p>\n On openSUSE:<\/p>\n For other OS or platforms, see the\u00a0official installation instructions<\/a>.<\/p>\n Basic filters and identifiers of jq<\/strong><\/p>\n jq could read the JSON data either from\u00a0stdin<\/strong>\u00a0or a\u00a0file<\/strong>. You\u2019ve to use both depending on the situation.<\/p>\n The single symbol of\u00a0.<\/strong>\u00a0is the most basic filter. These filters are also called as\u00a0object identifier-index<\/strong>. Using a single\u00a0.\u00a0<\/strong>along with jq basically pretty prints the input JSON file.<\/p>\n Single quotes<\/strong>\u00a0\u2013 You don\u2019t have to use the single quote always. But if you\u2019re combining several filters in a single line, then you must use them.<\/p>\n Double quotes<\/strong>\u00a0\u2013 You\u2019ve to enclose any special character like\u00a0@<\/strong>,\u00a0#<\/strong>,\u00a0$<\/strong>\u00a0within two double quotes, like this example,\u00a0jq .foo.\u201d@bar\u201d<\/strong><\/p>\n Raw data print<\/strong>\u00a0\u2013 For any reason, if you need only the final parsed data, not enclosed within a double quote, use the -r flag with the jq command, like this.\u00a0\u2013 jq -r .foo.bar<\/strong>.<\/p>\n Parsing specific data<\/strong><\/p>\n To filter out a specific part of JSON, you\u2019ve to look into the pretty printed JSON file\u2019s data hierarchy.<\/p>\n An example of JSON data, from Wikipedia:<\/p>\n I\u2019m going to use this JSON data as an example in this tutorial, saved this as\u00a0sample.json<\/strong>.<\/p>\n Let\u2019s say I want to filter out the address from sample.json file. So the command should be like:<\/p>\n Sample output:<\/strong><\/p>\n Again let\u2019s say I want the postal code, then I\u2019ve to add another\u00a0object identifier-index<\/strong>, i.e. another filter.<\/p>\n Also note that the\u00a0filters are case sensitive<\/strong>\u00a0and you\u2019ve to use the exact same string to get something meaningful output instead of null.<\/p>\n Parsing elements from JSON array<\/strong><\/p>\n Elements of JSON array are enclosed within square brackets, undoubtedly quite versatile to use.<\/p>\n To parse elements from a array, you\u2019ve to use the\u00a0[]identifier<\/strong>\u00a0along with other object identifier-index.<\/p>\n In this sample JSON data, the phone numbers are stored inside an array, to get all the contents from this array, you\u2019ve to use only the brackets, like this example.<\/p>\n Let\u2019s say you just want the first element of the array, then use the array object numbers starting for 0, for the first item, use\u00a0[0]<\/strong>, for the next items, it should be incremented by one each step.<\/p>\n Scripting examples<\/strong><\/p>\n Let\u2019s say I want only the the number for home, not entire JSON array data. Here\u2019s when scripting within jq command comes handy.<\/p>\n Here first I\u2019m piping the results of one filer to another, then using the select attribute to select a particular type of data, again piping the result to another filter.<\/p>\n Explaining every type of jq filters and scripting is beyond the scope and purpose of this tutorial. It\u2019s highly suggested to read the JQ manual for better understanding given below.<\/p>\n Resources:<\/strong><\/p>\nWhat is pretty printing?<\/h4>\n
Parse And Pretty Print JSON With Linux Commandline Tools<\/h4>\n
\n
JSON pretty printing<\/h4>\n
$ wget -cq http:\/\/jsonip.com\/ -O -<\/pre>\n
{\"ip\":\"111.222.333.444\",\"about\":\"\/about\",\"Pro!\":\"http:\/\/getjsonip.com\"}<\/pre>\n$ wget -cq http:\/\/jsonip.com\/ -O - | jq '.'<\/pre>\n
{\r\n\r\n \"ip\": \"111.222.333.444\",\r\n\r\n \"about\": \"\/about\",\r\n\r\n \"Pro!\": \"http:\/\/getjsonip.com\"\r\n\r\n}<\/pre>\n$ cat anything.json | python -m json.tool<\/pre>\n
\n\n
\nHow to parse JSON with jq<\/h4>\n
$ sudo pacman -S jq<\/pre>\n
$ sudo apt-get install jq<\/pre>\n
$ sudo dnf install jq<\/pre>\n
$ sudo zypper install jq<\/pre>\n
{\r\n\r\n \"firstName\": \"John\",\r\n\r\n \"lastName\": \"Smith\",\r\n\r\n \"age\": 25,\r\n\r\n \"address\": {\r\n\r\n \"streetAddress\": \"21 2nd Street\",\r\n\r\n \"city\": \"New York\",\r\n\r\n \"state\": \"NY\",\r\n\r\n \"postalCode\": \"10021\"\r\n\r\n},\r\n\r\n \"phoneNumber\": [\r\n\r\n{\r\n\r\n \"type\": \"home\",\r\n\r\n \"number\": \"212 555-1234\"\r\n\r\n},\r\n\r\n{\r\n\r\n \"type\": \"fax\",\r\n\r\n \"number\": \"646 555-4567\"\r\n\r\n}\r\n\r\n],\r\n\r\n \"gender\": {\r\n\r\n \"type\": \"male\"\r\n\r\n }\r\n\r\n}<\/pre>\n$ jq .address sample.json<\/pre>\n
{\r\n\r\n \"streetAddress\": \"21 2nd Street\",\r\n\r\n \"city\": \"New York\",\r\n\r\n \"state\": \"NY\",\r\n\r\n \"postalCode\": \"10021\"\r\n\r\n}<\/pre>\n$ cat sample.json | jq .address.postalCode<\/pre>\n
$ jq .phoneNumber[] sample.json<\/pre>\n
$ jq .phoneNumber[0] sample.json<\/pre>\n
$ cat sample.json | jq -r '.phoneNumber[] | select(.type == \"home\") | .number'<\/pre>\n
\n