![\"\"](\"https:\/\/linuxhint.com\/wp-content\/uploads\/2019\/01\/1-45.png\")
<\/p>\nIf you\u2019ve ever sat in front of a terminal, typed \u2018curl\u2019, pasted the URL of something you want to download, and hit enter, cool! You\u2019re going to be killing it with curl in bash scripts in no time. Here you will learn how to use curl in bash scripts and important tips and tricks for automation.<\/p>\n
Great! Now what? Before you kill anything in bash it is dire to know where to get help if in trouble. Here is what the man page for curl or curl help command looks like. Copy and paste. Try not to be overwhelmed by appearances. There are a lot of options that you only need later in life. More importantly, it serves as a quick reference to lookup options as you need.<\/p>\n
<\/p>\n
Here are some commands to get help within your terminal and other browser-friendly\u00a0 resources.<\/p>\n
Help commands for curl in bash<\/p>\n
Consult these resources anytime you need. In addition to this piece, they will serve as companions on your journey towards killing it with curl in bash scripts.<\/p>\n
Now that getting help and listing command line options is out of the picture, let\u2019s move on to the three ways.<\/p>\n
You may argue that there are more than three ways to curl in bash. However, for simplicity purposes, let\u2019s just say that there are. Also note that in practice, usage of each way is not mutually exclusive. In fact, you will find that ways may be intertwined depending on the intent of your bash script. Let\u2019s begin.<\/p>\n
All options aside curl downloads files by default. In bash, we curl to download a file as follows.<\/p>\n
This sends the content of the file we are downloading to standard output; that is, the your screen. If the file is a video or an image don\u2019t be surprised if you hear a few beeps. We need to save to a file. Here\u2019s how it looks.<\/p>\n
curl\u00a0${url}<\/span>\u00a0-o<\/span>\u00a0outfile curl\u00a0${url}<\/span>\u00a0-O<\/span> ## expect file saved as $( basename ${url} )<\/span><\/div>\n<\/div>\n Note that the download file save as option inheriting file name is particularly useful when using URL globbing, which is covered in the bash curl loop section.<\/p>\n Now let\u2019s move on to how to check headers prior to downloading a file with curl in bash.<\/p>\n There will come a time when you wish to get information about a file before downloading. To do this, we add the -I option to the curl command as follows.<\/p>\n Note that there are other ways to dump headers from curl requests, which is left for homework.<\/p>\n Here is a quick example to show how the second way works in bash scripts that can be used to serve as a part of a web page health checker.<\/p>\n Often, we want to get the response code for a curl request in bash. To do this, we would need to first request the headers of a response and then extract the response code. Here is what it would look like.<\/p>\n curl\u00a0${url}<\/span>\u00a0-I<\/span>\u00a0-o<\/span>\u00a0headers\u00a0-s<\/span> cat<\/span>\u00a0 headers cat<\/span>\u00a0headers\u00a0|<\/span>\u00a0head<\/span>\u00a0-n<\/span>\u00a01<\/span>\u00a0|<\/span>\u00a0cut<\/span>\u00a0‘-d ‘<\/span>\u00a0‘-f2’<\/span> My site is up. Great!<\/p>\n Now let\u2019s move on to making posts with curl in bash scripts.<\/p>\n There will come a time when you need to make posts with curl in bash to authenticate to access or modification of private content. Such is the case working with APIs and html forms. It may require multiple curl requests. The placeholder curl command line for this way is as follows.<\/p>\n Making posts involves adding corresponding headers and data to allow for authentication.\u00a0 I\u2019ve prepared some examples of making posts with curl in bash.<\/p>\n Here is an example of using curl in bash scripts to download a file requiring basic authentication. Note that credentials are stored in a separate file called bash-curl-basic-auth-example-config.sh, which is also included below.<\/p>\n curl-basic-auth.sh<\/u><\/p>\n
\n# download file save as option<\/span><\/p>\n
\n# download file inherit filename<\/span><\/p>\nThe second way: Checking headers<\/strong><\/h3>\n
\ncurl\u00a0-I<\/span>\u00a0${url}<\/span>
\n# download headers<\/span><\/div>\n<\/div>\nExample) bash curl get response code<\/h4>\n
\nurl<\/span>=https:\/\/<\/span>temptemp3.github.io
\n# just some url<\/span><\/p>\n
\n# download file<\/span><\/p>\n
\n# response headers<\/span>
\n## expect<\/span>
\n#HTTP\/2 200<\/span>
\n#server: GitHub.com<\/span>
\n#content-type: text\/html; charset=utf-8<\/span>
\n#strict-transport-security: max-age=31557600<\/span>
\n#last-modified: Thu, 03 May 2018 02:30:03 GMT<\/span>
\n#etag: “5aea742b-e12”<\/span>
\n#access-control-allow-origin: *<\/span>
\n#expires: Fri, 25 Jan 2019 23:07:17 GMT<\/span>
\n#cache-control: max-age=600<\/span>
\n#x-github-request-id: 8808:5B91:2A4802:2F2ADE:5C4B944C<\/span>
\n#accept-ranges: bytes<\/span>
\n#date: Fri, 25 Jan 2019 23:12:37 GMT<\/span>
\n#via: 1.1 varnish<\/span>
\n#age: 198<\/span>
\n#x-served-by: cache-nrt6148-NRT<\/span>
\n#x-cache: HIT<\/span>
\n#x-cache-hits: 1<\/span>
\n#x-timer: S1548457958.868588,VS0,VE0<\/span>
\n#vary: Accept-Encoding<\/span>
\n#x-fastly-request-id: b78ff4a19fdf621917cb6160b422d6a7155693a9<\/span>
\n#content-length: 3602<\/span><\/p>\n
\n# get response code<\/span>
\n## expect<\/span>
\n#200<\/span><\/div>\n<\/div>\nThe third way: Making posts<\/strong><\/h3>\n
\ncurl\u00a0-u<\/span>\u00a0-H<\/span>\u00a0–data<\/span>\u00a0${url}<\/span>
\n# send crafted request<\/span><\/div>\n<\/div>\nExample) Basic authentication<\/h4>\n
\n#!\/bin\/bash<\/span>
\n## curl-basic-auth<\/span>
\n## – http basic authenication example using<\/span>
\n##\u00a0\u00a0 curl in bash<\/span>
\n## version 0.0.1<\/span>
\n##################################################<\/span>
\n.\u00a0${SH2}<\/span>\/<\/span>cecho.sh\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0# colored echo<\/span>
\ncurl-basic-auth(<\/span>)<\/span>\u00a0{<\/span>
\ncecho yellow url:\u00a0${url}<\/span>
\nlocal<\/span>\u00a0username
\nlocal<\/span>\u00a0passwd<\/span>
\n.\u00a0${FUNCNAME}<\/span>-config.sh\u00a0# ${username}, ${passwd}<\/span>
\ncurl\u00a0-v<\/span>\u00a0-u<\/span>\u00a0${username}<\/span>:${password}<\/span>\u00a0${url}<\/span>\u00a0–location<\/span>
\n}<\/span>
\n##################################################<\/span>
\nif<\/span>\u00a0[<\/span>\u00a0${#}<\/span>\u00a0-eq<\/span>\u00a01<\/span>\u00a0]<\/span>
\nthen<\/span>
\nurl<\/span>=“${1}<\/span>“<\/span>
\nelse<\/span>
\nexit<\/span>\u00a01<\/span>\u00a0# wrong args<\/span>
\nfi<\/span>
\n##################################################<\/span>
\ncurl-basic-auth
\n##################################################<\/span>
\n## generated by create-stub2.sh v0.1.1<\/span>
\n## on Sun, 27 Jan 2019 14:04:18 +0900<\/span>
\n## see <https:\/\/github.com\/temptemp3\/sh2><\/span>
\n##################################################<\/span><\/div>\n<\/div>\n