Linux Blimp

In the last article, we have covered few useful tools like ‘rTorrent‘, ‘wget‘, ‘cURL‘, ‘w3m‘, and ‘Elinks‘. We got lots of response to cover few other tools of same genre, if you’ve missed the first part you can go through it..

1. 5 Command Line Tools for Downloading Files and Browsing Websites

This article aims at making you aware of several other Linux command Line browsing and downloading applications, which will help you to browse and download files within the Linux shell.

1. links

Links is an open source web browser written in C programming Language. It is available for all major platforms viz., Linux, Windows, OS X and OS/2. This browser is text based as well as graphical. The text based links web browser is shipped by most of the standard Linux distributions by default. If links is not installed in your system by default you may install it from the repo. Elinks is a fork of links.

# apt-get install links
# yum install links

After installing links, you can browse any websites within the terminal as shown below in the screen cast..

# links www.tecmint.com

Use UP and DOWN arrow keys to navigate. Right arrow Key on a link will redirect you to that link and Left arrow key will bring you back to the last page. To QUIT press q.

Here is how it seems to access Tecmint using links tool.

If you are interested in installing GUI of links, you may need to download latest source tarball (i.e. version 2.9) from http://links.twibright.com/download/.

Alternatively, you may use following wget command to download and install as suggested below.

# wget http://links.twibright.com/download/links-2.9.tar.gz
# tar -xvf links-2.9.tar.gz
# cd links-2.9
# ./configure –enable-graphics
# make
# make install

Note: You need to install packages (libpng, libjpeg, TIFF library, SVGAlib, XFree86, C Compiler and make), if not already installed to successfully compile the package.

2. links2

Links2 is a graphical web browser version of Twibright Labs Links web browser. This browser has support for mouse and clicks. Designed specially for speed without any CSS support, fairly good HTML and JavaScript support with limitations.

To install links2.

# apt-get install links2
# yum install links2

3. lynx

A text based web browser released under GNU GPLv2 license and written in ISO C. lynx is highly configurable web browser and Savior for many SYSAdmin. It has the reputation of being the oldest web browser that is being used and still actively developed.

To install lynx.

# apt-get install lynx
# yum install lynx

After installing lynx, type the following command to browse the website as shown below in the screen cast..

# lynx www.tecmint.com

If you are interested in knowing a bit more about links and lynx web browser, you may like to visit the below link:

1. Web Browsing with Lynx and Links Command Line Tools

4. youtube-dl

youtube-dl is a platform independent application which can be used to download videos from youtube and a few other sites. Written primarily in python and released under GNU GPL License, the application works out of the box. (Since youtube don’t allow you to download videos, it may be illegal to use it. Check the laws before you start using this.)

To install youtube-dl.

# apt-get install youtube-dl
# yum install youtube-dl

After installing, try to download files from the Youtube site, as shown in the below screen cast.

# youtube-dl https://www.youtube.com/watch?v=ql4SEy_4xws

If you are interested in knowing more about youtube-dl you may like to visit the below link:

1. YouTube-DL – A Command Line Youtube Video Downloader for Linux

5. fetch

It is a command utility for unix-like operating system that is used for URL retrieval. It supports a lot of options like fetching ipv4 only address, ipv6 only address, no redirect, exit after successful file retrieval request, retry, etc.

Fetch can be Downloaded and installed from the link below

http://sourceforge.net/projects/fetch/?source=typ_redirect

But before you compile and run it, you should install HTTP Fetcher. Download HTTP Fetcher from the link below.

http://sourceforge.net/projects/http-fetcher/?source=typ_redirect

6. Axel

Axel is a command-line based download accelerator for Linux. Axel makes it possible to download a file at much faster speed through single connection request for multiple copies of files in small chunks through multiple http and ftp connections.

To install Axel.

# apt-get install axel
# yum install axel

After axel installed, you may use following command to download any given file, as shown in the screen cast.

# axel http://mirror.cse.iitk.ac.in/archlinux/iso/2015.04.01/archlinux-2015.04.01-dual.iso

7. aria2

aria2 is a command-line based download utility that is lightweight and support multi-protocol (HTTP, HTTPS, FTP, BitTorrent and Metalink). It can use metalinks files to simultaneously download ISO files from more than one server. It can serve as a Bit torrent client as well.

To install aria2.

# apt-get install aria2
# yum install aria2

Once aria2 installed, you can fire up the following command to download any given file…

# aria2c http://cdimage.debian.org/debian-cd/7.8.0/multi-arch/iso-cd/debian-7.8.0-amd64-i386-netinst.iso

If you’re interested to know more at aria2 and their switches, read the following article.

1. Aria2 – A Multi-Protocol Command-Line Download Manager for Linux

8. w3m

w3m is a another open source text-based web browser very similar to lynx, which runs on a terminal. It uses emacs-w3m an Emacs interface for w3m to browse web sites within emacs interface.

To install w3m.

# apt-get install w3m
# yum install w3m

After installing w3m, fire up the following command to browse the website as shown below.

# w3m www.tecmint.com

That’s all for now.

Source

Browsh is an open source, simple and modern text-based browser that renders in TTY terminal environments. It is made up of a minimal Golang CLI front-end and a browser web-extension (headless Firefox) which actually offers most of the functionality to create a purely text-based version of web pages and web apps.

This browser renders anything that a modern browser can; HTML5, CSS3, JS, video as well as WebGL. It is importantly a bandwidth-saver, designed to run on a remote server and accessed via SSH/Mosh or the in-browser HTML service so as to notably reduce bandwidth.

Read Also: Command Line Web Browsing with Lynx and Links Tools

Browsh is useful only when you don’t have good Internet connection. It also helps you to avoid battery-drain of a modern browser from your laptop or low-powered device such as a Raspberry Pi.

Live SSH Demo – Just point your SSH client to ssh brow.sh, no authentication needed and session last 5minutes and are logged.

How to Install Browsh Text-Based Browser in Linux

Browsh’s requirements are a latest version of Firefox and a terminal client with true colour support. Once you have those you can download the suitable binary or package for your Linux distribution using following commands.

Install Browsh on RHEL/CentOS & Fedora

--------- On 64-bit --------- 
# wget https://github.com/browsh-org/browsh/releases/download/v1.4.12/browsh_1.4.12_linux_amd64.rpm
# rpm -Uvh browsh_1.4.12_linux_amd64.rpm

--------- On 32-bit ---------
# wget https://github.com/browsh-org/browsh/releases/download/v1.4.12/browsh_1.4.12_linux_386.rpm
# rpm -Uvh browsh_1.4.12_linux_386.rpm

Install Browsh on Debian/Ubuntu & Linux Mint

--------- On 64-bit --------- 
$ wget https://github.com/browsh-org/browsh/releases/download/v1.4.12/browsh_1.4.12_linux_amd64.deb
$ sudo dpkg -i browsh_1.4.12_linux_amd64.deb 

--------- On 32-bit ---------
$ wget https://github.com/browsh-org/browsh/releases/download/v1.4.12/browsh_1.4.12_linux_386.deb
$ sudo dpkg -i browsh_1.4.12_linux_386.deb 

If you don’t want to install .deb and .rpm versions, you can download static binaries and execute it as shown.

--------- On 64-bit --------- 
$ wget https://github.com/browsh-org/browsh/releases/download/v1.4.12/browsh_1.4.12_linux_amd64
$ chmod 755 browsh_1.4.12_linux_amd64
$ ./browsh_1.4.12_linux_amd64

--------- On 64-bit --------- 
$ wget https://github.com/browsh-org/browsh/releases/download/v1.4.12/browsh_1.4.12_linux_386
$ chmod 755 browsh_1.4.12_linux_386
$ ./browsh_1.4.12_linux_386

There are also a Docker image that comes with the latest version of Firefox bundled, all you need to do is just pull and run the TTY client with.

$ docker run -it --rm browsh/browsh

How to Use Browsh Text-Based Browser in Linux

Once you have Browsh installed, you can run browsh on the terminal as shown.

$ browsh

Most keys and mouse gestures should work as you would expect on a desktop browser, the following are the basic ones for you to get started.

• F1 – opens the documentation
• ARROW KEYS, PageUP, PageDown – scrolling
• CTRL+l – focus the URL bar
• CTRL+r – reload page
• CTRL+t – open new tab
• CTRL+w – close a tab
• BACKSPACE – go back in history
• CTRL+q – exit the program

You might also like to read these following related articles.

1. 8 Command Line Tools for Browsing Websites and Downloading Files in Linux
2. Googler: A Command Line Tool to Do ‘Google Search’ from Linux Terminal

1. Cloud Commander – Web File Manager to Control Linux File and Programs via Browser
2. Tig – A Command Line Browser for Git Repositories

For more information, go to: https://www.brow.sh/

That’s all! Browsh is a simple, fully-modern text-based browser that runs in TTY terminal environments and in any browser, and can render anything that a modern browser can. In this guide, we have explained how to install and use Browsh in Linux. Try it out and share your thoughts with us in the comments.

Source

Let us start by defining a bad sector/block, it’s a section on a disk drive or flash memory that can not be read from or written to anymore, as a result of a fixed physical damage on the disk surface or failed flash memory transistors.

As bad sectors continue to accumulate, they can undesirably or destructively affect your disk drive or flash memory capacity or even lead to a possible hardware failure.

It is also important to note that the presence of bad blocks should alert you to start thinking of getting a new disk drive or simply mark the bad blocks as unusable.

Therefore, in this article, we will go through the necessary steps that can enable you determine the presence or absence of bad sectors on your Linux disk drive or flash memory using certain disk scanning utilities.

That said, below are the methods:

Check Bad Sectors in Linux Disks Using badblocks Tool

A badblocks program enables users to scan a device for bad sectors or blocks. The device can be a hard disk or an external disk drive, represented by a file such as /dev/sdc.

Firstly, use the fdisk command with superuser privileges to display information about all your disk drives or flash memory plus their partitions:

$ sudo fdisk -l

Then scan your Linux disk drive to check for bad sectors/blocks by typing:

$ sudo badblocks -v /dev/sda10 > badsectors.txt

In the command above, badblocks is scanning device /dev/sda10 (remember to specify your actual device) with the -v enabling it to display details of the operation. In addition, the results of the operation are stored in the file badsectors.txt by means of output redirection.

In case you discover any bad sectors on your disk drive, unmount the disk and instruct the operating system not to write to the reported sectors as follows.

You will need to employ e2fsck (for ext2/ext3/ext4 file systems) or fsck command with the badsectors.txt file and the device file as in the command below.

The -l option tells the command to add the block numbers listed in the file specified by filename (badsectors.txt) to the list of bad blocks.

------------ Specifically for ext2/ext3/ext4 file-systems ------------ 
$ sudo e2fsck -l badsectors.txt /dev/sda10

OR

------------ For other file-systems ------------ 
$ sudo fsck -l badsectors.txt /dev/sda10

Scan Bad Sectors on Linux Disk Using Smartmontools

This method is more reliable and efficient for modern disks (ATA/SATA and SCSI/SAS hard drives and solid-state drives) which ship in with a S.M.A.R.T (Self-Monitoring, Analysis and Reporting Technology) system that helps detect, report and possibly log their health status, so that you can figure out any impending hardware failures.

You can install smartmontools by running the command below:

------------ On Debian/Ubuntu based systems ------------ 
$ sudo apt-get install smartmontools

------------ On RHEL/CentOS based systems ------------ 
$ sudo yum install smartmontools

Once the installation is complete, use smartctl which controls the S.M.A.R.T system integrated into a disk. You can look through its man page or help page as follows:

$ man smartctl
$ smartctl -h

Now execute the smartctrl command and name your specific device as an argument as in the following command, the flag -H or --health is included to display the SMART overall health self-assessment test result.

$ sudo smartctl -H /dev/sda10

The result above indicates that your hard disk is healthy, and may not experience hardware failures any soon.

For an overview of disk information, use the -a or --all option to print out all SMART information concerning a disk and -x or --xall which displays all SMART and non-SMART information about a disk.

In this tutorial, we covered a very important topic concerning disk drive health diagnostics, you can reach us via the feedback section below to share your thoughts or ask any questions.

Source

One of the most important and interesting topics under Linux administration is I/O redirection. This feature of the command line enables you to redirect the input and/or output of commands from and/or to files, or join multiple commands together using pipes to form what is known as a “command pipeline”.

All the commands that we run fundamentally produce two kinds of output:

1. the command result – data the program is designed to produce, and
2. the program status and error messages that informs a user of the program execution details.

In Linux and other Unix-like systems, there are three default files named below which are also identified by the shell using file descriptor numbers:

1. stdin or 0 – it’s connected to the keyboard, most programs read input from this file.
2. stdout or 1 – it’s attached to the screen, and all programs send their results to this file and
3. stderr or 2 – programs send status/error messages to this file which is also attached to the screen.

Therefore, I/O redirection allows you to alter the input source of a command as well as where its output and error messages are sent to. And this is made possible by the “<” and “>” redirection operators.

How To Redirect Standard Output to File in Linux

You can redirect standard output as in the example below, here, we want to store the output of the top command for later inspection:

$ top -bn 5 >top.log

Where the flags:

1. -b – enables top to run in batch mode, so that you can redirect its output to a file or another command.
2. -n – specifies the number of iterations before the command terminates.

You can view the contents of top.log file using cat command as follows:

$ cat top.log

To append the output of a command, use the “>>” operator.

For instance to append the output of top command above in the top.log file especially within a script (or on the command line), enter the line below:

$ top -bn 5 >>top.log

Note: Using the file descriptor number, the output redirect command above is the same as:

$ top -bn 5 1>top.log

How To Redirect Standard Error to File in Linux

To redirect standard error of a command, you need to explicitly specify the file descriptor number, 2 for the shell to understand what you are trying to do.

For example the ls command below will produce an error when executed by a normal system user without root privileges:

$ ls -l /root/

You can redirect the standard error to a file as below:

$ ls -l /root/ 2>ls-error.log
$ cat ls-error.log 

In order to append the standard error, use the command below:

$ ls -l /root/ 2>>ls-error.log

How To Redirect Standard Output/ Error To One File

It is also possible to capture all the output of a command (both standard output and standard error) into a single file. This can be done in two possible ways by specifying the file descriptor numbers:

1. The first is a relatively old method which works as follows:

$ ls -l /root/ >ls-error.log 2>&1

The command above means the shell will first send the output of the ls command to the file ls-error.log (using >ls-error.log), and then writes all error messages to the file descriptor 2 (standard output) which has been redirected to the file ls-error.log (using 2>&1). Implying that standard error is also sent to the same file as standard output.

2. The second and direct method is:

$ ls -l /root/ &>ls-error.log

You can as well append standard output and standard error to a single file like so:

$ ls -l /root/ &>>ls-error.log

How To Redirect Standard Input to File

Most if not all commands get their input from standard input, and by default standard input is attached to the keyboard.

To redirect standard input from a file other than the keyboard, use the “<” operator as below:

$ cat <domains.list 

How To Redirect Standard Input/Output to File

You can perform standard input, standard output redirection at the same time using sort command as below:

$ sort <domains.list >sort.output

How to Use I/O Redirection Using Pipes

To redirect the output of one command as input of another, you can use pipes, this is a powerful means of building useful command lines for complex operations.

For example, the command below will list the top five recently modified files.

$ ls -lt | head -n 5 

Here, the options:

1. -l – enables long listing format
2. -t – sort by modification time with the newest files are shown first
3. -n – specifies the number of header lines to show

Important Commands for Building Pipelines

Here, we will briefly review two important commands for building command pipelines and they are:

xargs which is used to build and execute command lines from standard input. Below is an example of a pipeline which uses xargs, this command is used to copy a file into multiple directories in Linux:

$ echo /home/aaronkilik/test/ /home/aaronkilik/tmp | xargs -n 1 cp -v /home/aaronkilik/bin/sys_info.sh

And the options:

1. -n 1 – instructs xargs to use at most one argument per command line and send to the cp command
2. cp – copies the file
3. -v – displays progress of copy command.

For more usage options and info, read through the xargs man page:

$ man xargs 

A tee command reads from standard input and writes to standard output and files. We can demonstrate how teeworks as follows:

$ echo "Testing how tee command works" | tee file1 

File or text filters are commonly used with pipes for effective Linux file operations, to process information in powerful ways such as restructuring output of commands (this can be vital for generation of useful Linux reports), modifying text in files plus several other Linux system administration tasks.

To learn more about Linux filters and pipes, read this article Find Top 10 IP Addresses Accessing Apache Server, shows a useful example of using filters and pipes.

In this article, we explained the fundamentals of I/O redirection in Linux. Remember to share your thoughts via the feedback section below.

Source

Gogo is an impressive way to bookmark directories inside your shell. It allows you to create shortcuts to long and complicated paths in Linux. This way, you don’t have to type or remember long and complicated paths anymore in Linux.

For example, if you have a directory ~/Documents/Phone-Backup/Linux-Docs/Ubuntu/, using gogo, you can create an alias (a shortcut name), for instance Ubuntu to access it without typing the whole path anymore. No matter your current working directory, you can move into ~/cd Documents/Phone-Backup/Linux-Docs/Ubuntu/by simply using the alias Ubuntu.

Read Also: bd – Quickly Go Back to a Parent Directory Instead of Typing “cd ../../..” Redundantly

In addition, it also allows you to create aliases for connecting directly into directories on remote Linux servers.

How to Install Gogo in Linux Systems

To install Gogo, first clone the gogo repository from Github and then copy the gogo.py to any directory in your PATH environmental variable (if you already have the ~/bin/ directory, you can place it here, otherwise create it).

$ git clone https://github.com/mgoral/gogo.git
$ cd gogo/
$ mkdir -p ~/bin        #run this if you do not have ~/bin directory
$ cp gogo.py ~/bin/

Then add a function from gogo.sh to your ~/.bashrc (for Bash) or ~/.zshrc (for Zsh) file and verity it as shown.

$ cat gogo.sh >> ~/.bashrc
$ tail  ~/.bashrc
OR
$ cat gogo.sh >> ~/.zshrc 

How to Use Gogo in Linux Systems

To start using gogo, you need to logout and login back to use it. Gogo stores its configuration in ~/.config/gogo/gogo.conf file (which should be auto created if it doesn’t exist) and has the following syntax.

# Comments are lines that start from '#' character.
default = ~/something
alias = /desired/path
alias2 = /desired/path with space
alias3 = "/this/also/works"
zażółć = "unicode/is/also/supported/zażółć gęślą jaźń"

If you run gogo run without any arguments, it will go to the directory specified in default; this alias is always available, even if it’s not in the configuration file, and points to $HOME directory.

To display the current aliases, use the -l switch. From the following screenshot, you can see that default points to ~/home/tecmint which is user tecmint’s home directory on the system.

$ gogo -l   

Below is an example of running gogo without any arguments.

$ cd Documents/Phone-Backup/Linux-Docs/
$ gogo
$ pwd

To create a shortcut to a long path, move into the directory you want and use the -a flag to add an alias for that directory in gogo, as shown.

$ cd Documents/Phone-Backup/Linux-Docs/Ubuntu/
$ gogo -a Ubuntu
$ gogo
$ gogo -l
$ gogo -a Ubuntu
$ pwd

You can also create aliases for connecting directly into directories on a remote Linux servers. To do this, simple add the following lines to gogo configuration file, which can be accessed using -e flag, this will use the editor specified in the $EDITOR env variable.

$ gogo -e

One configuration file opens, add these following lines to it.

sshroot = ssh://root@192.168.56.5:/bin/bash  /root/
sshtdocs = ssh://tecmint@server3  ~/tecmint/docs/

To display the gogo help message, use the -h option.

$ gogo -h

One notable limitation of gogo is its lack of support for auto-completion – when accessing subdirectories/child directories under an aliased long path.

Gogo github repository: https://github.com/mgoral/gogo

Gogo is a remarkable way that comes in handy, for creating shortcuts to long and complicated paths in Linux. Try it out and share your thoughts about it or ask any questions via the comment form below.

Source

In a recent article, we talked about Gogo – a tool to create shortcuts for long paths in a Linux shell. Although gogo is a great way to bookmark your favorite directories inside a shell, however, it has one major limitation; it lacks an auto-completion feature.

Because of the above reason, we went all out to find a similar utility with auto-completion support – where the shell can prompt with suggestions of the available aliases (shortcuts to long and complicated paths) and luckily, after crawling through Github, we discovered Goto.

Goto is a shell utility to quickly navigate to aliased directories, with support for auto-completion. It comes with a nice auto-completion script so that once you press the tab key after the goto command or after typing a few charters of an existing alias, bash or zsh prompts with suggestions of the aliases or auto complete the name, respectively.

Goto also has additional options for unregistering an alias, expanding an alias’s value as well as cleaning up aliases of deleted directories. Note that goto’s auto-completion only works for aliases; it is separate from shell auto-completion for commands or filenames.

How to Install and Use Goto in Linux Systems

To install Goto, start by cloning the goto repository from Github and move into the local repository directory, then run the install shell script with root user privileges using the sudo command as shown.

$ cd Downloads/
$ git clone https://github.com/iridakos/goto.git
$ cd goto
$ ls
$ sudo ./install

This will install goto in /usr/local/share/goto.sh, and it will add a line in your ~/.bashrc (for Bash) or ~/.zshrc (for Zsh) shell startup file, to source it.

Now restart your terminal to start using goto. To create an alias for a directory, register the alias with the -rflag as follows.

$ goto -r march ~/Documents/Tecmint.com-Articles/March/

To alias your current directory, use this syntax which will automatically be aliased to the whole path.

$ goto -r home . 

When you type goto and press the tab key, it will show all registered aliases and when you type a few letters of a registered alias, goto will auto complete the name. However, to view a list of your currently registered aliases, use the -l flag.

$ goto -l

To expand an alias to its value using the following command.

$ goto -x scripts
$ goto -x march

Goto also allows you to unregister an alias, using the -u option.

$ goto -l
$ goto -u march
$ goto -l

If you have removed aliased directories (for example if you have deleted the directories ~/Documents/Tecmint.com-Articles/March and ~/bin/shellscripts/recon from the filesystem), yet they still have aliases in goto, you can cleanup all these aliases from goto with the -c flag.

$ goto -c

A significant limitation of goto is that it doesn’t allow for accessing a sub-directory under an aliased directory, which is a feature present in Gogo.

For more information, consult the goto help message with the -h option.

$ goto -h

Goto Github repository: https://github.com/iridakos/goto

Goto is a powerful way to bookmark your favorite directories inside a shell, with auto-completion support, in Linux. It has more useful features compared to Gogo, as explained above. Give it a try and share with us, your thoughts about it via the feedback form below.

Source

As a newbie, one of the many things you should master in Linux is identification of devices attached to your system. It may be your computer’s hard disk, an external hard drive or removable media such USB drive or SD Memory card.

Using USB drives for file transfer is so common today, and for those (new Linux users) who prefer to use the command line, learning the different ways to identify a USB device name is very important, when you need to format it.

Suggested Read: 3 Ways to Delete All Files in a Directory

Once you attach a device to your system such as a USB, especially on a desktop, it is automatically mounted to a given directory, normally under /media/username/device-label and you can then access the files in it from that directory. However, this is not the case with a server where you have to manually mount a device and specify its mount point.

Linux identifies devices using special device files stored in /dev directory. Some of the files you will find in this directory include /dev/sda or /dev/hda which represents your first master drive, each partition will be represented by a number such as /dev/sda1 or /dev/hda1 for the first partition and so on.

$ ls /dev/sda* 

Now let’s find out device names using some different command-line tools as shown:

Find Out Plugged USB Device Name Using df Command

To view each device attached to your system as well as its mount point, you can use the df command (checks Linux disk space utilization) as shown in the image below:

$ df -h

Use lsblk Command to Find USB Device Name

You can also use the lsblk command (list block devices) which lists all block devices attached to your system like so:

$ lsblk

Identify USB Device Name with fdisk Utility

fdisk is a powerful utility which prints out the partition table on all your block devices, a USB drive inclusive, you can run it will root privileges as follows:

$ sudo fdisk -l

Determine USB Device Name with dmesg Command

dmesg is an important command that prints or controls the kernel ring buffer, a data structure which stores information about the kernel’s operations.

Run the command below to view kernel operation messages which will as well print information about your USB device:

$ dmesg

That is all for now, in this article, we have covered different approaches of how to find out a USB device name from the command line. You can also share with us any other methods for the same purpose or perhaps offer us your thoughts about the article via the response section below.

Source

Back in the mid-1990’s when the Internet was still in its infancy, a Swedish programmer named Daniel Stenbergstarted a project that eventually grew into what we know as curl today.

Initially, he aimed at developing a bot that would download currency exchange rates from a web page periodically and would provide Swedish Kronor equivalents in US dollars to IRC users.

Long story short, the project thrived, adding several protocols and features along the way – and the rest is history. Now let’s dive in with both feet and learn how to use curl to transfer data and more in Linux!

We have put together the following list of 15 curl commands for you.

1. View curl Version

The -V or --version options will not only return the version, but also the supported protocols and features in your current version.

$ curl --version

curl 7.47.0 (x86_64-pc-linux-gnu) libcurl/7.47.0 GnuTLS/3.4.10 zlib/1.2.8 libidn/1.32 librtmp/2.3
Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtmp rtsp smb smbs smtp smtps telnet tftp 
Features: AsynchDNS IDN IPv6 Largefile GSS-API Kerberos SPNEGO NTLM NTLM_WB SSL libz TLS-SRP UnixSockets 

2. Download a File

If you want to download a file, you can use curl with the -O or -o options. The former will save the file in the current working directory with the same name as in the remote location, whereas the latter allows you to specify a different filename and/or location.

$ curl -O http://yourdomain.com/yourfile.tar.gz # Save as yourfile.tar.gz
$ curl -o newfile.tar.gz http://yourdomain.com/yourfile.tar.gz # Save as newfile.tar.gz

3. Resume an Interrupted Download

If a download was interrupted for some reason (for example, using Ctrl + c), you can resume it very easily. The use of -C – (dash C, space dash) tells curl to resume the download beginning where it left off.

$ curl -C - -O http://yourdomain.com/yourfile.tar.gz

4. Download Multiple Files

With the following command you will download info.html and about.html from http://yoursite.com and http://mysite.com, respectively, in one go.

$ curl -O http://yoursite.com/info.html -O http://mysite.com/about.html 

5. Download URLs From a File

If you combine curl with xargs, you can download files from a list of URLs in a file.

$ xargs -n 1 curl -O < listurls.txt

6. Use a Proxy with or without Authentication

If you are behind a proxy server listening on port 8080 at proxy.yourdomain.com, do.

$ curl -x proxy.yourdomain.com:8080 -U user:password -O http://yourdomain.com/yourfile.tar.gz

where you can skip -U user:password if your proxy does not require authentication.

7. Query HTTP Headers

HTTP headers allow the remote web server to send additional information about itself along with the actual request. This provides the client with details on how the request is being handled.

To query the HTTP headers from a website, do:

$ curl -I www.tecmint.com

This information is also available in your browser’s developer tools.

8. Make a POST request with Parameters

The following command will send the firstName and lastName parameters, along with their corresponding values, to https://yourdomain.com/info.php.

$ curl --data "firstName=John&lastName=Doe" https://yourdomain.com/info.php

You can use this tip to simulate the behavior of a regular HTML form.

9. Download Files from an FTP Server with or without Authentication

If a remote FTP server is expecting connections at ftp://yourftpserver, the following command will download yourfile.tar.gz in the current working directory.

$ curl -u username:password -O ftp://yourftpserver/yourfile.tar.gz 

where you can skip -u username:password if the FTP server allows anonymous logins.

10. Upload Files to an FTP server with or without Authentication

To upload a local file named mylocalfile.tar.gz to ftp://yourftpserver using curl, do:

$ curl -u username:password -T mylocalfile.tar.gz ftp://yourftpserver

11. Specify User Agent

The user agent is part of the information that is sent along with an HTTP request. This indicates which browser the client used to make the request. Let’s see what our current curl version uses as default, and let’s change it later to “I am a new web browser”:

$ curl -I http://localhost --user-agent "I am a new web browser"

12. Store Website Cookies

Want to see which cookies are downloaded to your computer when you browse to https://www.cnn.com? Use the following command to save them to cnncookies.txt. You can then use cat command to view the file.

$ curl --cookie-jar cnncookies.txt https://www.cnn.com/index.html -O

13. Send Website Cookies

You can use the cookies retrieved in the last tip in subsequent requests to the same site.

$ curl --cookie cnncookies.txt https://www.cnn.com

14. Modify Name Resolution

If you’re a web developer and want to test a local version of yourdomain.com before pushing it live, you can make curl resolve http://www.yourdomain.com to your localhost like so:

$ curl --resolve www.yourdomain.com:80:localhost http://www.yourdomain.com/

Thus, the query to http://www.yourdomain.com will tell curl to request the site from localhost instead of using DNS or the /etc/hosts file.

15. Limit Download Rate

To prevent curl from hosing your bandwidth, you can limit the download rate to 100 KB/s as follows.

$ curl --limit-rate 100K http://yourdomain.com/yourfile.tar.gz -O

Summary

In this article we have shared a brief history of the origins of curl and explained how to use it through 15 practical examples.

Do you know of any other curl commands that we may have missed in this article? Feel free to share them with our community in the comments! Also, if you have questions feel free to let us know. We look forward to hearing from you!

Source

Have you ever typed a command and rushed to hit Enter, only to find that you had a typo in it? While you can use the up and down arrows to navigate the command history and edit the typo, there’s an easier and faster way.

Don’t Miss: The Power of Linux “History Command” in Bash Shell

In this tip, we shall cover a simple and handy method of dealing with a command line typo, let’s suppose you wanted to see if there’s a service listening on port 22, but accidentally typed nestat instead of netstat.

You can easily replace the typo with the correct command and execute it like so:

# nestat -npltu | grep 22
# ^nestat^netstat

That’s right. Using two carat signs (they should be followed by the typo and the right word, respectively) you can correct the typo and run the command automatically afterwards.

Assume you are running short on disk space and you wanted to free up, by searching something that’s a waste of space and removing it or moving it to an archive medium. How do you track down right stuff to delete, that saves maximum space?

Linux provides a standard du command, which scans entire disk and shows you which directories hold the huge amount of data. That can assist you narrow your search to the things most useful deleting.

However, that only shows you what’s huge. What you actually want to know is what’s too huge. By default, du command will not let you differentiate between data that’s huge because you are doing something that needs it to be huge, and data that’s huge because you unpacked it once and ignored about it.

Most Linux file systems, by default only shows when a file was last accessed, but not shown when it was written, modified or even read. So if you created a huge amount of data years ago, forgot to delete it and have never used it since, then it is important to use those last-access time stamps to know the difference between used and unused data.

Agedu pronounced as (age dee you) is an open source and free utility (much like du command) that helps system administrators to track down wasted disk space used by old files and delete them to free up some space.

Agedu does a complete scan and produces reports that shows how much disk space is utilized by each directory and sub directory along with the last-access times of files. In simple words, it simply helps you to free up disk space.

Agedu Features

1. Creates graphical reports.
2. Produces data output in HTML format.
3. Generates HTML reports with hyperlinks to other directories for easy navigation to gather reports.
4. Provides more configurable options.

How Does Agedu Works?

From the man page:

agedu is a program which does this. It does basically the same sort of disk scan as du, but it also records the last-access times of everything it scans. Then it builds an index that lets it efficiently generate reports giving a summary of the results for each subdirectory, and then it produces those reports on demand.

How to Install Agedu in Linux Systems

On Debian/Ubuntu, agedu is available to install from the default system repositories using following apt-get command as shown.

$ sudo apt-get install agedu

On RHEL/CentOS, you need to turn on EPEL repository to install Agedu using following yum command as shown.

# yum install epel-release
# yum install agedu

Fedora and Arch Linux users, simply type the following command to install Agedu.

$ sudo dnf install agedu  [On Fedora]
$ sudo yaourt -S agedu    [On Arch Linux]

On other Linux distributions, you can compile Agedu from source as shown.

$ wget https://www.chiark.greenend.org.uk/~sgtatham/agedu/agedu-20180329.af641e6.tar.gz
$ tar -xvf agedu-20180329.af641e6.tar.gz
$ cd agedu-20180329.af641e6
$ ./configure
$ make
$ sudo make install

How to Track Wasted Disk Space Using Agedu

The following command will do a complete scan of /home/tecmint directory and its sub-directories and creates a special index file containing its data structure.

# agedu -s /home/tecmint/

Built pathname index, 232578 entries, 22842517 bytes of index                                                                                                                
Faking directory atimes
Building index
Final index file size = 97485984 bytes

Next, type the following command to query newly created index file.

# agedu -w

Sample Output :

Using Linux /proc/net magic authentication
URL: http://localhost:34895/

Now, type the following command to open the URL using any web browser.

# http://localhost:34895/

Graphical Output :

The below screen shows the graphical representation of disk usage of /home/tecmint along with its sub directories using various colors to show the difference between disused and recently accessed data.

Click on any sub directory to see the reports of its sub-directories. To terminate this mode, just press [CTRL+D] on command line.

To create and set custom port number for agedu, type the following command.

# agedu -w --address 127.0.0.1:8081

Sample Output :

Using Linux /proc/net magic authentication
URL: http://127.0.0.1:8081/

Enable password protection for Agedu using the following command.

# agedu -w --address 127.0.0.1:8081 --auth basic

Username: agedu
Password: n2tx16jejnbzmuur
URL: http://127.0.0.1:8081/

Access Agedu reports using terminal mode.

# agedu -t /home/tecmint

8612        /home/tecmint/.AndroidStudio3.1
3684        /home/tecmint/.PlayOnLinux
604         /home/tecmint/.ScreamingFrogSEOSpider
2416        /home/tecmint/.TelegramDesktop
61960       /home/tecmint/.Write
1508        /home/tecmint/.adobe
20          /home/tecmint/.aptitude
48          /home/tecmint/.byobu
1215948     /home/tecmint/.cache
3096        /home/tecmint/.cinnamon
1421828     /home/tecmint/.config
12          /home/tecmint/.dbus
8           /home/tecmint/.emacs.d
780         /home/tecmint/.fonts
...

You see the output similar to du command. Let’s see the old files which are not accessed for long time. For example, to see only old files that are not accessed in last 12 months or more.

# agedu -t /home/tecmint -a 12m

2416        /home/tecmint/.TelegramDesktop
1500        /home/tecmint/.adobe
46776       /home/tecmint/.cache
1840        /home/tecmint/.cinnamon
142796      /home/tecmint/.config
636         /home/tecmint/.gconf
88          /home/tecmint/.gimp-2.8
12          /home/tecmint/.gnome
112         /home/tecmint/.java
108         /home/tecmint/.kde
8           /home/tecmint/.links2
16          /home/tecmint/.linuxmint
6804        /home/tecmint/.local
12          /home/tecmint/.mindterm
40920       /home/tecmint/.mozilla
4           /home/tecmint/.oracle_jre_usage
12          /home/tecmint/.parallel
24          /home/tecmint/.shutter
6840        /home/tecmint/.softmaker
336         /home/tecmint/.themes
....

Let’s find out how much disk space taken by MP3 files by using following command.

# agedu -s . --exclude '*' --include '*.mp3'

Again to see reports run the following command.

# agedu -w

To deleted files and free up disk space, use the following command.

# rm -rf /downloads/*.mp3

How to remove agedu index file? First see the size of the index file with the following command.

# ls agedu.dat -lh

Sample output :

-rw------- 1 tecmint tecmint 35M Apr 10 12:05 agedu.dat

To remove index file, just enter.

# agedu -R

For more information on agedu command options and usage, please read the man pages or visit agedu home page.

# man agedu

If you know any tool that we haven’t mentioned in this site. Please let us know about it via comment box below.

Source