{"id":13786,"date":"2019-04-06T06:27:33","date_gmt":"2019-04-06T06:27:33","guid":{"rendered":"http:\/\/www.appservgrid.com\/paw92\/?p=13786"},"modified":"2019-04-06T06:27:33","modified_gmt":"2019-04-06T06:27:33","slug":"how-to-limit-the-network-bandwidth-used-by-applications-in-a-linux-system-with-trickle","status":"publish","type":"post","link":"https:\/\/www.appservgrid.com\/paw92\/index.php\/2019\/04\/06\/how-to-limit-the-network-bandwidth-used-by-applications-in-a-linux-system-with-trickle\/","title":{"rendered":"How to Limit the Network Bandwidth Used by Applications in a Linux System with Trickle"},"content":{"rendered":"

Have you ever encountered situations where one application dominated you all network bandwidth? If you have ever been in a situation where one application ate all your traffic, then you will value the role of the trickle bandwidth shaper application. Either you are a\u00a0system admin<\/strong>\u00a0or just a\u00a0Linux user<\/strong>, you need to learn how to control the\u00a0upload<\/strong>\u00a0and\u00a0download<\/strong>\u00a0speeds for applications to make sure that your network bandwidth is not burned by a single application.<\/p>\n

\n

\"Bandwidth<\/a><\/p>\n

Install Trickle Bandwidth Limit in Linux<\/p>\n<\/div>\n

What is Trickle?<\/h3>\n

Trickle<\/strong>\u00a0is a network bandwidth shaper tool that allows us to manage the upload and download speeds of applications in order to prevent any single one of them to hog all (or most) of the available bandwidth. In few words, trickle lets you control the network traffic rate on a per-application basis, as opposed to per-user control, which is the classic example of bandwidth shaping in a client-server environment, and is probably the setup we are more familiar with.<\/p>\n

How Trickle Works?<\/h3>\n

In addition, trickle can help us to define priorities on a per-application basis, so that when overall limits have been set for the entire system, priority apps will still get more bandwidth automatically. To accomplish this task, trickle sets traffic limits to the way in which data is sent to, and received from, sockets using TCP connections. We must note that, other than the data transfer rates, trickle does not modify in any way the behavior of the process it is shaping at any given moment.<\/p>\n

What Can\u2019t Trickle do?<\/h3>\n

The only limitation, so to speak, is that trickle will not work with statically linked applications or binaries with the\u00a0SUID<\/strong>\u00a0or\u00a0SGID<\/strong>\u00a0bits set since it uses dynamic linking and loading to place itself between the shaped process and its associated network socket. Trickle then acts as a proxy between these two software components.<\/p>\n

Since trickle does not require superuser privileges in order to run, users can set their own traffic limits. Since this may not be desirable, we will explore how to set overall limits that system users cannot exceed. In other words, users will still be able to manage their traffic rates, but always within the boundaries set by the system administrator.<\/p>\n

In this article we will explain how to limit the network bandwidth used by applications in a Linux server with trickle. To generate the necessary traffic, we will use\u00a0ncftpput<\/strong>\u00a0and\u00a0ncftpget<\/strong>\u00a0(both tools are available by installing\u00a0ncftp<\/strong>) on the client (CentOS 7<\/strong>\u00a0server \u2013 dev1: 192.168.0.17<\/strong>), and\u00a0vsftpd<\/strong>\u00a0on the server (Debian Wheezy 7.5<\/strong>\u00a0\u2013 dev2:\u00a0192.168.0.15<\/strong>) for demonstration purposes. The same instructions also works on RedHat, Fedora and Ubuntu based systems.<\/p>\n

Prerequisites<\/h4>\n

1<\/strong>. For\u00a0RHEL\/CentOS 7\/6<\/strong>,\u00a0enable the EPEL repository<\/a>. Extra Packages for Enterprise Linux (EPEL<\/strong>) is a repository of high-quality free and open-source software maintained by the Fedora project and is 100% compatible with its spinoffs, such as Red Hat Enterprise Linux and CentOS. Both trickle and ncftp are made available from this repository.<\/p>\n

2<\/strong>. Install\u00a0ncftp<\/strong>\u00a0as follows:<\/p>\n

# yum update && sudo yum install ncftp\t\t[On RedHat based systems]\r\n# aptitude update && aptitude install ncftp\t[On Debian based systems]\t\r\n<\/pre>\n
3<\/strong>. Set up a FTP server in a separate server. Please note that although FTP is inherently insecure, it is still widely used in cases when security in uploading or downloading files is not needed. We are using it in this article to illustrate the bounties of trickle and because it shows the transfer rates in\u00a0stdout<\/strong>\u00a0on the client, and we will leave the discussion of whether it should or should not be used for another date and time :).<\/p>\n
# yum update && yum install vsftpd \t\t[On RedHat based systems]\r\n# aptitude update && aptitude install vsftpd \t[On Debian based systems]\r\n<\/pre>\n
Now, edit the\u00a0\/etc\/vsftpd\/vsftpd.conf<\/strong>\u00a0file on the FTP server as follows:<\/p>\n
anonymous_enable=NO\r\nlocal_enable=YES\r\nchroot_local_user=YES\r\nallow_writeable_chroot=YES\r\n<\/pre>\n
After that, make sure to start vsftpd for your current session and to enable it for automatic start on future boots:<\/p>\n
# systemctl start vsftpd \t\t[For systemd-based systems]\r\n# systemctl enable vsftpd\r\n# service vsftpd start \t\t\t[For init-based systems]\r\n# chkconfig vsftpd on\r\n<\/pre>\n
4<\/strong>. If you chose to set up the FTP server in a\u00a0CentOS<\/strong>\/RHEL 7<\/strong>\u00a0droplet with SSH keys for remote access, you will need a password-protected user account with the appropriate directory and file permissions for uploading and downloading the desired content OUTSIDE root\u2019s home directory.<\/p>\n
You can then browse to your home directory by entering the following URL in your browser. A login window will pop up prompting you for a valid user account and password on the FTP server.<\/p>\n
ftp:\/\/192.168.0.15\r\n<\/pre>\n
If the authentication succeeds, you will see the contents of your home directory. Later in this tutorial you will be able to refresh that page to display the files that have been uploaded during previous steps.<\/p>\n
\n
\"FTP<\/a><\/p>\n
FTP Directory Tree<\/p>\n<\/div>\n
How to Install Trickle in Linux<\/h3>\n
1. Install trickle via\u00a0yum<\/strong>\u00a0or\u00a0aptitude<\/strong>.<\/p>\n
To ensure a successful installation, it is considered good practice to make sure the currently installed packages are up-to-date (using\u00a0yum update<\/strong>) before installing the tool itself.<\/p>\n
# yum -y update && yum install trickle \t\t        [On RedHat based systems]\r\n# aptitude -y update && aptitude install trickle \t[On Debian based systems]\r\n<\/pre>\n
2<\/strong>. Verify whether trickle will work with the desired binary.<\/p>\n
As we explained earlier, trickle will only work with binaries using dynamic, or shared, libraries. To verify whether we can use this tool with a certain application, we can use the well-known\u00a0ldd<\/strong>\u00a0utility, where\u00a0ldd<\/strong>\u00a0stands for list dynamic dependencies. Specifically, we will look for the presence of\u00a0glibc<\/strong>\u00a0(the GNU C library) in the list of dynamic dependencies of any given program because it is precisely that library which defines the system calls involved in communication through sockets.<\/p>\n
Run the following command against a given binary to see if trickle can be used to shape its bandwidth:<\/p>\n
# ldd $(which [binary]) | grep libc.so\r\n<\/pre>\n
For example,<\/p>\n
# ldd $(which ncftp) | grep libc.so\r\n<\/pre>\n
whose output is:<\/p>\n
# libc.so.6 => \/lib64\/libc.so.6 (0x00007efff2e6c000)\r\n<\/pre>\n
The string between brackets in the output may change from system to system and even between subsequent runs of the same command, since it represents the load address of the library in physical memory.<\/p>\n
If the above command does not return any results, it means that the binary it was run against does not use\u00a0libc<\/strong>and thus trickle cannot be used as bandwidth shaper in that case.<\/p>\n
Learn How to Use Trickle<\/h3>\n
The most basic usage of trickle is in standalone mode. Using this approach, trickle is used to explicitly define the download and upload speeds of a given application. As we explained earlier, for the sake of brevity, we will use the same application for download and upload tests.<\/p>\n
Running Trickle in Standalone Mode<\/h4>\n
We will compare the download and upload speeds with and without using trickle. The\u00a0-d<\/strong>\u00a0option indicates the download speed in KB\/s, while the\u00a0-u<\/strong>\u00a0flag tells trickle to limit the upload speed by the same unit. In addition, we will use the\u00a0-s<\/strong>\u00a0flag, which specifies that trickle should run in standalone mode.<\/p>\n
The basic syntax to run trickle in standalone mode is as follows:<\/p>\n
# trickle -s -d [download rate in KB\/s] -u [upload rate in KB\/s]\r\n<\/pre>\n
In order to perform the following examples on your own, make sure to have\u00a0trickle<\/strong>\u00a0and\u00a0ncftp<\/strong>\u00a0installed on the client machine (192.168.0.17<\/strong>\u00a0in my case).<\/p>\n
Example 1: Uploading a 2.8 MB PDF file with and without trickle.<\/h6>\n
We are using the freely-distributable\u00a0Linux Fundamentals<\/strong>\u00a0PDF file (available from\u00a0here<\/a>) for the following tests.<\/p>\n
You can initially download this file to your current working directory with the following command:<\/p>\n
# wget http:\/\/linux-training.be\/files\/books\/LinuxFun.pdf \r\n<\/pre>\n
The syntax to upload a file to our FTP server without trickle is as follows:<\/p>\n
# ncftpput -u username -p password 192.168.0.15  \/remote_directory local-filename \r\n<\/pre>\n
Where\u00a0\/remote_directory<\/strong>\u00a0is the path of the upload directory relative to username\u2019s home, and\u00a0local-filename<\/strong>\u00a0is a file in your current working directory.<\/p>\n
Specifically, without trickle we get a peak upload speed of\u00a052.02 MB\/s<\/strong>\u00a0(please note that this is not the real average upload speed, but an instant starting peak), and the file gets uploaded almost instantly:<\/p>\n
# ncftpput -u username -p password 192.168.0.15  \/testdir LinuxFun.pdf \r\n<\/pre>\n
Output:<\/p>\n
LinuxFun.pdf:                                        \t2.79 MB   52.02 MB\/s\r\n<\/pre>\n
With trickle, we will limit the upload transfer rate at\u00a05 KB\/s<\/strong>. Before uploading the file for the second time, we need to delete it from the destination directory; otherwise,\u00a0ncftp<\/strong>\u00a0will inform us that the file at the destination directory is the same that we are trying to upload, and will not perform the transfer:<\/p>\n
# rm \/absolute\/path\/to\/destination\/directory\/LinuxFun.pdf \r\n<\/pre>\n
Then:<\/p>\n
# trickle -s -u 5 ncftpput -u username -p password 111.111.111.111 \/testdir LinuxFun.pdf \r\n<\/pre>\n
Output:<\/p>\n
LinuxFun.pdf:                                        \t2.79 MB\t4.94 kB\/s\r\n<\/pre>\n
In the example above, we can see that the average upload speed dropped to\u00a0~5 KB\/s<\/strong>.<\/p>\n
Example 2: Downloading the same 2.8 MB PDF file with and without trickle<\/h6>\n
First, remember to delete the PDF from the original source directory:<\/p>\n
# rm \/absolute\/path\/to\/source\/directory\/LinuxFun.pdf \r\n<\/pre>\n
Please note that the following cases will download the remote file to the current directory in the client machine. This fact is indicated by the period (\u2018.<\/strong>\u2018) that appears after the IP address of the FTP server.<\/p>\n
Without trickle:<\/p>\n
# ncftpget -u username -p  password 111.111.111.111 . \/testdir\/LinuxFun.pdf \r\n<\/pre>\n
Output:<\/p>\n
LinuxFun.pdf:                                        \t2.79 MB  260.53 MB\/s\r\n<\/pre>\n
With trickle, limiting the download speed at\u00a020 KB\/s<\/strong>:<\/p>\n
# trickle -s -d 30 ncftpget -u username -p password 111.111.111.111 . \/testdir\/LinuxFun.pdf \r\n<\/pre>\n
Output:<\/p>\n
LinuxFun.pdf:                                        \t2.79 MB   17.76 kB\/s\r\n<\/pre>\n
Running Trickle in Supervised [unmanaged] Mode<\/h3>\n
Trickle can also run in unmanaged mode, following a series of parameters defined in\u00a0\/etc\/trickled.conf<\/strong>. This file defines how trickled (the daemon) behaves and manages trickle.<\/p>\n
In addition, if we want to set global settings to be used, overall, by all applications, we will need to use the trickled command. This command runs the daemon and allows us to define download and upload limits that will be shared by all the applications run through trickle without us needing to specify limits each time.<\/p>\n
For example, running:<\/p>\n
# trickled -d 50 -u 10\r\n<\/pre>\n
Will cause that the download and upload speeds of any application run through trickle be limited to\u00a030 KB\/s<\/strong>\u00a0and\u00a010 KB\/s<\/strong>, respectively.<\/p>\n
Please note that you can check at any time whether trickled is running and with what arguments:<\/p>\n
# ps -ef | grep trickled | grep -v grep\r\n<\/pre>\n
Output:<\/p>\n
root \t16475 \t1  0 Dec24 ?    \t00:00:04 trickled -d 50 -u 10<\/pre>\n
\n
\n
\n
\n
Example 3: Uploading a 19 MB mp4 file to our FTP server using with and without trickle.<\/h6>\n
In this example we will use the freely-distributable \u201cHe is the gift<\/strong>\u201d video, available for download from\u00a0this link<\/a>.<\/p>\n
We will initially download this file to your current working directory with the following command:<\/p>\n
# wget http:\/\/media2.ldscdn.org\/assets\/missionary\/our-people-2014\/2014-00-1460-he-is-the-gift-360p-eng.mp4 \r\n<\/pre>\n
First off, we will start the trickled daemon with the command listed above:<\/p>\n
# trickled -d 30 -u 10\r\n<\/pre>\n
Without trickle:<\/p>\n
# ncftpput -u username -p password 192.168.0.15 \/testdir 2014-00-1460-he-is-the-gift-360p-eng.mp4 \r\n<\/pre>\n
Output:<\/p>\n
2014-00-1460-he-is-the-gift-360p-eng.mp4:           \t18.53 MB   36.31 MB\/s\r\n<\/pre>\n
With trickle:<\/p>\n
# trickle ncftpput -u username -p password 192.168.0.15 \/testdir 2014-00-1460-he-is-the-gift-360p-eng.mp4 \r\n<\/pre>\n
Output:<\/p>\n
2014-00-1460-he-is-the-gift-360p-eng.mp4:           \t18.53 MB\t9.51 kB\/s\r\n<\/pre>\n
As we can see in the output above, the upload transfer rate dropped to\u00a0~10 KB\/s<\/strong>.<\/p>\n
Example 4: Downloading the same video with and without trickle<\/h6>\n
As in\u00a0Example 2<\/strong>, we will be downloading the file to the current working directory.<\/p>\n
Without trickle:<\/p>\n
# ncftpget -u username -p password 192.168.0.15 . \/testdir\/2014-00-1460-he-is-the-gift-360p-eng.mp4 \r\n<\/pre>\n
Output:<\/p>\n
2014-00-1460-he-is-the-gift-360p-eng.mp4:           \t18.53 MB  108.34 MB\/s\r\n<\/pre>\n
With trickle:<\/p>\n
# trickle ncftpget -u username -p password 111.111.111.111 . \/testdir\/2014-00-1460-he-is-the-gift-360p-eng.mp4 \r\n<\/pre>\n
Output:<\/p>\n
2014-00-1460-he-is-the-gift-360p-eng.mp4:           \t18.53 MB   29.28 kB\/s\r\n<\/pre>\n
Which is in accordance with the download limit set earlier (30 KB\/s<\/strong>).<\/p>\n
Note<\/strong>: That once the daemon has been started, there is no need to set individual limits for each application that uses trickle.<\/p>\n
As we mentioned earlier, one can further customize trickle\u2019s bandwidth shaping through trickled.conf. A typical section in this file consists of the following:<\/p>\n
[service]\r\nPriority = <value>\r\nTime-Smoothing = <value>\r\nLength-Smoothing = <value>\r\n<\/pre>\n
Where,<\/p>\n
    \n
  1. [service]<\/b>\u00a0indicates the name of the application whose bandwidth usage we intend to shape.<\/li>\n
  2. Priority<\/b>\u00a0allows us to specify a service to have a higher priority relative to another, thus not allowing a single application to hog all the bandwidth which the daemon is managing. The lower the number, the more bandwidth that is assigned to [service].<\/li>\n
  3. Time-Smoothing<\/b>\u00a0[in seconds]: defines with what time intervals trickled will try to let the application transfer and \/ or receive data. Smaller values (something between the range of 0.1 \u2013 1s) are ideal for interactive applications and will result in a more continuous (smooth) session while slightly larger values (1 \u2013 10 s) are better for applications that need bulk transfer. If no value is specified, the default (5 s) is used.<\/li>\n
  4. Length-Smoothing<\/b>\u00a0[in KB]: the idea is the same as in Time-Smoothing, but based on the length of an I\/O operation. If no value is specified, the default (10 KB) is used.<\/li>\n<\/ol>\n
    Changing the smoothing values will translate into the application specified by\u00a0[service]<\/strong>\u00a0using transfer rates within an interval instead of a fixed value. Unfortunately, there is no formula to calculate the lower and upper limits of this interval as it mainly depends of each specific case scenario.<\/p>\n
    The following is a\u00a0trickled.conf<\/strong>\u00a0sample file in the CentOS 7 client (192.168.0.17<\/strong>):<\/p>\n
    [ssh]\r\nPriority = 1\r\nTime-Smoothing = 0.1\r\nLength-Smoothing = 2\r\n\r\n[ftp]\r\nPriority = 2\r\nTime-Smoothing = 1\r\nLength-Smoothing = 3\r\n<\/pre>\n
    Using this setup, trickled will prioritize SSH connections over FTP transfers. Note that an interactive process, such as SSH, uses smaller time-smoothing values, whereas a service that performs bulk data transfers (FTP) uses a greater value. The smoothing values are responsible for the download and upload speeds in our previous example not matching the exact value specified by the trickled daemon but moving in an interval close to it.<\/p>\n
    Conclusion<\/h3>\n
    In this article we have explored how to limit the bandwidth used by applications using trickle on Fedora-based distributions and Debian \/ derivatives. Other possible use cases include, but are not limited to:<\/p>\n
      \n
    1. Limiting the download speed via a system utility such as\u00a0wget<\/a>, or a torrent client, for example.<\/li>\n
    2. Limiting the speed at which your system can be updated via `yum<\/a>` (or `aptitude<\/a>`, if you\u2019re in a Debian-based system), the package management system.<\/li>\n
    3. If your server happens to be behind a proxy or firewall (or is the proxy or firewall itself), you can use trickle to set limits on both the download and upload, or communication speed with the clients or the outside.<\/li>\n<\/ol>\n
      Questions and comments are most welcome. Feel free to use the form below to send them our way.<\/p>\n
      Source<\/a><\/p>\n<\/div>\n<\/div>\n<\/div>\n<\/article>\n","protected":false},"excerpt":{"rendered":"
      Have you ever encountered situations where one application dominated you all network bandwidth? If you have ever been in a situation where one application ate all your traffic, then you will value the role of the trickle bandwidth shaper application. Either you are a\u00a0system admin\u00a0or just a\u00a0Linux user, you need to learn how to control … <\/p>\n
      Continue reading “How to Limit the Network Bandwidth Used by Applications in a Linux System with Trickle”<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-13786","post","type-post","status-publish","format-standard","hentry","category-linux"],"_links":{"self":[{"href":"https:\/\/www.appservgrid.com\/paw92\/index.php\/wp-json\/wp\/v2\/posts\/13786","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.appservgrid.com\/paw92\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.appservgrid.com\/paw92\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.appservgrid.com\/paw92\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.appservgrid.com\/paw92\/index.php\/wp-json\/wp\/v2\/comments?post=13786"}],"version-history":[{"count":1,"href":"https:\/\/www.appservgrid.com\/paw92\/index.php\/wp-json\/wp\/v2\/posts\/13786\/revisions"}],"predecessor-version":[{"id":13787,"href":"https:\/\/www.appservgrid.com\/paw92\/index.php\/wp-json\/wp\/v2\/posts\/13786\/revisions\/13787"}],"wp:attachment":[{"href":"https:\/\/www.appservgrid.com\/paw92\/index.php\/wp-json\/wp\/v2\/media?parent=13786"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.appservgrid.com\/paw92\/index.php\/wp-json\/wp\/v2\/categories?post=13786"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.appservgrid.com\/paw92\/index.php\/wp-json\/wp\/v2\/tags?post=13786"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}