{"id":313,"date":"2018-10-16T09:06:09","date_gmt":"2018-10-16T09:06:09","guid":{"rendered":"https:\/\/www.appservgrid.com\/paw93\/?p=313"},"modified":"2018-10-16T21:11:19","modified_gmt":"2018-10-16T21:11:19","slug":"debian-build-environment-in-a-docker-container-own-your-bits","status":"publish","type":"post","link":"https:\/\/www.appservgrid.com\/paw93\/index.php\/2018\/10\/16\/debian-build-environment-in-a-docker-container-own-your-bits\/","title":{"rendered":"Debian build environment in a docker container \u2013 Own your bits"},"content":{"rendered":"<p><a href=\"https:\/\/ownyourbits.com\/2017\/06\/20\/c-build-environment-in-a-docker-container\/\">Last post<\/a>, I shared a docker container for compilation in C with <em>ccache<\/em> and <em>colorgcc<\/em> included. This time, we will extend that base container for development and packaging of Debian packages.<\/p>\n<p>Not only it is handy to have the environment configured and packaged, but also opens some oportunities for optimization given the nature of docker, its catching overlays and its volumes.<\/p>\n<p>Finally, it makes it easy to start developing Debian packages from another distribution, such as Arch Linux.<\/p>\n<h4>Features<\/h4>\n<ul>\n<li>GCC 6<\/li>\n<li>Debian package development tools: <em>lintian<\/em>, <em>quilt<\/em>, <em>debuild<\/em>, <em>dh-make<\/em>, <em>fakeroot<\/em> \u2026<\/li>\n<li><a href=\"http:\/\/savannah.nongnu.org\/projects\/quilt\"><em>quilt<\/em><\/a> configured for debian patching<\/li>\n<li><a href=\"https:\/\/ccache.samba.org\/\"><em>ccache<\/em><\/a> for fast recompilation. Included in debuild and dpkg-buildpackage calls<\/li>\n<li><a href=\"https:\/\/www.flamingspork.com\/projects\/libeatmydata\/\"><em>eatmydata<\/em><\/a> for faster compilation times<\/li>\n<li>Only 192 MB uncompressed extra layer, totalling 435 MB for the whole container<\/li>\n<\/ul>\n<p><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/ownyourbits.com\/wp-content\/uploads\/2017\/06\/debiandev.png\" alt=\"\" width=\"295\" height=\"1099\" \/><\/p>\n<p>If you are reading this post, you probably do not need an explanation about those tools. Look at the references section otherwise.<\/p>\n<p>If you are wondering how this compares to <em>sbuild<\/em> and <em>pbuilder<\/em>, this approach is really very similar. The idea is the same: have another clean and isolated environment where compilation takes place. This solves several problems:<\/p>\n<ul>\n<li>You can build for a different version of Debian, such as <em>un<\/em><em>stable <\/em> or <em>testing<\/em>, without messing up your system with packages from those.<\/li>\n<li>You can be sure that the dependencies are right, as the environment is minimal.<\/li>\n<\/ul>\n<p>Well, docker containers can be used as a <em>chroot<\/em> in steroids, and can be regarded as an evolution of the concept using modern kernel features such as cgroups and namespaces.<\/p>\n<p>Another nice benefit: it is very simple to manage docker containers. You can pull them, push them, export them and save them.<\/p>\n<p>Last, a huge benefit at least for me personally is to be able to work from another Linux distribution, such as Arch.<\/p>\n<h4>Usage<\/h4>\n<p>Log into the development environment<\/p>\n<p>docker run &#8211;rm -v &#8220;\/workdir\/path:\/src&#8221; -ti ownyourbits\/debiandev<\/p>\n<p>We can now use the standard tools, the working directory ( <em>\/workdir\/path<\/em> in this example ) is an external folder accessible from the container, where you can do<br \/>\napt-get source and retrieve the <em>.deb<\/em> files.<\/p>\n<h6>Example: cross-compile QEMU for ARM64<\/h6>\n<p>In my experience, not all packages are configured well enough to support cross-compilation. Specially big packages tend to fail when it comes to the<br \/>\nbuild-dep step. I found this nice exception in <a href=\"http:\/\/suihkulokki.blogspot.com.es\/2017\/06\/cross-compiling-with-debian-stretch.html\">this post<\/a>.<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<table>\n<tbody>\n<tr>\n<td><\/td>\n<td>sudo dpkg &#8211;add-architecture arm64<\/p>\n<p>sudo apt-get update<\/p>\n<p>sudo apt-get build-dep -aarm64 qemu<\/p>\n<p>apt-get source qemu<\/p>\n<p>cd qemu-*<\/p>\n<p>dpkg-buildpackage -aarm64 -b<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<p>&nbsp;<\/p>\n<h6>Example: package and tweak PHP, with CCACHE cache already populated<\/h6>\n<p>I like to use this container as a base for each specific project. This way, I can take advantage of the catching layers of docker to speed up the process, and at the same time I end up with the building instructions compiled in the Dockerfile.<\/p>\n<p>If you decide to use a docker volume, you can always remove it if you want to start from zero. This has the benefit that upon running the container, <em>\/src<\/em> will be populated with the results and cache from the Dockerfile step again. A real time saver!<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<table>\n<tbody>\n<tr>\n<td>&nbsp;<\/p>\n<p>1<\/p>\n<p>2<\/p>\n<p>3<\/p>\n<p>4<\/p>\n<p>5<\/p>\n<p>6<\/p>\n<p>7<\/p>\n<p>8<\/p>\n<p>9<\/p>\n<p>10<\/p>\n<p>11<\/p>\n<p>12<\/p>\n<p>13<\/p>\n<p>14<\/p>\n<p>15<\/p>\n<p>16<\/p>\n<p>17<\/p>\n<p>18<\/p>\n<p>19<\/p>\n<p>20<\/p>\n<p>21<\/p>\n<p>22<\/p>\n<p>23<\/p>\n<p>24<\/p>\n<p>25<\/p>\n<p>26<\/p>\n<p>27<\/p>\n<p>28<\/p>\n<p>29<\/p>\n<p>30<\/p>\n<p>31<\/p>\n<p>32<\/p>\n<p>33<\/p>\n<p>34<\/p>\n<p>35<\/p>\n<p>36<\/p>\n<p>37<\/p>\n<p>38<\/p>\n<p>39<\/p>\n<p>40<\/p>\n<p>41<\/p>\n<p>42<\/p>\n<p>43<\/p>\n<p>44<\/p>\n<p>45<\/p>\n<p>46<\/p>\n<p>47<\/p>\n<p>48<\/p>\n<p>49<\/p>\n<p>50<\/p>\n<p>51<\/p>\n<p>52<\/p>\n<p>53<\/p>\n<p>54<\/p>\n<p>55<\/p>\n<p>56<\/p>\n<p>57<\/p>\n<p>58<\/p>\n<p>59<\/p>\n<p>60<\/p>\n<p>61<\/p>\n<p>62<\/p>\n<p>63<\/p>\n<p>64<\/p>\n<p>65<\/p>\n<p>66<\/p>\n<p>67<\/p>\n<p>&nbsp;<\/td>\n<td># PHP Debian build environment with GCC 6 and ccache, and all debian dev tools<\/p>\n<p>#<\/p>\n<p># Copyleft 2017 by Ignacio Nunez Hernanz &lt;nacho _a_t_ ownyourbits _d_o_t_ com&gt;<\/p>\n<p># GPL licensed (see end of file) * Use at your own risk!<\/p>\n<p>#<\/p>\n<p># Usage:<\/p>\n<p>#<\/p>\n<p># docker run &#8211;rm -v &#8220;src:\/src&#8221; -ti ownyourbits\/phpdev<\/p>\n<p>#<\/p>\n<p># Then, inside:<\/p>\n<p># cd php7.0-7.0.19<\/p>\n<p># debuild -us -uc -b<\/p>\n<p>#<\/p>\n<p># Note that with this invocation command, the code resides in a persistent volume called &#8216;src&#8217;.<\/p>\n<p># See &#8216;docker volume ls&#8217;<\/p>\n<p>#<\/p>\n<p># It has already been build once with CCACHE, so you can just start tweaking, and recompilation will<\/p>\n<p># be very fast. If you do &#8216;docker volume rm src&#8217;, then next time you run the container it will be<\/p>\n<p># populated again with the fresh build ( but you would lose your code changes ).<\/p>\n<p>#<\/p>\n<p># A second option is to do ` -v &#8220;\/path:\/src&#8221; and use &#8220;\/path&#8221; from your system, but then you have to<\/p>\n<p># do &#8216;apt-get source&#8217; and &#8216;debuild&#8217; yourself, because &#8220;\/path&#8221; will be originally empty.<\/p>\n<p>#<\/p>\n<p># Details at ownyourbits.com<\/p>\n<p>FROM ownyourbits\/debiandev:latest<\/p>\n<p>LABEL description=&#8221;PHP build environment&#8221;<\/p>\n<p>MAINTAINER Ignacio N\u00fa\u00f1ez Hernanz &lt;nacho@ownyourbits.com&gt;<\/p>\n<p>## Get source<\/p>\n<p>RUN sudo apt-get update;<\/p>\n<p>mkdir -p \/src; cd \/src;<\/p>\n<p>apt-get source -t stretch php7.0-fpm;<\/p>\n<p>## PHP build dependencies<\/p>\n<p>RUN sudo apt-get update;<\/p>\n<p>DEBIAN_FRONTEND=noninteractive sudo apt-get build-dep -y -t stretch php7.0-fpm;<\/p>\n<p>sudo apt-get autoremove -y; sudo apt-get clean; sudo rm \/var\/lib\/apt\/lists\/*;<\/p>\n<p>sudo rm \/var\/log\/alternatives.log \/var\/log\/apt\/* ; sudo rm \/var\/log\/* -r; sudo rm -rf \/usr\/share\/man\/*;<\/p>\n<p>## Build first<\/p>\n<p># this will build the package without testing but with the CCACHE options, so we are<\/p>\n<p># building and catching compilation artifacts<\/p>\n<p>RUN cd $( find \/src -maxdepth 1 -type d | grep php );<\/p>\n<p>CCACHE_DIR=\/src\/.ccache DEB_BUILD_OPTIONS=nocheck<\/p>\n<p>eatmydata debuild<\/p>\n<p>&#8211;prepend-path=\/usr\/lib\/ccache &#8211;preserve-envvar=CCACHE_* &#8211;no-lintian -us -uc;<\/p>\n<p># License<\/p>\n<p>#<\/p>\n<p># This script is free software; you can redistribute it and\/or modify it<\/p>\n<p># under the terms of the GNU General Public License as published by<\/p>\n<p># the Free Software Foundation; either version 2 of the License, or<\/p>\n<p># (at your option) any later version.<\/p>\n<p>#<\/p>\n<p># This script is distributed in the hope that it will be useful,<\/p>\n<p># but WITHOUT ANY WARRANTY; without even the implied warranty of<\/p>\n<p># MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the<\/p>\n<p># GNU General Public License for more details.<\/p>\n<p>#<\/p>\n<p># You should have received a copy of the GNU General Public License<\/p>\n<p># along with this script; if not, write to the<\/p>\n<p># Free Software Foundation, Inc., 59 Temple Place, Suite 330,<\/p>\n<p># Boston, MA 02111-1307 USA<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<p>&nbsp;<\/p>\n<h4>Code<\/h4>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<table>\n<tbody>\n<tr>\n<td>&nbsp;<\/p>\n<p>1<\/p>\n<p>2<\/p>\n<p>3<\/p>\n<p>4<\/p>\n<p>5<\/p>\n<p>6<\/p>\n<p>7<\/p>\n<p>8<\/p>\n<p>9<\/p>\n<p>10<\/p>\n<p>11<\/p>\n<p>12<\/p>\n<p>13<\/p>\n<p>14<\/p>\n<p>15<\/p>\n<p>16<\/p>\n<p>17<\/p>\n<p>18<\/p>\n<p>19<\/p>\n<p>20<\/p>\n<p>21<\/p>\n<p>22<\/p>\n<p>23<\/p>\n<p>24<\/p>\n<p>25<\/p>\n<p>26<\/p>\n<p>27<\/p>\n<p>28<\/p>\n<p>29<\/p>\n<p>30<\/p>\n<p>31<\/p>\n<p>32<\/p>\n<p>33<\/p>\n<p>34<\/p>\n<p>35<\/p>\n<p>36<\/p>\n<p>37<\/p>\n<p>38<\/p>\n<p>39<\/p>\n<p>40<\/p>\n<p>41<\/p>\n<p>42<\/p>\n<p>43<\/p>\n<p>44<\/p>\n<p>45<\/p>\n<p>46<\/p>\n<p>47<\/p>\n<p>48<\/p>\n<p>49<\/p>\n<p>50<\/p>\n<p>51<\/p>\n<p>52<\/p>\n<p>53<\/p>\n<p>54<\/p>\n<p>55<\/p>\n<p>56<\/p>\n<p>&nbsp;<\/td>\n<td># Debian build environment with GCC 6, ccache and all debian dev tools<\/p>\n<p>#<\/p>\n<p># Copyleft 2017 by Ignacio Nunez Hernanz &lt;nacho _a_t_ ownyourbits _d_o_t_ com&gt;<\/p>\n<p># GPL licensed (see end of file) * Use at your own risk!<\/p>\n<p>#<\/p>\n<p># Usage:<\/p>\n<p>#<\/p>\n<p># docker run &#8211;rm -v &#8220;\/workdir\/path:\/src&#8221; -ti ownyourbits\/debiandev<\/p>\n<p>#<\/p>\n<p># Details at https:\/\/ownyourbits.com\/2017\/06\/24\/debian-build-environment-in-a-docker-container\/<\/p>\n<p>FROM ownyourbits\/mmake:latest<\/p>\n<p>LABEL description=&#8221;Debian package development environment&#8221;<\/p>\n<p>MAINTAINER Ignacio N\u00fa\u00f1ez Hernanz &lt;nacho@ownyourbits.com&gt;<\/p>\n<p># install packages<\/p>\n<p>RUN sudo sh -c &#8220;echo deb-src http:\/\/httpredir.debian.org\/debian stretch main &gt;&gt; \/etc\/apt\/sources.list&#8221;;<\/p>\n<p>sudo apt-get update;<\/p>\n<p>DEBIAN_FRONTEND=noninteractive sudo apt-get install &#8211;no-install-recommends -y dpkg-dev devscripts dh-make lintian fakeroot quilt eatmydata vim;<\/p>\n<p>sudo apt-get autoremove -y; sudo apt-get clean; sudo rm \/var\/lib\/apt\/lists\/*;<\/p>\n<p>sudo rm \/var\/log\/alternatives.log \/var\/log\/apt\/*; sudo rm \/var\/log\/* -r;<\/p>\n<p># configure session<\/p>\n<p>RUN echo &#8220;alias debuild=&#8217;eatmydata debuild &#8211;prepend-path=\/usr\/lib\/ccache &#8211;preserve-envvar=CCACHE_*'&#8221; &gt;&gt; \/home\/builder\/.bashrc;<\/p>\n<p>echo &#8220;alias dpkg-buildpackage=&#8217;eatmydata dpkg-buildpackage'&#8221; &gt;&gt; \/home\/builder\/.bashrc;<\/p>\n<p># NOTE: dpkg-buildpackage and debuild do not play well with colorgcc<\/p>\n<p>echo &#8216;export PATH=&#8221;\/usr\/lib\/ccache\/:$PATH&#8221;&#8216;;<\/p>\n<p>sudo rm \/usr\/lib\/colorgcc\/*<\/p>\n<p>COPY _quiltrc \/home\/builder\/.quiltrc<\/p>\n<p># prepare work dir<\/p>\n<p>RUN sudo mkdir -p \/src; sudo chown builder:builder \/src; echo &#8216;cd \/src&#8217; &gt;&gt; \/home\/builder\/.bashrc<\/p>\n<p># remove previous entrypoint<\/p>\n<p>ENTRYPOINT []<\/p>\n<p>CMD [&#8220;\/bin\/bash&#8221;]<\/p>\n<p># License<\/p>\n<p>#<\/p>\n<p># This script is free software; you can redistribute it and\/or modify it<\/p>\n<p># under the terms of the GNU General Public License as published by<\/p>\n<p># the Free Software Foundation; either version 2 of the License, or<\/p>\n<p># (at your option) any later version.<\/p>\n<p>#<\/p>\n<p># This script is distributed in the hope that it will be useful,<\/p>\n<p># but WITHOUT ANY WARRANTY; without even the implied warranty of<\/p>\n<p># MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the<\/p>\n<p># GNU General Public License for more details.<\/p>\n<p>#<\/p>\n<p># You should have received a copy of the GNU General Public License<\/p>\n<p># along with this script; if not, write to the<\/p>\n<p># Free Software Foundation, Inc., 59 Temple Place, Suite 330,<\/p>\n<p># Boston, MA 02111-1307 USA<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<p>&nbsp;<\/p>\n<p><a href=\"https:\/\/github.com\/nachoparker\/debiandev-docker\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/ownyourbits.com\/wp-content\/uploads\/2018\/03\/GitHub-Mark-32px.png\" alt=\"\" width=\"32\" height=\"32\" \/><\/a><\/p>\n<h4>References<\/h4>\n<p><a href=\"https:\/\/www.debian.org\/doc\/manuals\/maint-guide\/build.en.html\">https:\/\/www.debian.org\/doc\/manuals\/maint-guide\/build.en.html<\/a><\/p>\n<p><a href=\"https:\/\/www.debian.org\/doc\/debian-policy\/ch-source.html\">https:\/\/www.debian.org\/doc\/debian-policy\/ch-source.html<\/a><\/p>\n<p><a href=\"https:\/\/wiki.debian.org\/BuildingTutorial\">https:\/\/wiki.debian.org\/BuildingTutorial<\/a><\/p>\n<p><a href=\"https:\/\/wiki.debian.org\/CrossCompiling\">https:\/\/wiki.debian.org\/CrossCompiling<\/a><\/p>\n<p><a href=\"https:\/\/wiki.debian.org\/Multiarch\/HOWTO\">https:\/\/wiki.debian.org\/Multiarch\/HOWTO<\/a><\/p>\n<p><a href=\"https:\/\/ownyourbits.com\/2017\/06\/24\/debian-build-environment-in-a-docker-container\/\" target=\"_blank\" rel=\"noopener\">Source<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Last post, I shared a docker container for compilation in C with ccache and colorgcc included. This time, we will extend that base container for development and packaging of Debian packages. Not only it is handy to have the environment configured and packaged, but also opens some oportunities for optimization given the nature of docker, &hellip; <\/p>\n<p class=\"link-more\"><a href=\"https:\/\/www.appservgrid.com\/paw93\/index.php\/2018\/10\/16\/debian-build-environment-in-a-docker-container-own-your-bits\/\" class=\"more-link\">Continue reading<span class=\"screen-reader-text\"> &#8220;Debian build environment in a docker container \u2013 Own your bits&#8221;<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[2],"tags":[],"class_list":["post-313","post","type-post","status-publish","format-standard","hentry","category-docker"],"_links":{"self":[{"href":"https:\/\/www.appservgrid.com\/paw93\/index.php\/wp-json\/wp\/v2\/posts\/313","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.appservgrid.com\/paw93\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.appservgrid.com\/paw93\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.appservgrid.com\/paw93\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.appservgrid.com\/paw93\/index.php\/wp-json\/wp\/v2\/comments?post=313"}],"version-history":[{"count":1,"href":"https:\/\/www.appservgrid.com\/paw93\/index.php\/wp-json\/wp\/v2\/posts\/313\/revisions"}],"predecessor-version":[{"id":423,"href":"https:\/\/www.appservgrid.com\/paw93\/index.php\/wp-json\/wp\/v2\/posts\/313\/revisions\/423"}],"wp:attachment":[{"href":"https:\/\/www.appservgrid.com\/paw93\/index.php\/wp-json\/wp\/v2\/media?parent=313"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.appservgrid.com\/paw93\/index.php\/wp-json\/wp\/v2\/categories?post=313"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.appservgrid.com\/paw93\/index.php\/wp-json\/wp\/v2\/tags?post=313"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}