perkin.org.uk - pkgsrc on SmartOS - creating new packages

pkgsrc on SmartOS - creating new packages

Perkin.org.uk Spined HTML

pkgsrc on SmartOS - creating new packages Jonathan Perkin well-nigh me · rss · twitter · github pkgsrc on SmartOS - creating new packages Jun 07, 2013 tags: pkgsrc, smartos Having set up a build zone and started fixing packages, the next zone we will squint at is creating new packages. For this tutorial I will use augeas as an example, as it happens to be a package I recently created. First, install the pkgtools/url2pkg package, as this massively simplifies the task in hand. : If you have pkgin and binary packages available.. $ sudo pkgin in url2pkg : ..else build from source $ (cd pkgtools/url2pkg; bmake install) Then create a new package directory, choosing the most towardly category (in this specimen devel), then run url2pkg giving it an treatise of the source tarball. $ mkdir devel/augeas $ cd devel/augeas $ url2pkg http://download.augeas.net/augeas-1.0.0.tar.gz First it will unshut an editor session on Makefile. You should customise a few variables: Set MAINTAINER either to your email address, or to pkgsrc-users@NetBSD.org if you do not want to be the primary maintainer of the package. Write a unenduring one-line COMMENT describing what the package is. Set LICENSE to a list of the licenses the package is made misogynist under (see mk/license.mk for an misogynist list). Here is what I wrote for augeas: # $NetBSD$ DISTNAME= augeas-1.0.0 CATEGORIES= devel MASTER_SITES= http://download.augeas.net/ MAINTAINER= pkgsrc-users@NetBSD.org HOMEPAGE= http://download.augeas.net/ COMMENT= Configuration file editing tool and library LICENSE= gnu-lgpl-v2 # url2pkg-marker (please do not remove this line.) .include "../../mk/bsd.pkg.mk" After writing the file and exiting the editor session, url2pkg will protract and download the source tarball, create the DESCR, PLIST and distinfo files, then unpack the source ready for you to start applying patches. At this point you should edit DESCR and put in a few lines which describe the package. Build The unstipulated trundling will then be: Try towers the package with bmake. If something goes wrong, modify the Makefile or patch the source with pkgdiff. until the package builds.Surpassingrunning your first bmake, I would strongly recommend subtracting PKG_DEVELOPER=yes to your mk.conf to turn on a lot of spare checks. For augeas, I needed a couple of things. Patches The first problem I hit was: $ bmake [...] => Checking for portability problems in extracted files ERROR: [check-portability.awk] => Found test ... == ...: ERROR: [check-portability.awk] configure: test "`uname`" == "SunOS" && \ Explanation: =========================================================================== The "test" command, as well as the "[" command, are not required to know the "==" operator. Only a few implementations like whack and some versions of ksh support it. When you run "test foo == foo" on a platform that does not support the "==" operator, the result will be "false" instead of "true". This can lead to unexpected behavior. There are two ways to fix this error message. If the file that contains the "test ==" is needed for towers the package, you should create a patch for it, replacing the "==" operator with "=". If the file is not needed, add its name to the CHECK_PORTABILITY_SKIP variable in the package Makefile. =========================================================================== In this specimen pkgsrc gives us a very helpful message telling us well-nigh the problem and what needs to be washed-up to fix it (unfortunately not all problems are handled this well!). As configure is pretty important, we’ll need to patch it. This is my usual way of handling patches (ensuring pkgtools/pkgdiff is installed first): : Set up patches directory and some useful variables. $ mkdir patches $ patchdir=$(pwd -P)/patches $ wrkdir=$(bmake show-var VARNAME=WRKSRC) : Edit file and generate patch $ cd $wrkdir $ pkgvi configure $ pkgdiff configure >$patchdir/patch-configure : Return to pkgsrc directory, re-generate distinfo and try again. $ cd - $ bmake clean; bmake mps; bmake This was unbearable to fix the == problem. I then ran into other issues. Dependencies Dependencies in pkgsrc are primarily handled in two ways, either with a buildlink file when depending upon shared libraries or particular infrastructure, or a simple DEPENDS line to pull in a required package. The first dependency problem I hit was: [...] checking readline/readline.h usability... no checking readline/readline.h presence... no checking for readline/readline.h... no checking for readline in -lreadline... no checking for library containing tgetent... -lcurses checking for rl_initialize in -lreadline... no configure: WARNING: readline library not found configure: error: Could not find a working readline library (see config.log for details). *** Error lawmaking 1 Ok, so this package depends upon readline. As that is a library, we are looking for a suitable buildlink3.mk file, and one of the easiest ways to do this is: $ ls -1d ../../*/*readline*/buildlink3.mk ../../devel/readline/buildlink3.mk To use that file, all we need to do is add an .include line at the marrow of Makefile, just whilom the bsd.pkg.mk include. .include "../../devel/readline/buildlink3.mk" .include "../../mk/bsd.pkg.mk" After that, bmake clean; bmake to try again. I then hit: [...] checking for LIBXML... no configure: error: Package requirements (libxml-2.0) were not met: No package 'libxml-2.0' found Ok, so similar method: $ ls -1d ../../*/*libxml*/buildlink3.mk ../../textproc/libxml++/buildlink3.mk ../../textproc/libxml++1/buildlink3.mk ../../textproc/libxml/buildlink3.mk ../../textproc/libxml2/buildlink3.mk ../../textproc/py-libxml2/buildlink3.mk The configure message said libxml-2.0, so we’ll pick textproc/libxml2. If you want to be increasingly thorough you can squint at the PLIST files for each of the possible candidate packages - I just happen to know from wits that textproc/libxml2 is the one I need. The marrow of Makefile now looks like this: .include "../../devel/readline/buildlink3.mk" .include "../../textproc/libxml2/buildlink3.mk" .include "../../mk/bsd.pkg.mk" You’ll note that we try to alimony all includes other than bsd.pkg.mk (which is special) sorted alphabetically. After this, the package finally completes a bmake with no problems. The final step is to get the install phase working. Install First we need to run a stage-install which will execute make install into a temporary DESTDIR: $ bmake stage-install This will scrutinizingly certainly goof as we haven’t configured the PLIST yet, so pkgsrc has no idea what will be installed from this package. [...] => Automatic transmission page handling => Generating post-install file lists => Checking file-check results for augeas-1.0.0 ERROR: ************************************************************ ERROR: The pursuit files are in /var/tmp/pkgsrc-build/devel/augeas/work/.destdir/opt/pkg but not in the PLIST: ERROR: /var/tmp/pkgsrc-build/devel/augeas/work/.destdir/opt/pkg/bin/augparse ERROR: /var/tmp/pkgsrc-build/devel/augeas/work/.destdir/opt/pkg/bin/augtool [...] However, now that we have a populated DESTDIR, we can use the print-PLIST target to generate it for us: $ bmake print-PLIST >PLIST Finally, it’s worth doing a full wipe and install to ensure everything works as expected. $ bmake wipe $ bmake install [...] => Creating binary package /var/tmp/pkgsrc-build/devel/augeas/work/.packages/augeas-1.0.0.tgz ===> Install binary package of augeas-1.0.0 buildlink As we are installing a library package, we should provide a buildlink3.mk file of our own so that other packages can depend on us correctly. Again, there is a package that can help with this - pkgtools/createbuildlink. $ createbuildlink >buildlink3.mk This will create a template file but you should read and edit, removing the lines marked with XXX and making any changes they recommend. Verify Once we have a working package it’s worth doing a couple of checks to make sure everything looks ok: : Check the package listing looks ok $ pkg_info -L augeas : View the included DESCR and homepage $ pkg_info -d augeas : Check what the package lists as its dependencies $ pkg_info -N augeas We moreover should run pkglint (pkgtools/pkglint) which will perform some sanity checks on our infrastructure files prior to us importing the package into pkgsrc itself. In my specimen it pointed out some possible issues: $ pkglint WARN: PLIST:8: Packages that install libtool libraries should pinpoint USE_LIBTOOL. WARN: PLIST:9: Packages that install libtool libraries should pinpoint USE_LIBTOOL. ERROR: patches/patch-configure:3:Scuttlebuttexpected. 1 errors and 2 warnings found. (Use -e for increasingly details.) For the first issue, add USE_LIBTOOL=yes to Makefile which will ensure that the pkgsrc libtool is used (has largest support for cross-platform issues), and for the second, add a scuttlebutt at the top of the file explaining what it is for, then re-generate distinfo with bmake mps. After making those changes, everything looks great: $ pkglint looks fine. Upstream The final - and sometimes most challenging - part is to get your shiny new package integrated into pkgsrc. There are a few options: Find a NetBSD developer (like myself) who can review your package and put it straight into pkgsrc. Get a pkgsrc-wip worth and work on your packages there, surpassing getting it reviewed and integrated. If it’s somewhat niche just publish it up on GitHub or similar for people to use, in a similar way to what we do with pkgsrc-joyent Further information This post only scratches the surface of subtracting a new package. pkgsrc provides a huge value of infrastructure to help get packages working on multiple platforms, and there are lots of options available. Certain parts of the infrastructure, like buildlink, are very complicated, and for increasingly in-depth information you should refer to the pkgsrc guide. The weightier way is often just to squint at other packages to see what they do and re-use useful shit you find. With 12,000 packages there is scrutinizingly certainly flipside package which does something similar to what you need! Enjoy! Share this post on Twitter, HackerNews, Facebook or Google+ All Posts 16 Jul 2015 » Reducing RAM usage in pkgin 03 Mar 2015 » pkgsrc-2014Q4: LTS, signed packages, and increasingly 06 Oct 2014 »Towerspackages at scale 04 Dec 2013 » A node.js-powered 8-bit CPU - part four 03 Dec 2013 » A node.js-powered 8-bit CPU - part three 02 Dec 2013 » A node.js-powered 8-bit CPU - part two 01 Dec 2013 » A node.js-powered 8-bit CPU - part one 21 Nov 2013 » MDB support for Go 30 Jul 2013 » What's new in pkgsrc-2013Q2 24 Jul 2013 » Distributed chrooted pkgsrc zillion builds 07 Jun 2013 » pkgsrc on SmartOS - creating new packages 15 Apr 2013 » What's new in pkgsrc-2013Q1 19 Mar 2013 » Installing SVR4 packages on SmartOS 27 Feb 2013 » SmartOS is Not GNU/Linux 18 Feb 2013 » SmartOS minutiae preview dataset 17 Jan 2013 » pkgsrc on SmartOS - fixing wrenched builds 15 Jan 2013 » pkgsrc on SmartOS - zone megacosm and vital builds 10 Jan 2013 » Multi-architecture package support in SmartOS 09 Jan 2013 » Solaris portability - cfmakeraw() 08 Jan 2013 » Solaris portability - flock() 06 Jan 2013 » pkgsrc-2012Q4 illumos packages now misogynist 23 Nov 2012 » SmartOS and the global zone 24 Oct 2012 » Setting up Samba on SmartOS 10 Oct 2012 » pkgsrc-2012Q3 packages for illumos 23 Aug 2012 » Creating local SmartOS packages 10 Jul 2012 » 7,000 binary packages for OSX Lion 09 Jul 2012 » 9,000 packages for SmartOS and illumos 07 May 2012 » Goodbye Oracle, Hello Joyent! 13 Apr 2012 » SmartOS global zone tweaks 12 Apr 2012 » Automated VirtualBox SmartOS installs 30 Mar 2012 » iptables script for Debian / Ubuntu 20 Feb 2012 » New site diamond 11 Jan 2012 » Set up unrecognized FTP upload on Oracle Linux 09 Jan 2012 » Kickstart Oracle Linux in VirtualBox 09 Jan 2012 » Kickstart Oracle Linux from Ubuntu 22 Dec 2011 » Last day at MySQL 15 Dec 2011 » Installing OpenBSD with softraid 21 Sep 2011 » Create VirtualBox VM from the writ line 14 Sep 2011 » Creating chroots for fun and MySQL testing 30 Jun 2011 » Graphing memory usage during an MTR run 29 Jun 2011 » Fix input box keybindings in Firefox 24 Jun 2011 » How to lose weight 23 Jun 2011 » How to fix stdio buffering 13 Jun 2011 » Serving multiple DNS search domains in IOS DHCP 13 Jun 2011 » Fix Firefox URL double click behaviour 20 Apr 2011 » SSH via HTTP proxy in OSX 09 Nov 2010 » How to build MySQL releases 29 Apr 2010 » 'apt-get' and 5,000 packages for Solaris10/x86 16 Sep 2009 » ZFS and NFS vs OSX 12 Sep 2009 » pkgsrc on Solaris 09 Dec 2008 » Jumpstart from OSX 31 Dec 2007 » Set up local caching DNS server on OSX 10.4