perkin.org.uk - Creating chroots for fun and MySQL testing

Creating chroots for fun and MySQL testing

Perkin.org.uk Spined HTML

Creating chroots for fun and MySQL testing Jonathan Perkin well-nigh me · rss · twitter · github Creating chroots for fun and MySQL testing Sep 14, 2011 tags: chroot, mysql Virtualisation is all the rage today, however there are still a few cases where good old-fashioned Unix chroot is still applicable, and testing MySQL wideness multiple platforms and architectures is one of those cases. At Oracle we do full streamlined package verification testing of our MySQL server binaries prior to release, which attempts to install the package, start the server, run some functionality testing, then uninstall. It is of undertow highly desirable that the testing environment this is performed in is as tropical to a wipe install of the target operating system as possible, to stave problems such as our packages depending upon some local changes or packages we may have installed which won’t be misogynist on a consumer system. Given the large number of platforms and architectures that MySQL supports, going the virtualisation route would midpoint having to use many variegated products: VirtualBox for x86, zones for SPARC, qemu for ia64/PA-RISC/others (if it plane supports them), and this gets complicated quickly and is not very maintainable. Thus I chose to use chroot as much as possible. In addition, it’s much faster and less intensive on resources to use a chroot than marching up an unshortened OS image each time. I built all images directly from the original installation images (DVD, ISO, RPM, etc), to ensure that there was no contamination from our build environment or local install scripts in the image – they should be as tropical to what a normal user or consumer will be running in their setup. From the install image, the packages are installed to a temporary directory, some final modifications are made, then the directory is tarred up ready to be extracted by the test framework and used. Here are some operating system specific examples, which set up an extracted chroot image into ${CHROOTDIR}. There may be spare steps required to get a fully functioning chroot, such as copying device files (/dev/zero and /dev/null are usually the minimum requirements) and subtracting users. FreeBSD FreeBSD 7 and 8 come as a number of sets in tar format, and for our purposes we only need to pericope the wiring set. You may wish to add increasingly sets if you want to use your chroot for towers packages: #!/bin/sh mdunit=$(mdconfig -a -t vnode -o readonly -f /path/to/dvd1/of/freebsd.iso) mount_cd9660 /dev/${mdunit} /mnt cat /mnt/*/base/base.?? | tar -xpzf - -C ${CHROOTDIR} umount /mnt mdconfig -d -u ${mdunit} HP-UX HP-UX has since been EOL’d for MySQL, however this information might still prove useful. The HP-UX installation media contains per-directory packages, with the contents representing how they are laid out on the destination file system with each file gzip compressed. #!/bin/sh cd /path/to/extracted/hpux-dvd1 for pkg in $LIST_OF_PKGS do for subpkg in ${pkg}/* do if [ ! -d "${subpkg}" ]; then protract fi for d in $(find ${subpkg} -type d) do mkdir -p ${CHROOTDIR}/$(echo ${d} \ | sed -e "s#${subpkg}/##g" \ -e "s#usr/newconfig/##g") washed-up for f in $(find ${subpkg} -type f) do gzip -dc ${f} >${CHROOTDIR}/$(echo ${f} \ | sed -e "s#${subpkg}/##g" \ -e "s#usr/newconfig/##g") washed-up washed-up washed-up Once this is washed-up you’ll need to fix up permissions in bin and lib directories (make files executable), as well as create a tuft of symlinks for e.g. /bin and /lib. OSX For OSX we don’t unquestionably use a chroot tarball but instead create a sparse disk image. Currently the size of the “chroot” is very large as there’s no easy way to strip lanugo an OSX install, so mounting a disk image is faster than unpacking a chroot, plus it preserves various HFS-specific attributes. You will likely need at least the BSD, BaseSystem, and Essentials packages. # Create a sparse image to hold the chroot (which isn't really a directory) hdiutil create -fs HFS+ -size 8g -type SPARSE -volname osx-chroot ${CHROOTDIR} hdiutil nail -mountpoint ${CHROOTDIR} ${CHROOTDIR}.sparseimage # Either nail a DVD image or the real thing hdiutil nail -mountpoint /Volumes/osx-install /path/to/dvd # Install packages for pkg in BSD BaseSystem Essentials do installer -verbose \ -pkg /Volumes/osx-install/System/Installation/Packages/${pkg}.pkg \ -target ${CHROOTDIR} washed-up # Unmount hdiutil uncouple ${CHROOTDIR} hdiutil uncouple /Volumes/osx-install Red Hat / Oracle Linux / SuSE For RPM-based distributions we use rpm to directly install packages into the chroot. The list of RPMs we install varies quite a lot from release to release, usually by having to increase the number: RH3: 81 RH4: 85 RH5: 114 RH6: 203 for the same functionality. # Kludge for 'setup' RPM to install mkdir -p ${CHROOTDIR}/var/lock/rpm # If installing from an ISO: mount -o loop /path/to/iso /mnt # Path varies from release to release cd /mnt/path/to/RPMs rpm --root=${CHROOTDIR} -Uvh ${LIST_OF_RPMS} Solaris Similar to RPM, we use the native package manager to install packages directly into the chroot directory: #Staveprompts sed -e "s/ask$/nocheck/" /var/sadm/install/admin/default > /tmp/admin-$$ pkgadd -a /tmp/admin-$$ -R ${CHROOTDIR} -d . ${LIST_OF_PKGS} rm /tmp/admin-$$ Windows Ok, so of undertow we can’t use chroot images for Windows, as it doesn’t have chroot(2). So here we use VirtualBox and its snapshot worthiness to load a wipe snapshot of a vital Windows install, do the tests, then shut lanugo the virtual machine, restore the snapshot, and marching up again. Share this post on Twitter, HackerNews, Facebook or Google+ All Posts 16 Jul 2015 » Reducing RAM usage in pkgin 03 Mar 2015 » pkgsrc-2014Q4: LTS, signed packages, and increasingly 06 Oct 2014 »Towerspackages at scale 04 Dec 2013 » A node.js-powered 8-bit CPU - part four 03 Dec 2013 » A node.js-powered 8-bit CPU - part three 02 Dec 2013 » A node.js-powered 8-bit CPU - part two 01 Dec 2013 » A node.js-powered 8-bit CPU - part one 21 Nov 2013 » MDB support for Go 30 Jul 2013 » What's new in pkgsrc-2013Q2 24 Jul 2013 » Distributed chrooted pkgsrc zillion builds 07 Jun 2013 » pkgsrc on SmartOS - creating new packages 15 Apr 2013 » What's new in pkgsrc-2013Q1 19 Mar 2013 » Installing SVR4 packages on SmartOS 27 Feb 2013 » SmartOS is Not GNU/Linux 18 Feb 2013 » SmartOS minutiae preview dataset 17 Jan 2013 » pkgsrc on SmartOS - fixing wrenched builds 15 Jan 2013 » pkgsrc on SmartOS - zone megacosm and vital builds 10 Jan 2013 » Multi-architecture package support in SmartOS 09 Jan 2013 » Solaris portability - cfmakeraw() 08 Jan 2013 » Solaris portability - flock() 06 Jan 2013 » pkgsrc-2012Q4 illumos packages now misogynist 23 Nov 2012 » SmartOS and the global zone 24 Oct 2012 » Setting up Samba on SmartOS 10 Oct 2012 » pkgsrc-2012Q3 packages for illumos 23 Aug 2012 » Creating local SmartOS packages 10 Jul 2012 » 7,000 binary packages for OSX Lion 09 Jul 2012 » 9,000 packages for SmartOS and illumos 07 May 2012 » Goodbye Oracle, Hello Joyent! 13 Apr 2012 » SmartOS global zone tweaks 12 Apr 2012 »StreamlinedVirtualBox SmartOS installs 30 Mar 2012 » iptables script for Debian / Ubuntu 20 Feb 2012 » New site diamond 11 Jan 2012 » Set up unrecognized FTP upload on Oracle Linux 09 Jan 2012 » Kickstart Oracle Linux in VirtualBox 09 Jan 2012 » Kickstart Oracle Linux from Ubuntu 22 Dec 2011 » Last day at MySQL 15 Dec 2011 » Installing OpenBSD with softraid 21 Sep 2011 » Create VirtualBox VM from the writ line 14 Sep 2011 » Creating chroots for fun and MySQL testing 30 Jun 2011 » Graphing memory usage during an MTR run 29 Jun 2011 » Fix input box keybindings in Firefox 24 Jun 2011 » How to lose weight 23 Jun 2011 » How to fix stdio buffering 13 Jun 2011 » Serving multiple DNS search domains in IOS DHCP 13 Jun 2011 » Fix Firefox URL double click behaviour 20 Apr 2011 » SSH via HTTP proxy in OSX 09 Nov 2010 » How to build MySQL releases 29 Apr 2010 » 'apt-get' and 5,000 packages for Solaris10/x86 16 Sep 2009 » ZFS and NFS vs OSX 12 Sep 2009 » pkgsrc on Solaris 09 Dec 2008 » Jumpstart from OSX 31 Dec 2007 » Set up local caching DNS server on OSX 10.4