FLOSS Weekly, a podcast about Free-Libre and Open Source software, episode 163 featured OpenCSW, a project I actively participate in.
Since I was not on the podcast, I would like to use this opportunity to add to what has been said there.
Q: 05:30 What is OpenCSW and what does it contribute to the world?
A: …to add to what Phil said (we provide packages free as in free beer), there are two parts of what we provide: one part is binary packages, and the other part is the source code to build these packages. It hasn’t been historically the culture at OpenCSW (or formerly, Blastwave) to release build recipes. At OpenCSW, the policy for all new maintainers is to release source code of all packages they build. However, there is still a number of old-timers, who build packages using own, unpublished scripts. We are making efforts to have all build recipes published as open source, and while we’re still not there yet, it’s one of the most important points on our agenda. In this sense, we do care about freedom and about being an open source project.
Q: 15:15 Do you think of OpenCSW as of a Solaris distribution?
A: Yes, as much as it is possible, while being based on commercial Solaris. The main difference between OpenCSW and Linux or BSD distributions is that OpenCSW does not provide the base OS, such as the kernel, libc or an installer. From the perspective of a business which runs third party applications, it’s important that their OS is supported by the vendor. Nexenta is a lovely Debian-based system with a Solaris kernel, but you can’t get support for an Oracle database on it.
OpenCSW is installable on top of commercial Solaris, allowing the best of the two worlds: it’s still the officially supported commercial Solaris, and Open Source software is easily available for it. Within these constraints, we consider OpenCSW a distribution in the sense that it can integrate well with the base OS. Installing packages such as vim or PostgreSQL is one thing you can do, but you can do more: you can replace Sun daemons with OpenCSW provided ones. For example, you can replace Sun SSH with OpenSSH, Sun printing system (lpd) with CUPS and stock Solaris syslogd with syslog_ng. We also provide a few mechanisms which are typical for distributions, for example an alternatives system, similar to Debians.
Q: 09:30 – What’s the trouble with not being able to depend on a specific version of a package?
A: It makes it harder to control upgrades, and it can lead to breakages for users who try to control them.
Here’s an example: if you want to upgrade package foo which depends on libbar, and libbar is not in the latest version on your system, pkgutil will want to upgrade libbar too. The reason is that it doesn’t know whether updated foo will work with the older libbar. At times, users don’t want to upgrade libbar, because they are concerned that this upgrade may cause problems with other software that they run. It sometimes happens that they try to go around this problem by using the –nodeps option, which forces pkgutil to ignore dependencies. It works some of the time. However, when new foo also depends on libbaz, libbaz is not getting installed and the user is no longer able to run foo, until they examine and repair dependencies by hand.
Q: 19:35 What was the impact of the acquisition of Sun by Oracle?
A: There was no visible impact on the community, because OpenCSW never directly depended on Sun. We would definitely be interested in testing our packages on Solaris 11, so there’s potential to collaborate.
Q: 20:25 How is your community structured?
A: We have a board, which is in charge of conflict resolution if needed, but day to day, OpenCSW is a dictatorship (with all its problems and benefits). Phil Brown, the release manager, makes final decisions as to what makes it to the official package catalog, and what doesn’t. On occasion, if it is required, we’re voting.
Q: 28:01 How many packages to you have?
A: We’re approaching 3000 packages at the moment. There is a web page showing graphs with the number of packages we provide. The recent increase in packages is partly thanks to new software, and partly due to the fact that we recently started splitting software into more fine-grained packages. One of the bigger changes was a standard of packaging shared libraries, which currently follows that of Debian.
Q: 28:25 How easy or difficult is it to create a package from a software I have locally?
A: Dagobert Michelsen has developed a build system called GAR, which abstracts away all the low-level packaging operations. You can write a declarative-style Makefile and GAR will do all the right things to create a Solaris package. After creating it, GAR automatically runs checkpkg, a lintian-style tool which checks packages for problems. If you do something wrong with shared libraries, checkpkg will give you a detailed explanation about what’s wrong.
The main hurdle for someone who wants to get started with Solaris packages, could be setting up the environment. We work on the buildfarm, which was set up once and is working since; we don’t test new installations very often. However, we’re always there on #opencsw to offer help to people who wish to build Solaris packages.
Q: 29:37 I found FreeBSD a lot more sensible to me, sorry guys…
A: Don’t be. I personally don’t think of OpenCSW as of a project advocating or promoting Solaris. It’s more of what happens once you’re stuck with Solaris. It can be for various reasons, for example your company running an Oracle database on sparc hardware. In such case OpenCSW is a project that can make your life easier.
Q: 32:05 What’s the package build activity?
A: As Ben indicated, there hasn’t been historically a lot of visibility to what other people were doing, but since recently, we have a page displaying the buildfarm activity. On average, we have from a dozen to a couple dozen package builds a day. We work on increasing visibility, so that it’s easier for package maintainers to see what others are doing. Apart from build activity, an interesting place to pay attention to, is our source code repository. It contains build recipes for our packages, and our package build system. We are promoting a culture in which all code commits are posted to a devel mailing list, where they are reviewed and commented on. It’s especially useful when new maintainers start submitting code.
Q: 34:04 Are you looking for more assistance? Are you looking for people to contribute to the project? If so, what kind of people?
A: At this point, we’re mainly looking for package maintainers. Skill set involved is mainly system administration knowledge, and C/C++ knowledge when code patching is required. There is a number of packages that are not receiving updates; as Ben mentioned, there is a number of packages created by maintainers who have since changed jobs or teams, and no longer work with software they packaged.
We’re constantly working on making it easy to jump in, and provide a package upgrade. If there’s a package that hasn’t received an upgrade in some time, and you’d be interested in providing that upgrade, talk to us on #opencsw on Freenode.
One large area are desktop-related packages. We mainly focus on server software, but there are also desktop packages, which currently don’t have active maintainers.
Another area is packaging tools and automation. As a Perl advocate, Randal might like this part: we work to creating an environment in which it’s easy to write tools in various languages. We’ve started using RESTful interfaces. There is a quite a bit of Python code in the infrastructure, but there is also Perl, Ruby and Java code. RESTful interfaces makes interoperability very easy. If you want to write a bit of automation that interacts with the package catalog, you can make a HTTP GET request, deserialize a JSON data structure and use it.
Phil said that we’re looking for people who like Solaris. I would like to add that we especially welcome people who dislike Solaris, and need to work with it for other reasons. Many OpenCSW maintainers are also Linux users, and agree that grep which doesn’t understand -q does not contribute to a satisfactory userland. This is one of the problems we are set out to fix.
Q: 34:51 What’s your infrastructure like? Are you building things with, I’m hoping, Perl, but it’s probably Python…
A: The buildfarm hardware is kindly provided by Baltic Online (thanks!). It’s mainly a big sparc machine with a lot of RAM and disk, running multiple Solaris versions at the same time, as Solaris zones. On the software side, there is GAR, which is based on Garnome, but has been greatly extended and has most of the parts replaced by new code. GAR is written in GNU Make, with a number of utilities written in Perl. There is also a database which holds information about all the package catalogs, and is able to check, for instance, whether there are any file collisions (e.g. two packages trying to deliver the same file). The tool to check packages (checkpkg) is written in Python.
Q: 35:35 Is there anything else that you would like our audience to know about OpenCSW?
A: It’s worth noting that pkgutil, an apt-get/aptitude/yum style package downloader , is able to use multiple package sources. You can point it at the OpenCSW package catalog, and at an in-house built package repository by specifying multiple “mirror=” lines in the pkgutil.conf file. It’s very handy for businesses which use Solaris and deploy software using Solaris packages.
It’s also possible to use Puppet, a configuration management tool mentioned in episode 93. Puppet can use a so-called pkgutil provider, which is currently available from OpenCSW thanks to patched puppet sources, and will be included in future official versions of Puppet.