Canon XM2 (DV) to DVD, on Linux

I wanted to transfer some material from DV cassettes to DVD. My main workstation is running Ubuntu 12.04, and I decided to use the tools that are available with the distribution. I tried multiple ways of doing each of the tasks, and git many dead ends, mainly due to crashing programs, bugs, or incompatible tools. For instance, tovid looked very promising until it turned out that it is not compatible with the new version of the ffmpeg utility. My source material was DV, recorded by Canon XM2, the video format was 768×576, interlaced (576i), with audio at 48kHz, PCM, stereo. Interlacing was giving me some headache, because the first attempts lead to unsightly stripey output. The camera outputs double-scan interlace, which should be interpreted as 50 frames per second with reduced resolution. Interlacing might be tricky

The first step is to capture the video from the camera. Connect the camera to the laptop, switch the camera to the playback mode, rewind the tape and:

dvgrab birthday-

The “birthday-” bit is a prefix that will be added to the saved .dv files. dvgrab will save multiple 1GB files, each file about 4 minutes long. Once the material is captured, you can merge the multiple files into one, by simply concatenating them:

cat birthday-001.dv birthday-002.dv birthday-003.dv > birthday.dv

Once you have one file with the complete material, fire off a player and note down (I used paper and pencil) the times of segments you want to extract. You won’t be able to do a lot of cutting that way, but if it’s a couple of segments, it shouldn’t be too labor intensive. Once you know what are the segments you want to extract, you can extract them and encode as .vob files. Suppose one fragment starts at 02:13 and is 135 seconds long:

avconv -i birthday.dv -target pal-dvd -flags +ilme+ildct -b:v 6000k -ss 02:13 -t 135 birthday-01.vob

The “+ilme+ildct” bit is responsible for correct handling of interlacing, because DV uses different field order than DVD. Repeat the above command for each segment, and you’ll get a list of VOB files. These VOB files are DVD compliant, and they are implementing the interlace correctly. They must not be re-encoded when transferred to DVD, otherwise the interlacing settings will be most likely lost. You can try if your interlacing settings are correct by watching the VOB file using VLC with automatic deinterlace detection:

vlc --deinterlace -1 --deinterlace-mode bob --play-and-exit birthday-01.vob

You should see no stripes during movement in the video, and the displayed frame rate should be 50fps (although the video frame rate is set to 25fps).

The next step is to create a DVD menu. There is a number of DVD authoring software. I had most success with DVD Styler. I also tried tovid, and Bombono.

In DVD Styler, I managed to create a DVD directory structure, but not an ISO image, and I was not able to burn a DVD directly from DVD Styler. Instead, I only generated the DVD structure on disk, and used k3b, using its DVD template. I created a new project, found the generated VIDEO_TS directory from DVD Styler, and added it to the project in k3b. This was enough to arrive at a working DVD.

DVD Styler would recognize that the files are already DVD compatible and did not attempt to re-encode them.

The above method is rather basic and crude, but gets the job done. There isn’t a video editor used at any stage; instead we just note down the times and then extract time regions using the -ss and -t options of avconv. I tried to use pitivi for video editing, but there were issues with rendered video, and since I didn’t really need any editing, I dropped pitivi from the workflow. The main problem to solve in pitivi would be to encode a DVD compliant VOB video file. You can select a DVD VOB as the output format, but there’s still a lot of things you can mess up, for instance accidentally encode audio in 44.1kHz instead of 48kHz, which results in a DVD disc with no audio.

I suspect that tovid will be reasonably soon adapted for use with the new ffmpeg tools (using /usr/bin/avconv instead of /usr/bin/ffmpeg), which will make it easier to script out the process if I had more of such (e.g. archival) DVDs to make.

Choir with bass and drums, can it work?

I was wondering why some choirs, when touring with two-piece band, go for piano+drums, as opposed to bass+drums. Piano can play bass notes, but the piano+drums setup leaves the overall sound not as full as a dedicated bass instrument.

Any choir, when singing á capella, will drop in pitch over the course of a song. Even the brilliant Perpetuum Jazzile . Try playing along to e.g. Mas Que Nada, and you’ll see that they drop at least a semitone. It’s a gradual process over a few minutes, rather hard to spot by an unguided ear.

When there’s a piano or guitar playing with the choir, it’ll help the choir hold the pitch. I thought that bass guitar would do that as well. It sounded so good as an idea: you can have a piano trio or a guitar trio, with one main harmonic instrument and a rhythm section. Choir is a type of a harmonic instrument, so all it needs is a rhythm section! Therefore, bass+drums+choir should work great.

It turns out, this combination doesn’t work out. The bass guitar does not in practice help the choir hold the pitch. One explanation could be that it’s generally hard to hear the pitch of low notes. I tried to play higher notes, but it still didn’t help. The idea of a choir with a rhythm section doesn’t work.

UPDATE 2012-07-24: I got a good effect by playing chords in some parts of the song (somewhere up the neck) and bass notes elsewhere. For instance, I’d play chords in the verse, and bass notes in the chorus. It was effective.

Headless VirtualBox setup

VirtualBox, unless you look deeper, is a desktop application. However, it is possible to install it on a server without a monitor, and run virtual machines there. The setup procedure is somewhat quirky, and it has changed since the last time I did it. If you search for VBoxManage and VBoxHeadless, you’re likely to hit outdated instructions. If you’re reading this in 2020, these instructions are probably also out of date. Here’s the 2012 edition. Tested on Ubuntu Oneiric Ocelot.

Initial setup for installation of the OS from an ISO image. In my case, it was Solaris 10, which will be my small private porting and OpenCSW Solaris package building host.

UPDATE 2012-06-04: I’ve checked in the code into a subversion repository on The script is called

VM_NAME="Solaris 10 x86"
DISK_SIZE=20000 # In MB

function setup {
VBoxManage createvm \
--name "${VM_NAME}" \
--basefolder "${DISK_DIR}" \
VBoxManage modifyvm "${VM_NAME}" \
--memory "${MEMORY_IN_MB}" \
--acpi on \
--boot1 dvd \
--nic1 bridged \
--bridgeadapter1 eth0
VBoxManage createhd \
--filename "${VDI}" \
--size "${DISK_SIZE}"
VBoxManage storagectl "${VM_NAME}" \
--name "IDE Controller" --add ide
VBoxManage storagectl "${VM_NAME}" \
--name "SATA Controller" --add sata
VBoxManage storageattach "${VM_NAME}" \
--storagectl "SATA Controller" \
--type hdd --device 0 --port 0 \
--medium "${VDI}"
VBoxManage storageattach "${VM_NAME}" \
--storagectl "IDE Controller" \
--type dvddrive --device 0 --port 0 \
--medium "${CD}"

If something went wrong and you need to start over, you can nuke your VM with:

VBoxManage unregistervm "${VM_NAME}"
rm -rf "${DISK_DIR}/${VM_NAME}"

To start your virtual machine:

VBoxHeadless -s "${VM_NAME}" --vnc

You can now connect to your virtual machine via VNC. You will probably need to set up a VNC tunnel and use a VNC viewer. Ubuntu has a remote desktop application.

When you’ve installed your guest OS, you need to remove the installation media, otherwise your VM will start the installation process all over again. You can stop the VM (I just press CTRL+C, there must be a better way).

VBoxManage storageattach "${VM_NAME}" \
--storagectl "IDE Controller" \
--type dvddrive --device 0 --port 0 \
--medium none

You can start your VM again, this time it will boot into the installed OS.

Zyxel P-660HW-T1 vs IPv6 tunnel vs SIP

I’m a SIP/VoIP user.  It allows me to make international calls for the cost of a local connection.  I have set up a Linksys PAP2T gateway with a regular phone connected to it. Worked great for calling out.  Unfortunately, the SIP gateway kept on logging out of the SIP server.  It could last a day or an hour, but it would always eventually lose the logged-in state and never return to it without human intervention.  Resetting the gateway would not help, it was usually necessary to switch it off for 15 minutes or so.  A quicker method was to change the SIP port from 5060 to 5061 and back every time I needed to restore the service.  I tried fiddling with the PAP2T settings, but no setting changes seemed to alleviate the issue.

I’m also a an IPv6 user.  I’ve got an OpenWRT installation on WRT54GL, running aiccu and providing an IPv6 tunnel from SixXS.  The tunnel had a similar ailment: it would go down every couple hours to days.  The workaround was to restart aiccu.  I would restart it when I needed it.

At some point, I started neglecting the IPv6 tunnel.  I didn’t need to use it, and I just didn’t bother to restart it.  At the same time, I noticed that the SIP gateway would stay logged in without dropping out for much longer than usual.  This state remained for about two weeks, until I needed to reinstate the IPv6 tunnel.  Right after doing that, I walked over to the SIP gate and… noticed that it had dropped out.  Correlation does not imply causation, but you know… it raised my suspicion.  What is it that these two devices have in common?  The router!

A search on Google for “zyxel sixxs sip” revealed a forum post, in which someone described the same symptoms I had, with a bit of diagnostics.  Both the IPv6 tunnel and the SIP service are using UDP, which is harder to NAT than TCP.  The Zyxel router would repeatedly get confused and misinterpret the IPv6 related UDP packets as SIP packets and vice versa.

The solution was to take away the logic out of the Zyxel router and make it act as a DSL modem only. I’ve reconfigured it to the bridge / transparent mode, and moved the NAT logic to OpenWRT.  I initially wasn’t sure how the bridging mode works, but it turned out to be simple enough.  What’s nice about my particular setup is that I have a static IP address.  The setup uses a /30 network block, so it’s effectively using up 4 of the IPv4 address space.  It’s essentially a 2-bit netblock, so we can use 0, 1, 2, and 3 as the addresses.  In practice it can be something like, and any CIDR calculator will tell you, that if your netmask is /30 (or, then it’s a 4-address network starting at and ending at Let’s consider the NAT mode first.

  • 0: netblock address
  • 1: the router tells you it’s the the remote side (the gateway)
  • 2: the router’s public IP address
  • 3: the broadcast address
It might look like the 1 address is remote, at the ISP side.  But if you configure your router in a bridging mode, you have 2 devices, and they both have public IP addresses.  Let’s call them Zyxel (acting as a DSL modem) and OpenWRT (doing NAT).
  • 0: netblock address
  • 1: the Zyxel router’s address (gateway)
  • 2: the OpenWRT router’s address
  • 3: the broadcast address
So what you might have thought of as the remote IP address, is your local router’s address.  What was there left to do, was configuring NAT on OpenWRT.  Linux knows how to interpret incoming UDP packets, and both my SIP gate and IPv6 tunnel are working correctly now.  Plus, I have more control.

FLOSS Weekly 163: OpenCSW, addendum

FLOSS Weekly, a podcast about Free-Libre and Open Source software, episode 163 featured OpenCSW, a project I actively participate in.

Since I was not on the podcast, I would like to use this opportunity to add to what has been said there.

Q: 05:30 What is OpenCSW and what does it contribute to the world?
A: …to add to what Phil said (we provide packages free as in free beer), there are two parts of what we provide: one part is binary packages, and the other part is the source code to build these packages.  It hasn’t been historically the culture at OpenCSW (or formerly, Blastwave) to release build recipes.  At OpenCSW, the policy for all new maintainers is to release source code of all packages they build.  However, there is still a number of old-timers, who build packages using own, unpublished scripts.  We are making efforts to have all build recipes published as open source, and while we’re still not there yet, it’s one of the most important points on our agenda.  In this sense, we do care about freedom and about being an open source project.

Q: 15:15 Do you think of OpenCSW as of a Solaris distribution?
A: Yes, as much as it is possible, while being based on commercial Solaris. The main difference between OpenCSW and Linux or BSD distributions is that OpenCSW does not provide the base OS, such as the kernel, libc or an installer.  From the perspective of a business which runs third party applications, it’s important that their OS is supported by the vendor.  Nexenta is a lovely Debian-based system with a Solaris kernel, but you can’t get support for an Oracle database on it.

Continue reading “FLOSS Weekly 163: OpenCSW, addendum”

A git-svn workflow

Git, the version control system, has a subversion integration feature.  In short, it allows you to use git to interact with a subversion repository: check out and commit code back to subversion.

It sounds like an excellent feature.  My first thought was that I’ll be able to make a subversion checkout on one host, and use git to propagate my changes throughout all other host I work on.  Finally, I thought I’d be able to merge changes back to where I did the subversion checkout, and commit my changes back to subversion.

However, there’s a problem with using multiple repositories, git merge and git svn. The most common way to move changes between git repositories is git push and git pull. In short, it does git fetch, and git merge (plus a bit of magic I’m not sure about).  If you use git pull/push and try to commit back to subversion, you will sooner or later fall into the trap that I fell into some time ago.  I’ve since worked out a relatively sane workflow which allows me to work with code on multiple hosts and send changes back to subversion with no horrible recurring code conflicts.

The executive summary is: Use “git fetch; git rebase origin/master” to propagate changes and format-patch + “git am” to propagate changes back to your root git repository (the one that was used to make the svn checkout).

host1> git svn clone foo
host2> git clone ssh://host:/home/joe/foo
host2> cd foo; edit files…
host2> git stage -p
host2> git commit
host2> git format-patch origin/master
host2> scp *.patch host1:
host1> cd foo
host1> git am ~/*.patch
host1> git svn dcommit
host1> some more work, commits, code incoming from subversion…
host1> git svn rebase
host2> cd foo
host2> git fetch
host2> git rebase origin/master

In this workflow, git pull and push are not used.

host1 → host2 [git rebase origin/master]
host2 → host1 [git format-patch; scp; git am]

You need to take care if you’re using things such as “git commit –amend” on host1 on commits that have been first made on another host (e.g. host2).  Otherwise, this workflow does not create any recurring conflicts.

A side note: it’s perfectly safe to branch and merge within one git repository.  The workflow described above deals with the problem of moving changes across multiple git repositories.