Qt sucks at option parsing

Qt doesn't handle parsing program options (ie --foo=1) very well. But don't worry, boost program options can save the day! The problem is how to hook up boost::program_options to Qt's arguments after it has parsed Qt specific options.

#include <boost/program_options.hpp>
std::string qstr2str(const QString &qstr){
   return qstr.toStdString();
}

int main(int argc, char *argv[]){
QApplication application(argc, argv);
namespace po = boost::program_options;
po::options_description clspec("Options");
clspec.add_options()
("help", "see available options")
("arg1,a", po::value(&arg1)->default_value("banana"), "the arg1 option")
("arg2,b", po::value(&arg2)->default_value(false), "the arg2 option")
;

QStringList qargs = application.arguments();
std::vector args(qargs.size());
std::transform(qargs.begin(), qargs.end(), args.begin(), qstr2str);

po::variables_map vmap;
po::basic_command_line_parser parser(args);
parser.options(clspec).style(0).extra_parser(po::ext_parser());
po::store(parser.run(), vmap);
po::notify(vmap);
}

And with that you have parsed the options from Qt after it's had it's way with them

cross compiling for android

The NDK, last time I looked, did an OK job of building small jni modules, but what if you needed to use third party libraries? Then you were in for a rough ride. But I managed to get something working together. First go get crystax's NDK so you have a real C++ environment. Not strictly necessary but let's keep surprises to a minimum shall we?

After installation I set the following variables, which you'll probably want to stuff into a file you can source from time to time:

export NDK_HOME=/opt/android/ndk
export TOOLCHAIN_ROOT=$NDK_HOME/build/prebuilt/linux-x86/arm-eabi-4.4.0/
export PLATFORM_ROOT=$NDK_HOME/build/platforms/android-8/arch-arm
export PATH=$PATH:${TOOLCHAIN_ROOT}/bin

You'll also need the following files:

ag++

agcc

and if you use CMake or it's needed for other libraries (ie OpenCV), here's a toolchain file:

android-arm.cmake

A note on the cmake toolchain file: It is setup mostly to be standalone but there are some complex things going on that agcc/ag++ take care of internally that would be difficult to add to the toolchain file to modify cmake's behavior appropriately. If you have improvements, please be vocal about them!

Anywho, place agcc/ag++ in your path and mark them executable. Then just set CC/CXX appropriately. Source the ndk variables set above and everything should takecare of itself with autoconf/automake projects. For cmake projects, use the cmake file above and the first line lists an example calling. Keep in mind that bionic, the c library used in android, is pretty small though so don't be surprised if you have to patch out some c functions you've never heard of.

Here's an example for installing libtiff to your ndk:

./configure --host=arm-eabi CC=agcc CXX=ag++ \
CC=agcc CXX=ag++ \
CPPFLAGS="-D__i386__ -DANDROID -D__ARM_ARCH_5__ -D__ARM_ARCH_5T__ -D__ARM_ARCH_5E__ -D__ARM_ARCH_5TE__ -D__ANDROID__  -I/opt/android/ndk/build/platforms/android-8/arch-arm/usr/include" \
CFLAGS="-nostdlib -march=armv7-a -mtune=xscale -mfloat-abi=softfp -mfpu=vfp -mthumb-interwork  -ffunction-sections -fstack-protector -fno-short-enums -Wno-psabi" \
LDFLAGS="-Wl,-rpath-link=/opt/android/ndk/build/platforms/android-8/arch-arm/usr/lib -L/opt/android/ndk/build/platforms/android-8/arch-arm/usr/lib -Wl,-rpath-link=/opt/android/ndk/build/platforms/android-8/arch-arm/lib/gcc/arm-eabi/4.4.0 -L/opt/android/ndk/build/platforms/android-8/arch-arm/lib/gcc/arm-eabi/4.4.0 -Wl,-rpath-link=/opt/android/ndk/build/platforms/android-8/arch-arm/arm-eabi/lib -L/opt/android/ndk/build/platforms/android-8/arch-arm/arm-eabi/lib -WWl,--fix-cortex-a8 -lgcc -lstdc++ -lsupc++ -lm -lc" \
--prefix=/opt/android/ndk/build/platforms/android-8/arch-arm/usr --disable-cxx
make install DESTDIR=/opt/android/ndk/build/platforms/android-8/arch-arm/

Note that this compiles and installs to the android 2.2 platform.

Kudos to Andrew Ross for the original agcc script, which I've modified above.

alfresco and glassfish a nono

Well, not just glassfish v3, but also postgresql. No, really, it's just not worth it to get it in your java container of choice. Sure it may work on first launch, but it may not restart cleanly. And it will take forever to launch. And you will have to tune the memory usage, which doesn't seem to correspond to configs for deploying on tomcat.

Just use the prepacked binaries for this one folks, I've never had a case of this so much as with alfresco. It starts fast, works as it should, no weird errors, and self-recovers much better.

Android Antics

Android's gui toolkit is, programmatically, nothing new and is pretty nice to work with given a phone. It's got a few problems though in how things work. Now stackoverflow is a great place to look to find answers to common programming problems ontop of #android-dev on freenode. But this simple problem I had yesterday and its simple solution took me so long to work through (at least 3 hours?). I hate these types of problems, the solution is simple yet you cannot find the answer cleanly in documentation or otherwise on the internet. So since it seems like I'm the first one to notice the problem in question, I will write about it here.

What is the problem?

Well, for multichoice listviews, you often want a checkbox. The checkbox is merely decoration though as the listview itself keeps all stateful information on being selected. Anyway, a checkbox is focusable and clickable. Because it has these attributes, it can automatically steal the onclick events from the entire listcell. You can solve this by setting the focusable attribute to false. This is where everybody else on the internet stops, and wrongly so because if a naughty user selects just the checkbox in a click, it will only fire the checkbox's events and not the listviews. Not to mention the state of the checkbox is then screwed up wrt the listview's. The solution after many maddening hours of trying different methods to have the checkbox "click" the listitem (which at the very least has a drawback of not highlighting the selection on press).. simply disable the clickable attribute that the button inherits from View.

So in summary, and hopefully so the searchengines catch it...

When putting buttons, checkables, or checkboxes into a listview's cell, set the focusable and clickable properties to false so the listview works properly.

pulseaudio, a wii, and the 10-dollar tvcard

So I've got a saa7134 tvcard (specifically a kworld tv-terminator) that was always troublesome to get working but worked when it worked (as in it's a PITA to get working but is stable thereafter). Over the years, the way to use this card has changed as the driver got fixed up more and as linux itself changed. MPlayer seems more able to control the card properly than tvtime, which gets stuck in some less-than acceptable options. Well anyway, the most troublesome part of this card is that if you want low latency, you have to tell the driver to do this and have to run another program to reroute audio from the tvcard to your soundcard - for whatever reason mplayer cannot do this and so... well thats where things get interesting. For reference, my mplayer command is:

mplayer tv:// -tv input=2:norm=1:immediatemode=1:amode=1 -vf pp=lb

This is for svideo-in/720p and low latency stereo mode that has a linear blur applied. The blur fixes jumpy video which has some analog source or something.

Now for the sound:

After opening mplayer you must do either something like (the old way which happened to work with low latency)

arecord -D hw:1,0 -r 32000 -c 2 -f S16_LE | aplay -N -R 0 -c 2 -r 32000

Or, since todays linux uses pulseaudio, you can use module-loopback. There are a bunch of other ways you could redirect the audio from the tvcard to your system's soundcard but the problem is that they are usually full of latency (by anywhere from .5 seconds to 2). The two options I propose are not and are essential for any kind of gaming where the time from button press to sound being heard is critical.

Are you ready for how to do it with pulseaudio? Well, at first it seems easy, and probably will be for most of you - simply

pacmd load-module module-loopback

It autodetected things pretty good, I used pavucontrol for some other slight configuration. But there was one problem, things sounded ok but there was this.... pausing. The default sampling rate on my pulse configuration is 48000hz and there seems to be a problem with all of the parameters of my cheap card and autodetection. On inspection of pacmd list-sources, I confirmed pulse had set itself to 48000hz while the hardware/alsa driver only genereted 32000... that means every second, there were 16 thousand samples completely missing which just somehow turned out to be zeroed (thus the pausing, there was logically no audio being played each pause as the pcm audio data was zeros). The solution was then to specify the 32000 to the module-alsa-source module for the tvcard. The exact command was

pacmd load-module module-alsa-card device_id="1" name="pci-0000_09_01.0" card_name="alsa_card.pci-0000_09_01.0" tsched=yes ignore_dB=no rate=32000

I derived these parameters from a pacmd list-modules, looking for the argument line who's card_name parameter matched the pci id of tvcard (you can see these using paman or pacmd list-sources). Note btw that before you execute the above command specifying rate, you must unload the module-alsa-card who has current ownership of the card (you use the index of the module to pacmd unload-module)

Also note that since things are going through pulse, this audio is now subject to the pulseaudio processing chain, and more specifically, the equalizer sink, and still with acceptable latency.

That's all folks - happy audio trails.

txamqp vs the java amqp client

Whew, what a difference.

I did some benchmarking with a modified version (not modified for performance) of the java amqp client (for rabbitmq) and txamqp. Man is it a world of hurt for txamqp.
I get a sustained 33k/sec msg rate with the java amqp client and there's some poor code in there which could use some hashtables. This is for the consumer program which I describe below.
The better setup I have in txamqp though only is about 1k/sec for the consumer program... a pretty huge difference.
Both clients are doing the same work - the benchmark consists of two programs for each client: a producer and consumer.
Producers produce a small json object built from the json.org library - it contains some random data per object so we always have to make/serialize a new object.
Consumers register a consumer to a queue and parse the json object for each message and then explicitly basic-ack the message has been processed.

Each is done as fast as possible with the rabbitmq (1.7.0) running completely in ram (no swap). However each is also done at a different time to assure that only one is executing at a time. I do it in bursts of 100k msgs. And finally, both version's producer's are faster than the consumer.
but again it's hugely in the java client's favor. 350 msecs to send 100k msgs. It takes about 20 seconds for the same workload with txamqp.

Here's hoping there's an improvement - I did some profiling though and couldn't figure anything in particular except maybe txamqp's codec implementation. Profile file here: http://archer.mine.nu/~jason/100k.prof

qpaeq dynamic band subdivision

So heres a slick little feature to qpaeq you aren't likely to find in any other equalizer out there... simply widen the window and watch the magic happen.

Normal - this is the default number of bands available in most equalizers

A little more...

a little bit mooore...

and here's the widest I've ever made it

Notice that this still all has an octave feel to it. You get diminishing returns with that - all you've got to do is change the DEFAULT_FREQS in the qpaeq.py though, preferably to something linear. I'll make a linear bands an option in the future (along with the default of octaves) - it's useful for my own purposes as I explain below.

Why did I do this? Elementary my dear Watson - hand crafted notch filters, that's why! I've got a bunch of songs with horrible noise in them that I can never find the frequency the noise belongs to - this makes it only a few minute ordeal and I've gotten results much better than audacity's stuff in less time. Oh and it just looks cool - go get qpaeq and try it out - it's nice and interactive while you stretch and shrink.

One of these days I'll try and record filtered audio (not too hard thanks to pulseaudio and monitors) to show people the difference equalization can make.

Oh, btw - I'm finally in pulseaudio mainline. You can also get qpaeq in releasish form here: https://sourceforge.net/projects/qpaeq/