Friday, February 5, 2010

Android Antics

Android's gui toolkit is, programmatically, nothing new and is pretty nice to work with given a phone. It's got a few problems though in how things work. Now stackoverflow is a great place to look to find answers to common programming problems ontop of #android-dev on freenode. But this simple problem I had yesterday and its simple solution took me so long to work through (at least 3 hours?). I hate these types of problems, the solution is simple yet you cannot find the answer cleanly in documentation or otherwise on the internet. So since it seems like I'm the first one to notice the problem in question, I will write about it here.
What is the problem?
Well, for multichoice listviews, you often want a checkbox. The checkbox is merely decoration though as the listview itself keeps all stateful information on being selected. Anyway, a checkbox is focusable and clickable. Because it has these attributes, it can automatically steal the onclick events from the entire listcell. You can solve this by setting the focusable attribute to false. This is where everybody else on the internet stops, and wrongly so because if a naughty user selects just the checkbox in a click, it will only fire the checkbox's events and not the listviews. Not to mention the state of the checkbox is then screwed up wrt the listview's. The solution after many maddening hours of trying different methods to have the checkbox "click" the listitem (which at the very least has a drawback of not highlighting the selection on press).. simply disable the clickable attribute that the button inherits from View.

So in summary, and hopefully so the searchengines catch it...
When putting buttons, checkables, or checkboxes into a listview's cell, set the focusable and clickable properties to false so the listview works properly.

Thursday, January 21, 2010

pulseaudio, a wii, and the 10-dollar tvcard

So I've got a saa7134 tvcard (specifically a kworld tv-terminator) that was always troublesome to get working but worked when it worked (as in it's a PITA to get working but is stable thereafter). Over the years, the way to use this card has changed as the driver got fixed up more and as linux itself changed. MPlayer seems more able to control the card properly than tvtime, which gets stuck in some less-than acceptable options. Well anyway, the most troublesome part of this card is that if you want low latency, you have to tell the driver to do this and have to run another program to reroute audio from the tvcard to your soundcard - for whatever reason mplayer cannot do this and so... well thats where things get interesting. For reference, my mplayer command is:
mplayer tv:// -tv input=2:norm=1:immediatemode=1:amode=1 -vf pp=lb
This is for svideo-in/720p and low latency stereo mode that has a linear blur applied. The blur fixes jumpy video which has some analog source or something.

Now for the sound:
After opening mplayer you must do either something like (the old way which happened to work with low latency)
arecord -D hw:1,0 -r 32000 -c 2 -f S16_LE | aplay -N -R 0 -c 2 -r 32000
Or, since todays linux uses pulseaudio, you can use module-loopback. There are a bunch of other ways you could redirect the audio from the tvcard to your system's soundcard but the problem is that they are usually full of latency (by anywhere from .5 seconds to 2). The two options I propose are not and are essential for any kind of gaming where the time from button press to sound being heard is critical.

Are you ready for how to do it with pulseaudio? Well, at first it seems easy, and probably will be for most of you - simply
pacmd load-module module-loopback
It autodetected things pretty good, I used pavucontrol for some other slight configuration. But there was one problem, things sounded ok but there was this.... pausing. The default sampling rate on my pulse configuration is 48000hz and there seems to be a problem with all of the parameters of my cheap card and autodetection. On inspection of pacmd list-sources, I confirmed pulse had set itself to 48000hz while the hardware/alsa driver only genereted 32000... that means every second, there were 16 thousand samples completely missing which just somehow turned out to be zeroed (thus the pausing, there was logically no audio being played each pause as the pcm audio data was zeros). The solution was then to specify the 32000 to the module-alsa-source module for the tvcard. The exact command was
pacmd load-module module-alsa-card device_id="1" name="pci-0000_09_01.0" card_name="alsa_card.pci-0000_09_01.0" tsched=yes ignore_dB=no rate=32000
I derived these parameters from a pacmd list-modules, looking for the argument line who's card_name parameter matched the pci id of tvcard (you can see these using paman or pacmd list-sources). Note btw that before you execute the above command specifying rate, you must unload the module-alsa-card who has current ownership of the card (you use the index of the module to pacmd unload-module)

Also note that since things are going through pulse, this audio is now subject to the pulseaudio processing chain, and more specifically, the equalizer sink, and still with acceptable latency.

That's all folks - happy audio trails.

Sunday, November 22, 2009

txamqp vs the java amqp client

Whew, what a difference.

I did some benchmarking with a modified version (not modified for performance) of the java amqp client (for rabbitmq) and txamqp. Man is it a world of hurt for txamqp.
I get a sustained 33k/sec msg rate with the java amqp client and there's some poor code in there which could use some hashtables. This is for the consumer program which I describe below.
The better setup I have in txamqp though only is about 1k/sec for the consumer program... a pretty huge difference.
Both clients are doing the same work - the benchmark consists of two programs for each client: a producer and consumer.
Producers produce a small json object built from the json.org library - it contains some random data per object so we always have to make/serialize a new object.
Consumers register a consumer to a queue and parse the json object for each message and then explicitly basic-ack the message has been processed.

Each is done as fast as possible with the rabbitmq (1.7.0) running completely in ram (no swap). However each is also done at a different time to assure that only one is executing at a time. I do it in bursts of 100k msgs. And finally, both version's producer's are faster than the consumer.
but again it's hugely in the java client's favor. 350 msecs to send 100k msgs. It takes about 20 seconds for the same workload with txamqp.

Here's hoping there's an improvement - I did some profiling though and couldn't figure anything in particular except maybe txamqp's codec implementation. Profile file here: http://archer.mine.nu/~jason/100k.prof

Wednesday, October 14, 2009

qpaeq dynamic band subdivision

So heres a slick little feature to qpaeq you aren't likely to find in any other equalizer out there... simply widen the window and watch the magic happen.

Normal - this is the default number of bands available in most equalizers

A little more...

a little bit mooore...

and here's the widest I've ever made it


Notice that this still all has an octave feel to it. You get diminishing returns with that - all you've got to do is change the DEFAULT_FREQS in the qpaeq.py though, preferably to something linear. I'll make a linear bands an option in the future (along with the default of octaves) - it's useful for my own purposes as I explain below.

Why did I do this? Elementary my dear Watson - hand crafted notch filters, that's why! I've got a bunch of songs with horrible noise in them that I can never find the frequency the noise belongs to - this makes it only a few minute ordeal and I've gotten results much better than audacity's stuff in less time. Oh and it just looks cool - go get qpaeq and try it out - it's nice and interactive while you stretch and shrink.

One of these days I'll try and record filtered audio (not too hard thanks to pulseaudio and monitors) to show people the difference equalization can make.

Oh, btw - I'm finally in pulseaudio mainline. You can also get qpaeq in releasish form here: https://sourceforge.net/projects/qpaeq/

Configuring redmine to be your all in one project manager

Well, it was a lot of work (when I did it the first time), but it could have been worse. The goal was to take redmine's user/group information and tie it into the sourcecode management stuff with all the groups preserved. There are recipes for subversion integration of this kind and this was a great base, but not only was this subversion only, I didn't like that a magic script ran every x minutes to create repositories that didn't exist. But I used all of the stuff up to and including the mysql auth stuff. So follow that tutorial first and break off when you've completed mysql nss/pam configuration. A word of warning btw, I tried to make this stuff as secure as possible, but I could not do everything needed without resorting to two very careful entries in the /etc/sudoers file.

I should also point out that you probably will want to run redmine as a system user. I created a redmine user for this and wrote an appropriate configuration for lighttpd. I also made redmine work over HTTPS, but this is something googleable. From here on out, it's assumed that the redmine user exists and is whom redmine runs as.

Lets configure and take a look at the environment.

Perform the following actions as root.
mkdir /home/redmineusers
chown -R root:root /home/redmineusers
cd /home/redmineusers
mkdir bin projects
chown -R root:root bin
chown -R redmine:root projects


After following the above tutorial, make a change to the shell_column property:
users.shell_column = "/bin/rbash";
users.homedir_column = "/home/redmineusers/";

/home/redmineusers/.profile
umask 0007
export PATH=$HOME/bin/
There are symbolics links from .bashrc and .bash_login. This seemed necessary for ssh to work ok with the environment.

/usr/local/bin/hg.wrapper
#!/bin/bash
SRVSTRING="^-R projects/[a-z0-9-]+ serve --stdio$"
export PATH=/bin:/usr/bin:$PATH
CWD="$(dirname "$PWD/")/$(basename "$PWD/")"
HOME="$(dirname "$HOME/")/$(basename "$HOME/")"
if [[ $CWD == $HOME && $@ =~ $SRVSTRING ]]
then
/usr/bin/hg -R "$2" serve --stdio
if [[ $? == 0 ]]
then
/usr/bin/sudo /usr/local/bin/fixrepoperms "$2" 2>/dev/null
exit 0
else
exit 1
fi
else
exit 1
fi
Make a symbolic link in /home/redmineusers/bin/hg -> /usr/local/bin/hg.wrapper

/usr/local/bin/createrepo
#!/bin/bash
ROOT_PATH=/home/redmineusers/projects/
#RE="^[a-zA-Z0-9 ]+$"
RE="^[a-z0-9-]+$"
if [[ $1 =~ $RE ]]
then
cd "$ROOT_PATH"
/usr/bin/hg init "$1"
chown -R redmine:"$1" "$1"
chmod -R o-rwx "$1"
find "$1" -type d -exec chmod a+s "{}" ";"
chmod -R g+rw "$1"
true
else
false
fi

/usr/local/bin/fixrepoperms
#!/bin/bash
#RE="^[a-zA-Z0-9 ]+$"
RE="projects/[a-z0-9-]+"
export PATH=/bin:/usr/bin:$PATH
CWD="$(dirname "$PWD/")/$(basename "$PWD/")"
HOME="$(dirname "$HOME/")/$(basename "$HOME/")"
if [[ $CWD == $HOME && $@ =~ $RE ]]
then
prj=`basename "$1"`
chown -R redmine:"$prj" "$1"
chmod -R o-rwx "$1"
find "$1" -type d -exec chmod a+s "{}" ";"
chmod -R g+rw "$1"
else
false
fi
And now two additions to to your /etc/sudoers:
redmine ALL = NOPASSWD: /usr/local/bin/createrepo
ALL ALL = NOPASSWD: /usr/local/bin/fixrepoperms
And now you apply a patch. Its dirtyish code but it works ok and is much cleaner than things I've seen in the past for this stuff - for instance when you create your project, you won't have to wait x minutes for the repo to show up!

Index: app/controllers/projects_controller.rb
===================================================================
--- app/controllers/projects_controller.rb (revision 2924)
+++ app/controllers/projects_controller.rb (working copy)
@@ -74,6 +74,13 @@
@project.enabled_module_names = Redmine::AccessControl.available_project_modules
else
@project.enabled_module_names = params[:enabled_modules]
+ r = Repository.factory("Mercurial",:root_url => "/home/redmineusers/projects/#{@project.identifier}",
+ :url => "/home/redmineusers/projects/#{@project.identifier}")
+ @project.repository = r
+ g = Group.new({ "lastname" => @project.identifier} ) #blidly create a new accompanying group
+ gr = Role.givable.find_by_name("Developer")
+ gm = Member.new(:principal => g, :roles => [gr], :project => @project)
+ @project.members << m =" Member.new(:user"> User.current, :roles => [r])
@project.members << controller =""> 'projects', :action => 'settings', :id => @project
end
end
+
+
end

def copy
Index: extra/svn/create_views.sql
===================================================================
--- extra/svn/create_views.sql (revision 2924)
+++ extra/svn/create_views.sql (working copy)
@@ -13,7 +13,7 @@
from projects;

CREATE OR REPLACE VIEW nss_users AS
-select login AS username, CONCAT_WS(' ', firstname, lastname) as realname, (id + 5000) AS uid, 'x' AS password
+select login AS username, CONCAT_WS(' ', firstname, lastname) as realname, (id + 5000) AS uid, 'x' AS password,
from users
where status = 1;
Make sure you rerun the create_views.sql script. It should rerun just fine and now users have to be active rather than just exist.

If you've completed the above, congratulations. You now have a secure, private sandboxed environment for every project (in which you're protected by good old unix permissions) that is tied to your redmine user/group database. Each project gets it's own group on creation and you must add developers to this group to get sourcecode management access. You also have a secured / restricted environment for users to ssh into - but really only as a gateway to hg serve. You should be able to configure this to your needs for your flavor of SCM. Redmine should also have your repository entered upon project creation (it seems you must make a commit though before it looks like it works, this is understandable).

project management - the FOSS choices

Long time no post. Well, I've recently had to come up with a sourcecode management system, one which would work well for a growing buisiness. I took to researching the available options and here is a summary of my results. I checked out many options, but mainly two came out on top (and the only two I have time give a blog based comparison to).
  • Trac
  • Redmine
Trac is well known and quite good if you've only got one project. But given it's limitation's, it's had many offshoots now. Officially, trac is only subversion, but things are changing and branched projects exist for using git at least. Of keen interest though is that while there is a issue/ticket tracking system - there is only support for one project. There is a goal to change this in the future but we're all in the here and now.

Redmine, though. Here's the underdog. I mean who's heard of it, right? It's built on ruby on rails, if you care about that. For me, given prior knowledge of similar frameworks, that made it pretty easy to hop in and get some interesting things done, in a more proper way than the typical alternatives (these still are hackish things though). I'll go into those changes next post. But there's a few great features up for consideration that are built in to redmine:
  • Built in non hackish support for multiple projects and the management framework for users/groups.
  • tickets are cross project (after the option is clicked in the admin settings)
  • support for most any SCM (mercurial, git, bzr, subversion) out there
Now of considerable importance to me was mercurial. I view mercurial/git/bzr as mostly equivilent but their crossplatform support is anything but. mercurial has tortoisehg for the non cli window users though and works without a hitch other places though and thus is my SCM of choice for a business environment. What's that, why not subversion? Subversion, while useable sucks. The master repo goes down, well guess what - you can't check in your changes or have a VCS until its back up. Alternatively, this means you can work locally without having inet or being connected to that master server. Additionally, there's no tree support - this kills parallel development by multiple users. Well anyway, to recap -
I was motivated to find project management software which handled multiple projects, had ticket tracking, and SCM integration (focusing on mercurial) with users/groups working as expected all throughout the stack. I didn't find a complete and ready solution, but I got very close, and I'd be interested if anyone found something better. Until then, I thank the redmine crew, they've made an excellent system.

Wednesday, September 2, 2009

single disk backups and splitting the songs... such a pain

Ever backup a cd and find that you ripped it as one freaking file? Yes, I've been there and done that. It's a real pain to get the files to sane names and preserve the metadata on the cue sheet. So I found this blog entry, which works almost ok . I'm surprised it even handles a few formats. But guess what? The whole freaking thing FAILS at spaces and anything beyond basic ascii? Why oh why do people insist on using bash for these things? The end product is always so quirky. But you know what? I'm stepping up to this. I've already created the audiomap project with similar goals in mind. And now I'm expanding my bullcrap free softwares to include tagrename: a program which renames files based on their metadata and format strings. I'd like to expand it a little more in terms of the formatting strings (I whipped up this guy in about a half hour), but it's quite usable right now.

It accepts format strings.. use like:
./tagrename.rb -f "%track - %title" *.flac -v

Get help like:
./tagrename.rb -h

(with all possible tag properties title,artist,album,comment,genre,year,track). Tags can properties can even appear multiple times if that is your thing. I will try to include printf like formatting for numbers in the future. As usual though, any audio format is supported with no variance in behavior. Get your menial tasks done and be on your way with your life.

Sorry to those who only know how to use GUIs. I might add utilities to guitize my programs in the future, but for now you're on your own.

ps. If the file is not in the release copies, go check git, I may not have packaged it yet.