I'm very sorry to say my problem persists....
I've got it working like this now:
oggenc2_aoTuV -q %q -t %1 -a %2 -l %3 -d %4 -N %5 -o %o -
oggenc2_aoTuV is OggEnc v2.8 (libvorbis aoTuVb4.51) from www.rarewares.org/ogg.html
. With the Xiph-build (the non-optimized one), you've got more chance that it works. %q is the quality you want, works from -1 (-2 with aoTuV) to 10, I know it works up to 7. Also, I found out that adding "-G %6" or "-c %7" results in a broken file.
[5 minutes later]
Great! Now it doesn't give any working ogg anymore
If I run OggInfo on the file while it is being created, it reports a working ogg, even if the resulting ogg in the end if broken.
I'd say the problem is still mysterious. Ian sent me some modified versions and the last one actually worked with all OggEncs I had, but now it feels more like there's an external factor messing things up...