gbadev.org forum archive

This is a read-only mirror of the content originally found on forum.gbadev.org (now offline), salvaged from Wayback machine copies. A new forum can be found here.

DS Misc > DSM Files

#129209 - JaJa - Sun May 20, 2007 6:09 pm

Okay. Someone recently mentioned this new file format that Moonshell may be extended to support (not too sure about that, but there is a stand alone player).

You can find a windows encoder and the nds binary here:
http://mdxonline.dyndns.org/archives/2007/05/dsmplay_ver05.shtml

However, linux and mac users have once again been left without an encoder.

I had a look and this format appears to use mpeg1 video and a heavily downsampled audio track encoded with True Audio.

The binaries for encoding included with DSMplay are:

Mencoder - appears to be used to spit out the wav file that is used by ssrc and to get info on the file.

ffmpeg - used to encode the mpeg1 video

ssrc_hp - Shibatch Sample Rate converter, this is used to downsample (and dither and such) the wav spat out from mencoder.

ttaenc - true audio encoder. used to create true audio stream from the downsampled wav.

Unfortunately ssrc fails with an 'unknown error 1' on my G3 ibook under Tiger (maybe someone can try running it under Linux or Intel Tiger?), so I haven't been able to generate the streams (let alone think of combining them).

The encoder uses the following commands to prepare the streams:

ffmpeg -t 1 -v 1 -y -vcodec mpeg1video -qscale 4 -acodec mp3 -ab 160 -i <input file> <input file>.ffmpeg.mpg
ffmpeg -v 1 -y -vcodec mpeg1video -qscale 4 -acodec mp3 -ab 160 -i <input file> <input file>.ffmpeg.mpg
mencoder -v <input file>.ffmpeg.mpg -noautosub -ovc copy -af format=s16le,resample=48000:1:2,channels=2,volume=0:1 -oac pcm -of rawaudio -o <input file>.dsm1.wav
ssrc_hp --rate 32768 --dither 0 --bits 8 --normalize --pdf 2 --twopass <input file>.dsm1.wav <input file>.dsm2.wav
ttaenc -e <inpute file>.dsm2.wav

Of course I have since whipped out my hex editor to try and decipher the container format (it's quite simple actually). Details of the header to follow in my next post.
_________________
LAWL HOOGE
My Blog

#129218 - tepples - Sun May 20, 2007 6:47 pm

Is this any better than DPG?
_________________
-- Where is he?
-- Who?
-- You know, the human.
-- I think he moved to Tilwick.

#129221 - JaJa - Sun May 20, 2007 6:55 pm

Okay onto the header.

Once again the first 36bytes contain the offset (in bytes) of various things in the file. The audio file starts straight after this header. All this data is based on a 30 second sample of Death Note episode 12 (created using mencoder <death note ep> -enpos 30 -oac copy -ovc copy <output>) encoded with the dsmenc.exe file included with DSMplay 0.41 (he released 0.5 today, haven't seen if there are any changes in the output).

Hex Decimal Value What it is
44 53 4D 31 D S M 1 dsm header
B8 C7 11 00 1,165,240 video data
00 66 FC 00 16,541,184 video data bytes
B8 2D 0E 01 17,706,424 video seek table data
88 04 00 00 1,160 video seek table data bytes
24 00 00 00 36 audio data
14 C7 11 00 1,165,076 audio data bytes
38 C7 11 00 1,165,112 audio seek table data
80 00 00 00 128 audio seek table data bytes

(I apologize for the mess, I can't do tabs or even multiple spaces! So I'll upload my crappy text document with the same annotations later. To help I'll write the hex in red, the decimal in black and the right most column in green.)

After this 36 bytes the audio stream starts.

54 54 42 31 T T B 1 true audio version tag
02 00 00 00 2 number of channels
08 00 00 00 8 bits
50 0F 00 00 3,920 hz
00 10 00 00 4,096 frame length in samples
EE 00 00 00 238 frame count
80 04 00 00 1,152 ???

I only got this far in the stream. After this it's a bunch of zeros (in my sample file) and then some data after a few hundred bytes (maybe my sample is silent at the start?). Next I looked at the video stream.

Mpeg1video (I assume) file header..

CF 02 00 00 719 frames signed 32
00 01 00 00 256 h-width signed 32
90 00 00 00 144 v-height signed 32
00 08 00 00 32,768 sample rate signed 32
20 13 00 00 25,900 ?? signed 32
00 00 00 00 0 ?? signed 32

Again only the first few bytes of the stream (the stream is some 16mbytes) but I feel it shows the important bits.

Now, this new format has seek tables. I had a look and they simply seem to indicate how many bytes in the file to jump.

audio seek table..

10 00 00 00 16 ???
0F 00 00 00 15 ???
00 00 00 00 0 ???
00 00 00 00 0 ???
00 00 01 00 65,536 signed 16bit?? 1 +65,491
64 C2 00 00 49,764 bytes skip audio stream? +81,160
00 00 02 00 131,072 2 +65,536
64 FF 01 00 130,924 +85,796
00 00 03 00 196,608 3 +65,536
90 4E 03 00 216,720 +85,476
00 00 04 00 262,144 4 +65,536
74 9C 04 00 302,196 +86,216
00 00 05 00 327,680 5 +65,536
3C ED 05 00 388,412 +86,388
00 00 06 00 393,216 6 +65,536
B0 3E 07 00 474,800
00 00 07 00 458,752 7 +65,536
6C 70 08 00 553,068
00 00 08 00 524,288 8
34 8B 09 00 625,460
00 00 09 00 589,824 9
48 AB 0A 00 699,208
00 00 0A 00 655,360 10
48 D1 0B 00 774,472
00 00 0B 00 720,896 11
44 13 0D 00 856,900
00 00 0C 00 786,432 12
14 56 0E 00 939,540
00 00 0D 00 851,968 13
88 90 0F 00 1,020,040
00 00 0E 00 917,504 14
D0 C4 10 00 1,098,960

That's the full audio seek table. The numbers on the right are the amount of to add the the decimal in on the same line to get to the next value. There appears to be a 16bit signed int (indicating some kind of 'sync point' maybe?) and a 32bit signed int, which is the number of bytes in the file to skip (I think). I have colored the 16bit ints in blue and the 32bit ones in red.

Next post is the video seek table.
_________________
LAWL HOOGE
My Blog

#129226 - JaJa - Sun May 20, 2007 7:15 pm

The video seek table appears to follow a similar format (value then number of bytes to skip) although both values are 32bit ints (I think).
No idea what the first 16 bytes represent though, in either table.

video seek table..

05 00 00 00 5 ??
09 00 00 00 144 ??
00 00 00 00 0 ??
00 00 00 00 0 ??
B1 1A 00 00 6,833 'syncpoint' number
AC 5F 00 00 24,492 bytes to skip
62 35 00 00 13,666
6C BF 00 00 49,004 bytes to skip
14 50 00 00 20,500
B4 22 01 00 74,420 bytes to skip
C5 6A 00 00 27,333
8C ED 01 00 126,348 bytes to skip

... this continues for some 1000 bytes which I won't annotate (i'll show last entry though)

24 E9 0E 00 977,188
74 A5 FA 00 16,426,356 bytes to skip

Notice that in both the seek tables the last entry is a few hundred less than the length of the stream. This is what led me to suspect that it's the number of bytes to skip.

That's all I have for now. If anyone wants my set of samples, logs, encodes and notes leave a message and I'll upload them to rapidshare.

The main problems I face now are:

Trying to find another sample rate converter (probably use sox again) and working out what causes an entry to be made in the audio/video seek tables and how they relate to each other.

@Tepples; I feel that it looks slightly sharper. The main problem is that a 30 second clip comes out to 17Mbyte! If you look at some of the screen shots on the moonshell website you see a 24 minute anime show comes to 1.1Gbyte, so the format needs some kind of compression (or lower bitrates I guess). It's main additions are the seek tables. This makes seeking through video a lot less painless (in the half hour or so I've played with it).
_________________
LAWL HOOGE
My Blog

#129297 - h0t1ce - Mon May 21, 2007 6:24 pm

@JaJa

Thank you so much for looking into this new video container. Are you planning to make a easy to use script for people to use? if not, do you mind if I try making one once you're done finding how the header, seek tables, etc works?

#129309 - JaJa - Mon May 21, 2007 8:14 pm

Sure, go ahead. I lack the programming skills to write anything (Juhees wrote the headermaker for the dpg file format all that time ago, I just worked out the format and how to make the streams.).

I just thought it might be fun to try and work out the format.

Probably not going to have any more time until next weekend to look at the seek tables though.
_________________
LAWL HOOGE
My Blog