BitBuster 1.2

by snout on 26-11-2003, 11:41
Topic: Software
Tags: Utilities
Languages:

Team Bomba have released a new version of BitBuster, a set of tools to compress and decompress files. The compression rate is more or less equal to that of POPCOM, but BitBuster decompresses a lot faster. BitBuster is used in Team Bomba productions like the WOEI! demo and their upcoming masterpiece Bombaman. You can use BitBuster in your own productions as well. New in this version:

  • Support for multiple filenames when compressing (also with wildcards)
  • Somewhat faster compression
  • Faster decompressor for Z80
  • ARM-version of decompressor
  • Binaries for linux

Relevant link: Bombaman HQ

Comments (16)

By anonymous

incognito ergo sum (116)

anonymous's picture

26-11-2003, 12:46

Mwuhahaha, BitBuster has nothing on TCF!
But ofcourse, TCF isn't publicly available Tongue ^^;

Still nice work Smile

By Arjan

Paladin (718)

Arjan's picture

26-11-2003, 13:37

the big question of course is: will you release it? Smile

btw, when Bombaman is finished, I will create a better version of bitbuster which will compress the files faster, and it will also compress files better. Tthe compression ratio can be improved without breaking the format, but maybe I'll change the format so files can be compressed even better.

hmm it would be cool to create a MSX compression tools comparison page too Smile

By anonymous

incognito ergo sum (116)

anonymous's picture

26-11-2003, 15:45

Good work !

I only want to know 2 things: (hehe, someone said "comparison" ?)

1) how many bytes Z80 unpacking code takes ?

2) how fast Z80 decompressing works ? For example, how much time it takes in average to unpack 16-24K of data ?

I am using PRO PACK, and it unpacks 24K in 0.5-1.0 sec.

Is BitBuster faster ?

By Arjan

Paladin (718)

Arjan's picture

26-11-2003, 15:50

I already answered it at our HQ, but I'll post it here as well...

The decompressor is only 172 bytes. If that's too big for you, can can de-unroll it to save some bytes but decompressing stuff will be slightly slower.

The bitbuster decompressor takes ~0.25 seconds to decompress 13K of data back to 16K. In another testcase it took 0.2 seconds to decompress 3.5K back to 9K. So, files that are compressed better also take some more time to decompress.

In the first case, bitbuster would be able to unpack 64K per second, while in the second case it would be 45K per second. So it is faster than PRO PACK, but if I remember correctly PRO PACK compresses files better.

With bitbuster, it is allowed to overwrite crunched data. For example, if you have loaded a compressed file of 8K that decompresses back to 16K, you could load that data at ~ $B010 and decompress the data to $8000.

By Arjan

Paladin (718)

Arjan's picture

26-11-2003, 15:51

err, $B010 should be $A010

By ro

Scribe (4364)

ro's picture

26-11-2003, 17:06

And does it beat RT (Redundancy Terminator, a Fuzzy compressor)?
(sorry, that wasn't fair) It's cool to see ppl using own (de) compressors and squeezing crap out of data. I always thought it wasn't done that much (well, maybe a new trend)
(I mean building own routines)

I should benchmark the RT (de) compressor one of these days... hmmm, been yeears since we build it.

By Arjan

Paladin (718)

Arjan's picture

26-11-2003, 18:00

we could do some comparisons at the developers meeting...

By cax

Prophet (3736)

cax's picture

26-11-2003, 18:19

I see, BitBuster really beats PRO PACK - it's unpacking code is about 256 bytes and BitBuster unpacks faster. Too bad, I didn't start with BitBuster and I am too lazy to rewrite MicroWAVer's code...

I also don't see myself repacking all these multi-bload ROMs, where better unpacking time between files may save 1-2 seconds in loading time... (will you really feel the change between loading Usas in 2 min 33 seconds or 2 min 31 seconds ?)

But maybe one day I will rewrite, if there will be big demand in making MicroWAVer cross-platform (currently it uses embedded PRO PACK executable, sources of which I don't have).

As for packing rate itself, I found BitBuster 1.1 and PRO PACK performing equally, on some files one beats another, on others - vise versa. Seems like now BitBuster 1.2 is better in packing, but I can't confirm this without testing again.

By Arjan

Paladin (718)

Arjan's picture

26-11-2003, 19:00

bitbuster 1.2 is not really better than 1.1 when it comes to compression ratio (most of the time it's the same, sometimes +- a few bytes). Bitbuster can handle small files fairly well (it sometimes beats LHPACK!), but then again, only with files of about 1K Tongue

and indeed, just changing to bitbuster because of the faster decompressor might be too much of a trouble.

By sjoerd

Hero (599)

sjoerd's picture

26-11-2003, 19:05

Maybe at the developers meeting you could come up with the ultimate cruncher, taking the best from bitbuster/tcf/rt.

and ofcourse TCF isn't available. Uhm, why not? Wait, I know why... :-)

By Arjan

Paladin (718)

Arjan's picture

26-11-2003, 21:20

there is no such thing as the ultimate cruncher Tongue

By anonymous

incognito ergo sum (116)

anonymous's picture

27-11-2003, 00:42

sjoerd wrote: "and ofcourse TCF isn't available. Uhm, why not? Wait, I know why... :-) "

You do?

I will release TCF one day, but I want to tweak it some more before I do.

As a small test I packed the Circus Charlie ROM with a few utils:
Original: 16384
BitBuster: 13116 (LZ77 derivative)
TCF: 13088 (LZ77 derivative)
ApLib: 12626 (LZ77 derivative)
LZH: 12507 (LZSS + Dynamic Huffman)
ZIP: 12499 (LZ77 + Static Huffman)
RAR: 12275 (lZ78 and/or(?) PPM)

BTW, the TCF decompressor is only 152 bytes.

By cax

Prophet (3736)

cax's picture

27-11-2003, 14:15

Circus charlie ROM:
PRO PACK - 13192 (with method 2 - the only one supported by decoder)

By Arjan

Paladin (718)

Arjan's picture

27-11-2003, 16:26

the not-unrolled version of Bitbuster is only 124 bytes, and for my example it only takes 0.01 second longer to decompress the file. It's possible to bring that back to 102 bytes, which costs an additional 0.05 second extra (0.26 seconds total).

By anonymous

incognito ergo sum (116)

anonymous's picture

04-12-2003, 14:39

Like GuyveR800 said:

ApLib: 12626 (LZ77 derivative)

The speed-optimised version of the depacker is 262 bytes, the size-optimised one is 159 bytes and you could even save a couple more bytes if you're not interested in the depack routine returning the depacked size. I hope to release the depacker it the not too distant future Wink

I could do some depack-speed-tests as well, but I think it would be wise to first come up with a default set of files to test all MSX (de)packers with, otherwise speed comparisons become a little meaningless.

By anonymous

incognito ergo sum (116)

anonymous's picture

04-12-2003, 16:32

Yeah, a corpus for MSX files would be cool.
We could include Moonblaster files, GE5 files (both full screen drawings and tile graphics), program code files, and ofcourse some random data stuff Smile