Gunzip for MSX 1.0 released

Gunzip for MSX 1.0 released

by Grauw on 18-10-2015, 17:56
Topic: Software
Tags: grauw, gunzip, gzip
Languages:

Yesterday Laurens Holst released gunzip, a high-performance gzip (.gz) archive decompressor. Gzip is a relatively common file format to compress single files, with compression ratios comparable to most other archive formats.

You can get it from the downloads page on the project site.

This gunzip implementation is at least 2x as fast as other decompression tools for msx (excluding special-purpose formats like BitBuster). Tests showed that a 720K disk image can be decompressed in 26 seconds on turboR, and 133 seconds on MSX2. Full performance test results are presented in the forum thread, which also contains some technical details about the techniques used to achieve this high performance.

Gunzip is open source and liberally licensed, so you can reuse the code in any way you want. Gzip’s deflate compression algorithm is also used by various other file formats such as zip and png, so hopefully it will come in useful.

The project was started because VGM files for VGMPlay are commonly compressed with gzip. Until now, to play back these .vgz files you had to extract them on PC, but now you can use gunzip to decompress these files on the MSX. Later it will be integrated into VGMPlay.

Relevant link: Gunzip for MSX

Comments (16)

By Louthrax

Prophet (2093)

Louthrax's picture

18-10-2015, 18:59

This looks like a major contribution to the Z80 scene, and it started on MSX Cool Thanks Grauw !

By snout

Ascended (15187)

snout's picture

18-10-2015, 20:58

I cannot put into words how epic this is, particularly when looking at the sheer speed of the decompressor. First ZIP, now gzip. Not a bad year for MSX I'd say.

By syn

Paragon (1920)

syn's picture

18-10-2015, 21:04

The guy who makes runrar will be a hero for ever! Tongue

By Grauw

Ascended (8516)

Grauw's picture

18-10-2015, 21:34

The problem with rar and 7zip is that the dictionary sizes are too big (megabytes / gigabytes). Effectively on MSX it means you have to decompress the entire file into memory before writing to disk. Whereas the dictionary size for gzip and zip is just 32K which will fit in main memory just fine without even dealing with memory mapping.

So although they achieve a little higher compression ratios, I don’t think rar and 7z are suitable for MSX due to the memory requirements. There’s no need for them either, as gzip and zip are still extremely popular formats with built-in support in nearly every modern desktop OS.

By syn

Paragon (1920)

syn's picture

18-10-2015, 22:02

Hehe I was joking, as a play on the names of the msx unpackers released this year Big smile

By Grauw

Ascended (8516)

Grauw's picture

18-10-2015, 23:59

Hehe yeah, I know, but it was worth mentioning anyway Smile.

By cbsfox

Champion (327)

cbsfox's picture

19-10-2015, 06:28

Congratulations for your great unzip tool.
Is it possible to create a zip tool for MSX or it would take one day to zip a 16k file?

Regards
Marcos

By Grauw

Ascended (8516)

Grauw's picture

19-10-2015, 09:55

I don’t know how long compression would take but probably not a day Smile. I think it could be done at comparable speeds to lharc (which is pretty quick). There’s a trade-off between compression ratio and speed and memory consumption, which could be controlled by a command line option. Atm I don’t have plans to work on a that though...

By cbsfox

Champion (327)

cbsfox's picture

19-10-2015, 16:43

OK. But is it difficult to implement a zip tool?

By Grauw

Ascended (8516)

Grauw's picture

19-10-2015, 16:58

Assuming that a gzip compressor is a project comparable to gunzip;

Technically it’s quite challenging. Time-wise, looking at the gunzip commit history I worked on it for 14 days over the span of 6 months (excluding the test implementation in JavaScript).

Though now that I’ve implemented gunzip I could do gzip a little quicker, it’d probably still be a full holiday of work.

By Parn

Champion (427)

Parn's picture

19-10-2015, 17:10

This is awesome news and should be in the Spotlight. Why isn't it? Congrats to Grauw for his hard work and amazing achievement. Big smile

By cbsfox

Champion (327)

cbsfox's picture

19-10-2015, 20:11

Cool. Hope you can create a grip compressor as well.

Congrats
Marcos Daniel

By Pippo

Champion (421)

Pippo's picture

20-10-2015, 11:28

Hi Grauw,
you are a nice expert! Smile
This is my opinion.

By Whizzy

Master (197)

Whizzy's picture

20-10-2015, 22:15

Nice work Laurens ! very impressive

By Pippo

Champion (421)

Pippo's picture

21-10-2015, 07:48

Hi Whizzy,
I'm glad to meet you.
Please, can you reply to my post in software section?
Thank you.

By jlemos

Resident (56)

jlemos's picture

24-10-2015, 02:16

Very cool!

Thanks Grauw!!!!

Big smile