Idea for full motion video - MSX 2 - Screen 4 - NTSC 20fps - 16 shades of gray

Champion (467)

12-09-2021, 21:36

Premises:

Names table and patterns table fixed. All patterns 00001111. Will be changed only color table.

There are 59736 cycles per frame on NTSC.
One OUTI takes 18 cycles (Z80 + M1), so I can push around 3300 bytes per frame.

3 frames = 9900 bytes

Entire color table = 32 x 192 = 6144 bytes
Sprite pattern table for 32 16x16 sprites = 32 x 32 = 1024 bytes
Sprite color table for the 32 sprites = 32 x 16 = 512 bytes
Sprite attributes table for 32 sprites = 32 x 4 = 128 bytes

Total = 7808 bytes, fitting well inside 3 frames

The idea is using the 32 16x16 sprites (magnified, so covering a 32x32 area) to complete the image where it is most needed.

This way I will have a frame of a resolution of 64x192, plus a sprite layer covering 67% of the screen to improve the resolution, especially horizontal. The sprites can cover an area of 256x128 pixels, of course with big (2x2) pixels.
They will be positioned at each frame were it make more sense to compose a better picture. That's why it's necessary to blit the Sprite Attr table also.

All these data will be pre calculated on PC and put in a big MegaROM. The MSX assembly code will just read and spit bytes to VRAM. Double buffering with 2 pages of VRAM (there are 8 on screen 4). Pretty simple code on MSX side.

Each second of video will use 20 x 7808 = 156 kb, a full 256 page MegaROM (4MB) will store 25 seconds of video.

Sounds like a cool idea?

By albs_br

Champion (467)

12-09-2021, 22:29

albs_br wrote:

Each second of video will use 20 x 7808 = 156 kb, a full 256 page MegaROM (4MB) will store 25 seconds of video.

Maybe there are enought cycles remaining to decompress data using Pletter, optimizing space.

By albs_br

Champion (467)

12-09-2021, 22:40

albs_br wrote:

Total = 7808 bytes, fitting well inside 3 frames

Maybe 192 lines are unnecessary and wastefull, doubling each line saves 3072 bytes, and a lot of cycles making it more feasible to uncompress data in real time.

Entire color table = 32 x 96 = 3072 bytes
Sprite pattern table for 32 16x16 sprites = 32 x 32 = 1024 bytes
Sprite color table for the 32 sprites = 32 x 16 = 512 bytes
Sprite attributes table for 32 sprites = 32 x 4 = 128 bytes

Total = 4736 bytes, leaving one frame and a half to uncompress data. That should be enought.

By santiontanon

Paragon (1769)

12-09-2021, 23:51

For most frames you don't even need to store that much. Just the bytes that change from one frame to another (trying to have contiguous block to maximize transfer-to-vdp speed), so, it should be much less!

By Metalion

Paragon (1622)

13-09-2021, 08:14

Yes indeed, what you're explaining is the case for a raw video file.
Using simple techniques from MPEG (like this 'code-difference-only') will lower the bitrate.

By wolf_

Ambassador_ (10092)

13-09-2021, 17:59

Regardless of the technical bits, what would we do with it?

There already exists a full motion video concept by nyyrikki: https://www.youtube.com/watch?v=xt2d_jdVaQg
And while fun to see it's possible, it also raises ye olde content question. Playback of random videos from random film makers doesn't make sense, you'd better just go to youtube.

An intro video as part of a scene game? Fine. Who's going to make/direct that video? And will it be in-line with the rest of the game, regarding style? See, it wouldn't make sense to make a full-motion video with all kinds of visual effects if the actual game would be something like Eggerland. :)

Demos? Could be. I'd say Sphere and Wings are a reasonable size in that context, considering that half of it is just wavekits. The other half would be like, oh, dunno, 400kb for 5 minutes o' fun? Quite a bit less compared to 4 MB for 25 seconds (the example from the first post). ;)

Now, don't let me stop you if you just want to prove it can be done. The above is just wondering about what the practical applications would be for such content, also regarding there are probably not many video makers/editors/directors in this scene. B-)

By Grauw

Ascended (10706)

13-09-2021, 18:47

albs_br wrote:

The idea is using the 32 16x16 sprites (magnified, so covering a 32x32 area) to complete the image where it is most needed.

It seems to me that writing the converter which determines this “where it is most needed” will be the most difficult part. Finding an optimal result is probably not feasible so an approximation will be needed, but even that seems difficult to approach while getting sensible results, especially since “pixel” sizes are different (4x1 vs 2x2).

To keep things simple I would skip the sprites part of the idea and focus on the basics. There’s also ready plenty of challenges there like setting up the conversion pipeline and colour reduction. I’m sure it’ll be fun to try even if it has no real practical applications other than to rick-roll the unsuspecting msx.org audience .

By santiontanon

Paragon (1769)

13-09-2021, 23:47

As for one use taken from one of my games: I used a similar idea of a "video converting tool" for the flag effect of my XRacing game. But just converted a 16 frame loop that was played over and over. So, a more general video converting tool (mine was VERY limited) can be used for creating these types of small loops to include in games

By albs_br

Champion (467)

14-09-2021, 00:03

wolf_ wrote:

Regardless of the technical bits, what would we do with it?

There already exists a full motion video concept by nyyrikki: https://www.youtube.com/watch?v=xt2d_jdVaQg

That's indeed a very impressive video, but seems very noisy. And looks like was captured by filming a real CRT TV/monitor (even the screen 0 chars before loading the video are blured), making it hard to know what is the real quality of the footage (looks like 4x1 pixels).

Is there a ROM/DSK or even a WebMSX link of this video available?

Edit:
The link at http://msx.fi/nyyrikki/software.html looks like it's broken

By wolf_

Ambassador_ (10092)

14-09-2021, 06:24

Iirc it's just complete frames converted to sc4, with its own palette per frame, using something like bmp2msx. Because it runs at full speed, and because the images are always on the move, it makes the 8x8 blocks less obvious.

By wolf_

Ambassador_ (10092)

15-09-2021, 10:20

Manuel just uploaded a pixel-perfect version of Dragon's Lair. Not entirely sure it's better than seeing a blurry version, but at least it's good for study.

https://www.youtube.com/watch?v=v75R_BQTCQo