IDE HDD via SMB very slow [solved]

Page 3/4
1 | 2 | | 4

By ro

Scribe (4363)

ro's picture

09-01-2021, 15:29

Took me a while, but finally got things set-up as I prefer it to be. Thus far I really love openMSX.
Still using network share for my HDD, it works fine.

But, savestates are giving me the same problem as reverse. The checksum kicks in... Again taking a while. For obvious reasons.

I've used blueMSX for years, no problem there.

2cents

By wouter_

Champion (453)

wouter_'s picture

09-01-2021, 16:10

I understand that the 'blueMSX approach' (no hard disk image checksum in the savestate) can be more convenient.

Though there's also a risk. The (emulated) MSX always makes some assumptions about the state of the (emulated) hard disk. E.g. from the emulated MSX point of view it may have only 'recently' read some sectors from disk and as far as the MSX is aware those sectors cannot have changed. So it can safely make changes to the disk based on those assumptions. One common example is keep the FAT sectors cached in memory. Based on this cache, the MSX might think that some clusters are still free, and thus can be safely overwritten for new files.

However if the hard disk image can change then these assumptions are no longer valid. For example you might make a savestate (or record a replay/reverse), then make some hard disk changes, and after that you load the savestate (or go back in the replay/reverse). In combination with the above this can lead to a corrupt hard disk image and/or lost data.

To protect against this openMSX includes a checksum (hash) of the hard disk image in the savestate/replay. On load that checksum is compared against the current disk content, and if there's a mismatch the emulated hard disk is made read-only.

Maybe the chance of actually triggering corruption using the above scenario is not that high, but it's certainly not zero. And it might be devastating if it triggers on an important project you're working on. Therefor in openMSX we made the tradeoff of being safe at the cost of taking more time. For not too big hard disk images on a local filesystem it still works pretty well. But I can understand that for network filesystems that may be less the case.

By ro

Scribe (4363)

ro's picture

09-01-2021, 18:01

Thanx for the clarification, Wouter. It sure makes sence, and I'm praising the level of detail in openMSX. No doubts there.
Optimizing diskreads should fix the overall HD usage for checkums, local and network, I reck'n.

By Manuel

Ascended (17747)

Manuel's picture

09-01-2021, 23:51

FYI: reverse = savestate + event log and some extra savestates for performance in searching through the timeline.

By wouter_

Champion (453)

wouter_'s picture

23-01-2021, 17:20

ro wrote:

Hai Wouter, thanx for chiming in.

Reading a file in 512bytes, on a modern file system with 4k of blocks sure slows down. Let alone using SMB of CIFS on that. You'd rather read a big chunk to memory and create a hash on 512bytes from there.

Hope you can optimize some Smile

Hi,
OpenMSX now reads blocks of 64kB instead of 512 bytes. I did a few simple tests (with a local harddisk image of 100MB) and it improved startup time by a factor 2.5x (0.40 -> 0.14 seconds). I suspect the speedup might be larger on a network filesystem and/or with larger harddisk images.

By Manuel

Ascended (17747)

Manuel's picture

23-01-2021, 23:15

Can you give that a try ro, with the latest development build?

By sdsnatcher73

Paragon (1708)

sdsnatcher73's picture

24-01-2021, 07:05

Nice development! Great work.

By Manuel

Ascended (17747)

Manuel's picture

24-01-2021, 23:49

Well, let's first get ro to test it with that version, to see what the improvement is in his situation.

By ro

Scribe (4363)

ro's picture

05-02-2021, 13:22

hai, I missed this post completely. Great to hear such a quick follow up. I was just reminding myself to check the forum since I really miss doing savestates while coding. I'll download and test it, see you in a bit..

By ro

Scribe (4363)

ro's picture

05-02-2021, 14:08

got openmsx-16.0-360-g908732227-windows-vc-x64 downloaded and installed. great.
I noticed doing savestate doesn't do a hash, just like loadstate. So, it's quick. I like that but is it correct?
(previously it did a hash on the HDA)

Did some other tests, indeed hashing is a bit quicker now. more test will reveal if it is workable.

thanx

Page 3/4
1 | 2 | | 4