Nandemo PiGa the binary <-> wav converter for tapes

Page 5/6
1 | 2 | 3 | 4 | | 6

By pgimeno

Resident (58)

pgimeno's picture

19-12-2019, 23:48

mcolom wrote:

To me the problem is that sampling a continuous signal (the tape signal after the analog filter) and applying a digital filter to an already digital signal is completely different. When you increase the sampling rate of the input signal you're not adding new information (actually, it's just an oversampling), but allows to load the game with the emulator. It means that the filter is not stable enough.

That's not my interpretation of what happened. My interpretation is that one single sample of the 44100 Hz WAV file, made the difference between a 0 and a 1 for certain bits that were very close to the threshold for them to be interpreted one way or the other. Therefore, having sub-sample resolution helped to more accurately determine the point at which the zero crossing happened. That information isn't completely lost; think linear interpolation.

Let's focus on the two samples that cause a particular zero crossing. If you trace a line from one to the other, the point at which this line intersects zero will be an approximation to the point at which the original sampled signal crossed zero, better than with no interpolation, with very high probability (assuming that in the original signal there was no significant component with a frequency above the sampling rate).

So, if the first sample is closer to zero than the second one, the zero crossing most likely happened near the first sample; if the opposite, the zero crossing happened most likely near the second sample. That information is there, thanks to the relative distance of each sample to zero.

That's my interpretation of what happened. That's independent of the HP filter. The actual interpolation algorithm was cubic, not linear, but that doesn't change the argument; it should just produce a zero crossing which is, with high probability, closer to the actual position than the linear one.
 

mcolom wrote:
pgimeno wrote:

(In case you wonder, no, it doesn't work without the filter either, I still don't know the reason as the signal is not too bad).

The example you show is a case where the BIOS should be able to perfectly to load data.

Yet it doesn't. That and other examples have me puzzled.
 

mcolom wrote:

The BIOS takes care of computing the width of the short pulses (and it assumes that the long ones take twice). It needs to be robust to cassette players with different speeds and jitters. A case when it could fail is when it computes a pulse width in the header and then it's really different later. Or if there's a slow width increase that makes the BIOS fail to recognize a short or long width before a header re-synchronizes again.

The algorithm, as I understand it, consists of measuring how many zero crossings occur in the time window that was determined in advance from the leading tone, immediately after a zero crossing. If there are 0 or 1 zero crossings, the bit is taken as a zero; if 2 or 3, the bit is taken as a 1. More crossings = failure. If the number of zero crossings is even, it waits for the next one, so as to keep in sync. Then it waits for the next zero crossing and the cycle repeats.

Now, if the point used as the start of the window is moved, as happens with the falling edges of the transitions between 0's and 1's, that can confuse the algorithm.

I haven't traced into the code yet, though, so I don't know at which point something goes wrong. I plan to do that at some point, because it's puzzling that this wav is not correctly read.
 

mcolom wrote:

However, indeed you can estimate a convolution kernel in the range of frequencies of the filtered image, and yes... you would correct some (little) distortions. A deconvolution (say, Wiener) is fast, but I don't think it would help much with this particular problem. I don't know, I have that impression.

I don't know either. I've tried to approach this problem from the frequency point of view, learning about some interesting stuff along the way such as the fast Hadamard transform (spoiler alert: it doesn't help), and came to the conclusion that this point of view is not fully adequate.
 

mcolom wrote:

increasing the sampling rate made the emulator load the game. But that doesn't add new information.

It doesn't add information, but it changes the zero crossing points. The emulator rejects most of the information of the wav, anyway, and keeps the crossing points.

By pgimeno

Resident (58)

pgimeno's picture

20-12-2019, 00:00

Manuel wrote:

And only one game (Gnome Ranger) was able to be loaded in openMSX by increasing the sample rate with an interpolation.

Actually Ingrid's Back. Gnome Ranger does not load either way.

By pgimeno

Resident (58)

pgimeno's picture

20-12-2019, 02:43

I've traced the loading process of Gnome Ranger, and found the primary cause for it to fail. It's all in the leading 2600 Hz tone. Following the labelling here: http://www.msxvillage.fr/upload/bios.txt , the comparison at SYN11 fails often, and execution never reaches SYN20 (placing a breakpoint there, it's not triggered). As an additional verification, I replaced the leading tone with Audacity up to the machine code block, and everything up to that point seemed to load fine. The data is OK, the leading tones are not. As a final test, I patched the ROM to increase that comparison to 6. Both Gnome Ranger and Ingrid's Back worked even when disabling the interpolation. To my surprise, even GP.wav worked! BIT2MULTITIMBRAL and Jet Fighter still didn't work.

In the case of GR and IB, this is presumably caused by a deformed belt in the tape recorder at recording time. Any ideas of what kind of filter could help here?

By Manuel

Ascended (16027)

Manuel's picture

20-12-2019, 07:50

How could only the lead tone be deformed? So, have a different frequency than the rest of the signal?

By pgimeno

Resident (58)

pgimeno's picture

20-12-2019, 12:03

No, the frequency vibrates too much, more than the BIOS can cope with.

By mcolom

Resident (57)

mcolom's picture

20-12-2019, 13:23

I've made an histogram of pulse widths of Kaeru and Gnome.
This is Kaeru: https://pasteboard.co/IM7vryf.png[
And this is Gnome: https://pasteboard.co/IM7w8UB.png

In Kaeru it's easy: all the short pulses are of width 8 and the long transitions are 16 or 18.
For Gnome the short pulses seem OK, but the long ones don't. Actually, we find widths from 15 up to 21, with a peak in 18. As pgimeno said, this is too much for the BIOS.

I think this WAV can be easily fixed, just by assuming that pulses from 15 to 21 are actually 18.
However, this would be an external tool (is it worth it? or it's just a few weird tapes which don' t load?). If openMSX can't load this, it's good emulation. A real MSX probably wouldn't neither.

By pgimeno

Resident (58)

pgimeno's picture

20-12-2019, 14:22

mcolom wrote:

For Gnome the short pulses seem OK, but the long ones don't. Actually, we find widths from 15 up to 21, with a peak in 18. As pgimeno said, this is too much for the BIOS.

I don't quite understand. The leading tone is made exclusively of short pulses, and my testing indicates clearly that it's there where the BIOS has trouble, and the data is OK (OK enough for the BIOS to process it correctly, at least).

Side note: There's something wrong with the interpolation. Patching the ROM as I indicated previously, works for Gnome Ranger and GP, but only if interpolation is OFF (it's enabled with no possibility to turn it off, so you need to recompile in order to disable it). With interpolation ON, it seems to detect too many zero crossings, which doesn't make sense to me. I'll discuss this with Wouter on IRC when I can.

By pgimeno

Resident (58)

pgimeno's picture

21-12-2019, 00:38

pgimeno wrote:

Side note: There's something wrong with the interpolation.

Sorry for quoting myself. The problem has been identified and fixed in current master. With this change, and with an unpatched ROM, Gnome Ranger, Ingrid's Back, GP and BIT2MULTITIMBRAL all work, and Kaeru Shooter did not regress. Jet Fighter still doesn't work, as some pulses are too high or too low and the zero crossing is too short. That can be corrected with further HP filtering, but that's not a task for the emulator; that needs to be done with e.g. Audacity. If it works in a real machine, it must be a real machine with a more aggressive HP filter than usual, in which other files will probably fail.

By mcolom

Resident (57)

mcolom's picture

21-12-2019, 00:53

I've just seen in the commit 46c7264f2cdf5a69f8a9db874b3359380b7e9621 that Gnome Ranger now works! Smile

pgimeno wrote:

I don't quite understand. The leading tone is made exclusively of short pulses, and my testing indicates clearly that it's there where the BIOS has trouble, and the data is OK (OK enough for the BIOS to process it correctly, at least).

I computed the histogram after DC removal and 800 Hz high-pass, after implementing the same filter openMSX uses, before adding the cubic interpolation. And both the histogram and the input signal itself looked fine when examined with Audacity. And the larger variance was in the long pulses, not the short ones.

I see (from the commit's message) that actually the problem was caused by ringing after the high-pass frequency and overflow (I guess those were the spurious undue zero-crossings).

I think the emulator is way better now. I thought at the beginning that it was a problem of the filter, but it turned out to be a sampling problem (of the emulator). It can't happen in a real machine, since you can sample at any time and you'll always get a sample. But it's no longer true with a signal which is already digital. Interpolating to get a sample at any time is sort of turning the signal continuous.

It's closer to what happens inside a real MSX and it's correct signal processing.
I think the problem is quite solved now Smile

By pgimeno

Resident (58)

pgimeno's picture

21-12-2019, 01:54

Agreed Smile

And Manuel reported on IRC that CD Sequential also works now, so this interpolation has solved quite some problems.

Apologies to Takamichi for the blatant thread hijacking.

Page 5/6
1 | 2 | 3 | 4 | | 6