Experts of tape recording ( formats )

[gilesgoat]

So "late nights thinking" ..

Tape recordings .. for computers .. there's been quite a few .. as far as I know they all ( ?? ) more or less end up in being a thing like :

Code: Select all

! 00000011 ! 00001111 ! 
+   bit cell  +   bit cell  + 

Where the "bit cell" is also "the max bit rate" and if there's a mayority of '0' is usually '0' otherwise is '1' or something like that.

Fundamentally "measuring the duty cycle of a square wave".

Now "all I know" about stuff like that is :

- tape can slow/speed a bit ( so not having constant speed ) so you need some "slack"
- you must have some "AC wave" to record as you can NOT record a constant stream of '1' or '0' ( DC )
- you need some "way to know" you got a bit , I am guessing or you assume the bit is long 'x' or you count the '0's+'1's until you get 'N'
- you need some way to "synch" on the incoming data
- you can't have "too many flips of 0/1" in a row

The question is "is there any OPTIMAL data recording format for tape where you can't do better than that ?"

The other question is "suppose you want to push even more data per tape" .. is QAM modulation "the ultimate answer" or there's else that can be done ?

The last question is "has anyone here ever implemented/used REED SOLOMON coding or stuff like that ?"

Also, given that stereo tapes recorders existed since long time .. has anyone ever tried/implemented a data save/load using both separate tracks to double
the data capacity/rate ( i.e. save one bit on channel L and one bit on channel R ) ?

Has anyone ever studied the subject ?

I am sure that DAT tapes and all must have explored the thing but I never found much documentation about.

"Things I am thinking at night" ( sometime even in bed ).

[Kelloggs]

To my shame, I've never dived deeply into tape recording, but that's mainly because, when I reached the point that I had a) spare equipment to wreck and b) a desire to delve deeply, we were well past regular tape use.

One thought strikes me, about your (ingenious, if I may say so) idea to use L and R channels to double the data rate. I don't see a reason why not (maybe crosstalk?) but for the fact that you would then need a stereo read/write head in the data recorder (cost), and (maybe more importantly) the computer in question would have to be able to write two bits to tape at once and also read them back simultaneously. If there's one thing I know about micros that used tape, approximately zero pence was spent on doing so (the most expensive component in the tape subsystem was a relay to start and stop the motor, which is possibly why most didn't have them). I'm most familiar with the Spectrum, and I think that would pretty much need an entire additional audio channel to do this, so I think the short answer would be "lack of 8-bit capabilities". And, of course, cost.

[gilesgoat]

I am TEMPTED to "add a small circuit to my Z80 beast" so it can write/read a STEREO tape and see where it gets.

[KlaxonCow]

You could even quadruple it, by using both left and right channels on both sides of the tape.

I had a "4 track" back in the day, and that's basically how those things actually worked.

The four tracks were actually, physically on the tape, just side A left, side A right, side B left, side B right of an ordinary stereo cassette. So you could record up to 4 inputs onto the tape, and then it had pan and volume controls for each "track" to take those 4 mono tracks and mix them however you wanted in the stereo field.

So though a track was stored on, say, "side B left", you could turn the dials and mix these 4 separate tracks into a different stereo mix and then record that output to a normal stereo cassette.

(And you could do all manner of "bouncing" tricks - take 4 mono tracks and bounce them down to a stereo mix. Do that twice and you've got two stereo mixes that you can bounce down to one. And so on and so forth. By juggling the tracks and "bouncing" them down to fewer tracks, you could just keep layering things on. And this technique has pedigree - while the Beatles were using the big fancy expensive studio version of a 4 track recorder, that's how they recorded Sgt.Pepper's. And Queen were doing all manner of "bouncing" for the "choral" aspects of Bohemian Rhapsody. By the '80s / '90s, though, I could get a cheap portable 4 track that used ordinary tapes and was just a glorified cassette player.)

It's like the cheapest possible multi-track recorder. But it did work, and many music demos - even the odd famous album (Bruce Spingsteen recorded "Nebraska" on a 4 track) - were actually recorded with one.

And the theory of how they work is pretty much what you're thinking here, except you're thinking data, while this was for having multiple music tracks.

But, yeah, taking inspiration from those 4 tracks, you could potentially double the data rate again, by simply reading / writing both sides of the tape at the same time - as well as stereo left and right for both. This does sacrifice the other side, of course. A 4 track tape couldn't be turned over (well, it could, but it'd be backwards and the tracks in reverse order).

And, to do this, you'd need a read / write head that can actually read / write both sides.

But the concept has pedigree. That's how the old 4 track multi-track recorders actually worked. Instead of two stereo mixes on either side of the tape, it was treated as 4 separate mono tracks - and then, as I say, there were dials for panning and volume controls each track separately. And you could record to individual tracks as well, to build up your song one instrument / track at a time and then "bounce" them down to a stereo mix.

It's more elaborate, but you could get a read / write head that can read both sides at the same time (I also had a Walkman-like portable cassette player that could do this - allowing you to "flip sides" on a tape with a press of a button, as it just swapped to the other heads and then reversed the direction of the tape. So I'm thinking that these "dual" read / write heads are probably pretty common, if you know where to look for them). Use both channels of the stereo mix on both sides...

...et voila! You're reading / writing 4 bits at a time.

[Rhythm]

Why stop at 4 tracks?

[Kelloggs]

I did have an idea, a long time ago (mid nineties would be about right) about deriving a timecode from the data on a Red Book audio CD in a CD-ROM drive (P and Q subchannels?) and a separate cue file that, at certain points in a track, could instruct a piece of software to do... something. Specifically, I was doing some lightsynth / sound-to-light things, and it would have been nice to sync up on-screen occurences to specific points within the music. I suppose you could use that for a similar idea to Automata's Deus Ex Machina, but the code doesn't need to be written to make assumptions about where the tape is up to (because the timing is available directly).

But, another of life's great ideas that I never looked in to...

[Rhythm]

I did have an idea, a long time ago (mid nineties would be about right) about deriving a timecode from the data on a Red Book audio CD in a CD-ROM drive (P and Q subchannels?) and a separate cue file that, at certain points in a track, could instruct a piece of software to do... something. Specifically, I was doing some lightsynth / sound-to-light things, and it would have been nice to sync up on-screen occurences to specific points within the music. I suppose you could use that for a similar idea to Automata's Deus Ex Machina, but the code doesn't need to be written to make assumptions about where the tape is up to (because the timing is available directly).

But, another of life's great ideas that I never looked in to...

I've often pondered about that kind of thing, but relating to Ambilight (and similar A/V lighting solutions). Ambilight normally checks the edge of the screen to extend the light past the screen. It's a cool effect but I'd love to see it respond to (hidden) cues so it could do things like, in a horror film, putting shadows in motion before you see the action on the screen

[Dio]

I've got a white paper on all this that I should polish up.

Basically, yes, with a bit more effort on HW and SW you could pretty certainly have delivered 10kbit/s per head formats, and multiple heads would have allowed transfer rates to stretch up into the 20-40kbit range.

I've managed to load Lords of Midnight from tape in about 45 seconds on a real Spectrum, but not terribly reliably (and using a computer to generate the signal rather than actual tape, although I think that would be a reasonably small effect). If the Spectrum had a slightly better tape ADC (the one on the Harlequin is awesome) then higher rates would be possible, but the Spectrum interface isn't particularly DC-bias tolerant, and that tends to mess up really high efficiency coding schemes like MFM and monopulse techniques.

Here's the table of techniques I've tried, which unfortunately paste has turned into a bit of a mess:

Code: Select all

Technique	Peak rate	Reliability	Adv. Coding	DC bias	Decoder complexity	Notes
PDK	3kbit/s	Excellent	No	None	Trivial	
FM	5+kbit/s	Excellent	No	None	Trivial	Constant bitrate
																		Exploitable dead zone
																		Can work at very high symbol rates, probably with degraded reliability
Narrowband PDK	3.5-4kbit/s	Excellent	No	None	Trivial	
Monopulse PDK	6-8kbit/s	Unclear	No	Bounded	Trivial	
Multiband PDK	6-12 kbit/s	Good / Excellent	Unbounded	None	Complex	
Multiband monopulse PDK	Up to 20kbit/s	Unclear / limited	Yes	Yes	Complex	Requires DC correction
PWM	4kbit/s	Good	No	Bounded	Simple	Constant bitrate
																		Unpleasant audio
Multiband PWM	10-15 kbit/s	OK?	Yes	Bounded	Very complex	

The estimates of peak rate are very much "if it actually worked" which not all of them do on real Spectrum hardware. In practical use, high rate FM appears the most bombproof technique on an actual Spectrum, and I managed to do a proof of concept that an FM loader could play Space Invaders while it was loading since the dead zone when you can do other things is massive.

Next time I come back to it I'll probably see if I can make multiband monopulse work on the Harlequin and see if a better input circuit would have made a major difference back in the day.

If you're wanting to do any research on this, I have a whole bunch of utilities I could share - frequency / pulse width analysers, python scripts to generate WAV files, that sort of thing.

As for advanced coding, I'm fairly sure an at-the-time Z80 doesn't have the throughput for Reed-Solomon or other error correction methods. Even moderately complex RLL codes are very much a stretch. I think MFM is just practical, but I haven't written the code yet.

[Dio]

Oh, and to answer your earlier question: in terms of writing an FM decoder, it's really simple. FM encodes everything as 11 (two state changes) or 10 (one state change).

- Find a clock, by waiting for the port to change state
- Wait 3/4 of the expected time between clocks
- If the state has changed, the bit is a 1, otherwise the bit is an 0
- Repeat

You need to sync up by tracking at least a few 10 pulses beforehand, and a longer pilot tone of 3-5 seconds is usually required to lock on and solve AGC issues (that applies to all formats, not just FM).

PDK (the standard Spectrum format) is also easy:
- Accurately count the time between two state changes on the input port.
- If this is > some threshold, the bit is a 1, otherwise it is an 0.

The big simplicity advantage with FM decoders is that the only accurately timed bit is the wait from clock to sample and it's fairly tolerant of error (although getting it exactly right does help reliability quite a lot). With a PDK loader you have to absolutely accurately count every cycle without fail along every possible path in the routine.

[Dio]

I found my notes on 4-head FM. Here I think it's possible to transfer 7 or 8 bits per code:
- 4 bits from the 4 FM streams
- 3-4 bits from having only one head's clock pulse absolutely fixed in time. The other three streams can have the clock pulse 30-50us early or late. There needs to be a little trickery to find the one extra bit here.

Operated at 5kHz base rate (which is solidly reliable on a standard Spectrum) this would offer 40kbit/s transfers and is basically about the same as disk transfer on 8-bit systems. The decoder is pretty trivial too and would retain most of the 75% dead zone for doing other things. You can probably push this scheme up to 60kbit with the right hardware, but once you can load an entire game in 20 seconds it's better to focus on reliability I suspect.

[gilesgoat]

Oh, and to answer your earlier question: in terms of writing an FM decoder, it's really simple. FM encodes everything as 11 (two state changes) or 10 (one state change).

- Find a clock, by waiting for the port to change state
- Wait 3/4 of the expected time between clocks
- If the state has changed, the bit is a 1, otherwise the bit is an 0
- Repeat

You need to sync up by tracking at least a few 10 pulses beforehand, and a longer pilot tone of 3-5 seconds is usually required to lock on and solve AGC issues (that applies to all formats, not just FM).

PDK (the standard Spectrum format) is also easy:
- Accurately count the time between two state changes on the input port.
- If this is > some threshold, the bit is a 1, otherwise it is an 0.

The big simplicity advantage with FM decoders is that the only accurately timed bit is the wait from clock to sample and it's fairly tolerant of error (although getting it exactly right does help reliability quite a lot). With a PDK loader you have to absolutely accurately count every cycle without fail along every possible path in the routine.

Intriguing .. I'd like to know more details about this FM modulation, what do you mean about '11' and '10' ?

I may set up some "stupid HW" at some point to make some experiments.

[Dio]

When talking pure digital magnetic storage, 1 means a state reversal and 0 means no state reversal. So 111111... is double the frequency of 10101010...

https://en.wikipedia.org/wiki/Modified_ ... modulation is a good starting point for reading about (constant rate) coding, then https://en.wikipedia.org/wiki/Run-length_limited is a bit more advanced.

The formula for the throughput of a pure digital channel is then code_rate = bits_per_code * distance_between_1s / code_length. So for FM on tape this is 1 * 1 / 2 or 0.5; multiply that by 2x the max frequency on the channel (say 2kHz-8kHz) and you get rates in the 2-8kbit/s range. MFM is twice as efficient with a code rate of 1 * 2 / 2 == 1 but it's not DC-free.

[gilesgoat]

When talking pure digital magnetic storage, 1 means a state reversal and 0 means no state reversal. So 111111... is double the frequency of 10101010...

https://en.wikipedia.org/wiki/Modified_ ... modulation is a good starting point for reading about (constant rate) coding, then https://en.wikipedia.org/wiki/Run-length_limited is a bit more advanced.

The formula for the throughput of a pure digital channel is then code_rate = bits_per_code * distance_between_1s / code_length. So for FM on tape this is 1 * 1 / 2 or 0.5; multiply that by 2x the max frequency on the channel (say 2kHz-8kHz) and you get rates in the 2-8kbit/s range. MFM is twice as efficient with a code rate of 1 * 2 / 2 == 1 but it's not DC-free.

You made me spent 15 mins with paper and pencil thinking ..

So if I get it right .. the algorithm for your FM should be like this :

For "EVERY BIT CELL" ( i.e. 1 bit time ) let's ASSUME WE START with low, '1' is encoded as "01 01", '0' is encoded as "0111" i.e. 2 changes of state is '1', 1 change of state is '0'.

As you say the algorithm seems to me ( and seems to be working in all cases ) :

1. state A = read INPUT
2. state B = read INPUT
3. if ( state B == state A) GOTO 2

Now you found a clock transition

4. WAIT 3/4 (BIT TIME/2)
5. state A = read INPUT
6. if ( State A <> state B ) bit is '1' otherwise bit is '0'
7. WAIT 5/4 (BIT TIME/2)
8. State B = read input
9. GOTO 4

If I reasoned correctly IT DOES NOT MATTER if it starts synching on a 0 to 1 OR a 1 to 0 transition, the result will be the same.

Yes understood you need some "header" of pulses to start sincronizing but that seems also as you set to allow the AGC to settle to some level.

But now I am wondering, you say 5kbps .. this means you are recording a frequency of around 10Khz ? Isn't that a bit "much" for tape ?

[edit] - can't think too much now, I am not sure if 9 has to be a goto 4 or goto 5 ...

[NorthWay]

The C= 64 folks have pondered a bit on speed theory, and I think the csdb.dk forum talk saw 8Kbit/s as possible from a standard 1530.
(Tangentially, we have discussed MFM floppy density/encoding over at EAB and how you can be creative with valid encodings.)

I was kinda disappointed when I read that the commercial tape duplicators couldn't support tape speeds faster that sth like 2600(?)bit/s which meant that tape loaders were slowed down to be compatible.

[Dio]

That sounds like they defined a 4kHz tone limit, because 2.6kbit is about the rate you get from (evenly biased) PDK using 2 and 4kHz tones, and that was indeed the maximum speed format you ever saw in the wild on the Spectrum. It could be very reliable (Travels With Trashman never gave me any trouble) or a f***ing nightmare (Doomdark's Revenge would only load on the right phase of the moon).

But that's just equal interval PDK, which is code-inefficient - staying within the 4kHz limit you can do a lot better than that with better coding. Beyond that, I can't see any reason why any DC-free method isn't acceptable on any audio equipment. Although I bet that there would have been a lot of dubious sucking-in of air through teeth if I walked in with some of my wackier formats and lots of mumbling about not replacing any of these tapes if there are any problems...

As I understand it the C64 is a different beast to the Spectrum: although it has a more digital tape processing path, which helps, I believe it can only detect one edge rather than both positive and negative transitions, which means it can't use FM, PWM or monopulse methods. I may however have misunderstood. I can believe 8kbit/s is just about viable using PDK with say 8.5 and 7.5kHz tones though - if you can get a signal that high frequency onto the tape.

But now I am wondering, you say 5kbps .. this means you are recording a frequency of around 10Khz ? Isn't that a bit "much" for tape ?

No - one FM bit is encoded in either two transitions (i.e. one squarewave cycle at the higher frequency) or one transition and no transition (half a cycle at the lower frequency).

For your code, don't do a wait after you sample the bit. Just go back to the "find a clock pulse" routine and then it's resynchronised for the next pulse.