Show us yer programming projects!

Coders forum. Discuss techniques, swap code snippets, ask for help etc. relating to all forms of software development.
Post Reply
User avatar
gilesgoat
Sheepie Whore
Sheepie Whore
Posts: 27259
Joined: Wed Sep 03, 2003 7:54 am 0
Location: Monfalcone, Italy
Contact:

Re: Show us yer programming projects!

Post by gilesgoat »

Had to be done ...

Code: Select all

Mandelbrot
""""""""""""""##############$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$#############
""""""""""""##########$$$$$$$$$$$$$$$$$$$$$$$$$%%%%%%%&')+(&&&%%%%%$$$$$$#######
""""""""""#######$$$$$$$$$$$$$$$$$$$$$$$$$%%%%%%%%%&&&&'(* ,+ *'&%%%%%%%$$$$$$##
""""""""######$$$$$$$$$$$$$$$$$$$$$$$$%%%%%%%%%%&&&&&') +.   +)('&&&%%%%%%$$$$$$
"""""""####$$$$$$$$$$$$$$$$$$$$$$$%%%%%%%%%%&&&''''(()+2       +)('&&&&&&%%%%$$$
""""""##$$$$$$$$$$$$$$$$$$$$$$%%%%%%%%&&&&'*./.,**+.,-.0      /-,+-+)((((+)&%%$$
"""""#$$$$$$$$$$$$$$$$$$$$%%%%%&&&&&&&&'''()+                        /     (&&%%
""""#$$$$$$$$$$$$$$$%%%%&&&&&&&&&&&''''((  31                           /+)('&&%
"""#$$$$$$$%%%%%%%%&&'')0*)(())3*)((())),                                  /-'&%
"""$$$%%%%%%%%%&&&&&&'((*+3         3,,-0                                  +)'&%
""$%%%%%%%%%&&&&&''.)**+.                                                  -'&&%
""%&&'''()((''())**2/                                                    +(''&&%
""%&&'''()((''())**2/                                                    +(''&&%
""$%%%%%%%%%&&&&&''.)**+.                                                  -'&&%
"""$$$%%%%%%%%%&&&&&&'((*+3         3,,-0                                  +)'&%
"""#$$$$$$$%%%%%%%%&&'')0*)(())3*)((())),                                  /-'&%
""""#$$$$$$$$$$$$$$$%%%%&&&&&&&&&&&''''((  31                           /+)('&&%
"""""#$$$$$$$$$$$$$$$$$$$$%%%%%&&&&&&&&'''()+                        /     (&&%%
""""""##$$$$$$$$$$$$$$$$$$$$$$%%%%%%%%&&&&'*./.,**+.,-.0      /-,+-+)((((+)&%%$$
"""""""####$$$$$$$$$$$$$$$$$$$$$$$%%%%%%%%%%&&&''''(()+2       +)('&&&&&&%%%%$$$
""""""""######$$$$$$$$$$$$$$$$$$$$$$$$%%%%%%%%%%&&&&&') +.   +)('&&&%%%%%%$$$$$$
""""""""""#######$$$$$$$$$$$$$$$$$$$$$$$$$%%%%%%%%%&&&&'(* ,+ *'&%%%%%%%$$$$$$##
""""""""""""##########$$$$$$$$$$$$$$$$$$$$$$$$$%%%%%%%&')+(&&&%%%%%$$$$$$#######
""""""""""""""##############$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$#############
Giles the billy goat
"Baa"
User avatar
gravy
Sheepie Whore
Sheepie Whore
Posts: 19572
Joined: Wed May 01, 2002 1:28 pm 0
Location: The Other Place

Re: Show us yer programming projects!

Post by gravy »

chicane
Posts: 5
Joined: Sat Jan 04, 2014 12:01 pm 0

remake of Pole Position using reverse engineering

Post by chicane »

Hi everyone - I've been lurking around this forum for years but have only felt the urge to post for the first time this morning. My contribution to the thread is my work-in-progress remake of the Pole Position arcade game. Like many other remakes, it features snazzy updated graphics, but the real standout feature is (in my opinion at least) the fact that the game mechanics (timing, driving model, opponent AI) have been derived by reverse engineering the Z8002 code in the original arcade rom. The game should therefore play exactly like the arcade original, but look and sound a bit nicer.

It's written in a combination of C and C++, and uses the OSG and Gorilla Audio libraries. A video speaks a thousand words, so here's one I made earlier. The video first shows the demo sequence (which unsurprisingly bears a creepy resemblance to that of the arcade game). After that, watch out for the ~10 seconds of blank screen (due to be replaced by the title screen), after which there's a full qualifying and race sequence.

User avatar
Istar
Baaaaaa!
Baaaaaa!
Posts: 4869
Joined: Mon Apr 22, 2002 4:42 pm 0
Location: London, UK.

Re: Show us yer programming projects!

Post by Istar »

Cool. Why does it say "TIOE" instead of "TIME"?
chicane
Posts: 5
Joined: Sat Jan 04, 2014 12:01 pm 0

Re: Show us yer programming projects!

Post by chicane »

Istar wrote:Cool. Why does it say "TIOE" instead of "TIME"?
Ha ha - no flies on the people on this forum! It's very much a work in progress at the mo - clearly i've made a mistake in rewriting the z8002 code somewhere such that the letter I of TIME gets overwritten with what looks like the zero of a score!

I'm mostly done with implementing the 'big picture' of this project so i'll be free to focus on the more granular details (like this) in a couple of weeks time.
User avatar
Mark
Baaaaaa!
Baaaaaa!
Posts: 3611
Joined: Mon Apr 22, 2002 7:53 pm 0
Location: England

Re: Show us yer programming projects!

Post by Mark »

Love the look of that - it's so crisp. The cars look especially good.

My fondest memory of Pole Position is that the wheel of the machine was free spinning, so I'd go full tilt into corners and give the wheel a good spin to steer as sharply as possible, and then catch it to straighten up again afterwards.
Benevolent Dictator of Bearcat Games
Twitter: #BearcatGames
User avatar
Firefox
Sheepie Whore
Sheepie Whore
Posts: 11412
Joined: Fri Apr 26, 2002 6:15 pm 0
Location: Sheffield

Re: Show us yer programming projects!

Post by Firefox »

Welcome in. :)

Really love the concept of authentic gameplay with updated graphics! It looks really good in the video.
"I am a very considerate person. If I want to go out all night on my miniature golf course and shoot lizards, well, no reason to keep the neighbours awake..."
User avatar
failrate
Baaaaaa!
Baaaaaa!
Posts: 2354
Joined: Sun May 23, 2004 6:54 am 0
Location: Sandy Eggo
Contact:

Re: Show us yer programming projects!

Post by failrate »

@chicane, that looks really nice! Is it going to be a faithful implementation to the end, or are you going to enhance it (ex, track editor, multiplayer, ...)?
chicane
Posts: 5
Joined: Sat Jan 04, 2014 12:01 pm 0

Re: Show us yer programming projects!

Post by chicane »

Thanks everybody for your kind words.

@failrate - my first priority is to replicate the game experience of the arcade machine in full (i.e. demo, qualifying and race). After that I'm hoping that I'll have a sufficient understanding of the game logic to add some new features. Interestingly, the first two additional features I had in mind were indeed a track editor and split screen multiplayer (and possibly network at a later date). The former should be reasonably simple, but the latter is likely to be a lot more difficult and time-consuming. It's a real shame to me that split-screen multiplayer has pretty much disappeared in the modern era of gaming so I'd love the opportunity to help a new generation of people discover the joy of punching the person next to you when they knock you off the track!
User avatar
gilesgoat
Sheepie Whore
Sheepie Whore
Posts: 27259
Joined: Wed Sep 03, 2003 7:54 am 0
Location: Monfalcone, Italy
Contact:

Re: Show us yer programming projects!

Post by gilesgoat »

Wow I had a quick look, for pure curiosity, yesterday at the technical manual of Pole Position.

I did not know ( well, I did know nothing about it really ) it actually has two Z8002 CPUs and I got quite a bit impressed how they seems to have almost "a bit of hw
for everything", like "background" is some HW to does it, the road is another bit of HW that does it, the signs is another bit, your car is a bit, the other cars is another bit ...

I think must have been a heck of a nightmare to understand that code, I'd be curious to know a little bit about "how you actually started analyzing something like that" because I think you can't really get anywhere without a quite deep understanding of that HW.

Just curious, did you have something allowing you someway to replace/upload rom code in the original board "and see what happens ?" or did you do it all via
someway else ?

How many months/years have you been working on this ? :)
Giles the billy goat
"Baa"
User avatar
GoatFoam
Baaaaa!
Baaaaa!
Posts: 1255
Joined: Tue Dec 12, 2006 11:05 am 0
Location: Brizzle
Contact:

Re: Show us yer programming projects!

Post by GoatFoam »

A little project I have been working on.

Apologies for the horrid web page. I couldn't find any of my good datasets & really need some nice piccies to tart it up. Also I threw it together in about zero minutes.

http://www.spectral3d.co.uk/Vapourware/star_trails/
chicane
Posts: 5
Joined: Sat Jan 04, 2014 12:01 pm 0

Re: Show us yer programming projects!

Post by chicane »

@gilesgoat, thanks for your interest in the project.

I hope I've not given the wrong impression by implying that I did some hardcore tinkering with the actual arcade hardware in order to get to this point! Much as I like the idea of reviewing technical manuals, burning replacement ROM chips and so on, the truth is that I'm a bit crap when it comes to hardware and the only source material I worked with was dumps of the arcade roms (i.e. the files you'd use to play a game in MAME). I too, had a look at the Pole Position schematics but they might as well have been in Greek.

Anyway, i digress. I started the project in April 2013 and I've been working on it for a couple of days on most days to the present. My impetus for starting the project was clear - my first attempt at a Pole Position remake ("Grid Leader") was a bit disappointing - I was pleased with the graphical style (which has a lot in common with this newer project) but the gameplay and game logic was derived through observation of the arcade game rather than an understanding of the internal game mechanics. In short, Grid Leader looked and sounded nice but wasn't that satisfying to play.

I started off by observing that the arcade hardware was running on one Z80 and two Z8002s (as you've observed), and imagined that the game logic was likely to be driven by one or more of these three chips. Having identified the ROM holding the Z80 code through examination of the MAME source code, I had a poke around using commonly available Z80 disassembly tools and came to the conclusion that there was little or no game logic in the Z80 code. In fact, to this day, I'm not sure what the Z80 in Pole Position does - I can only imagine it's there to help coordinate the activities of all the other hardware in the machine. While looking around this ROM, I tried inserting NOPs in various places to comment out subroutines, but couldn't seem to make any discernible changes to the game logic. In fact, any changes I made would either break the game completely, or make no difference at all.

I moved onto the Z8002 after a couple of days and stumbled upon a rather unfortunate reality - there are (to my knowledge) no publicly available tools to disassemble Z8000/Z8002 code. But at this point, I was reasonably convinced that the game logic lay in the Z8002 ROMs, so I had no choice but to work out some way to disassemble the two Z8002 ROM files in order to progress. Again, with the help of MAME, I was able to establish that the Z8002 ROMs were four 8K files, two of which belong to the first Z8002 ('sub') and the other two belonging to the second ('sub2'). So each of the Z8002s has its own 16K block of code to execute, with the code being stored in a slightly peculiar but thankfully uncomplicated interleaved format within the ROM files.

If you've done any disassembly work in the past, you'll know that one of the major challenges lies in working out which parts of a ROM or memory dump represent code, and which parts represent data. Fortunately, MAME's debugger features a 'trace' function, where you can dump the instruction pointer address along with the instruction at that address as the emulated CPU executes it. Which, once you've picked out all the unique lines (using good old Linux command line tools like 'sort' and 'uniq'), will give you a list of addresses within the ROM at which instructions lie, along with the actual instructions at those addresses. Having played through a game of Pole Position within a build of MAME with some subtle modifications, I had such a list at my disposal (for the 'sub' CPU) and quickly knocked together a PHP script to combine this list with the ROM file, writing out instructions at instruction addresses, and '.byte' directives at addresses where no instruction was found.

I was keen at this point to understand whether my chosen approach was anything approaching correct. In order to establish this, I needed a tool that would take the disassembly generated by my PHP script, and assemble it back into the original ROM file. Open source software came to the rescue once again - a couple of hours digging around the net and I had a functioning Z8002 cross-assembler running courtesy of the GNU binutils project. The first run of the assembler against my assembly source file, rather disapppointingly, resulted in thousands of errors. Thankfully, the cause of these errors was merely a couple of syntactical differences between the instruction format generated by MAME and that expected by the assembler. A bit of tactical hacking on the PHP script, and the instructions generated in MAME were being converted into the format expected by the cross assembler. I managed to get a successful run of the assembler, wrote some further tools to convert the generated binary file back into the interleaved ROM format, and replaced the original Z8002 ROM files in my MAME roms directory with the ones I'd just generated. I ran Pole Position in MAME, expecting something unpleasant to happen, but amazingly the game ran as normal! Over time, I played several games of Pole Position, trying to execute as much of the codebase as possible by performing a wide variety of actions (e.g. crashing into other cars, running out of time, finishing the race with time to spare, getting a high score, not getting a high score...). As I did so, the size of the instruction map grew along with the number of instructions in the generated assembly file.

This was quite a major milestone in the project - I was equipped with a (somewhat flaky) set of tools to disassemble the Z8002 ROMs to generate an assembly listing, and reassemble this assembly listing back into the ROM file. I made a series of changes to these tools (which would become known as 'ppda' over time) to improve the legibility of the generated assembly file, using a series of text files to (amongst other things) insert labels, replace addresses with variable names and replace lists of .byte directives with ASCII strings:

Code: Select all

QUALIFYING_NUMBER_IS_WHITE: ! LOC_1F70
    ldb     rh1,#0x01

DRAW_QUALIFYING_NUMBER: ! LOC_1F72
    ld      @r12,r1                      ! write the current qualifying number to the screen
    inc     r12,#4                       ! move 2 characters to the right
    incb    rl1,#1                       ! move onto the next qualifying number
    djnz    r0,DETERMINE_QUALIFYING_NUMBER_COLOUR ! go and display another number if we're not done yet
    jr      DRAW_QUALIFYING_BONUS

DRAW_POLE_POSITION_MESSAGE: ! LOC_1F7C
    ld      r12,#0x9e12
    call    DISPLAY_MESSAGE_CONTROL_IN_RH0
    .ascii "POLE POSITION"
    .byte 0x21
    .byte 0x40
    .byte 0xff

DRAW_QUALIFYING_BONUS: ! LOC_1F94
    ldb     rh0,rl6
    ld      r12,#0x9e98
    call    DISPLAY_MESSAGE_CONTROL_IN_RH0
    .ascii "BONUS"
It's a bit hard to describe how the process of unravelling and documenting the assembly listing happens in reality - it's a very organic and unstructured process. Often, it was a case of randomly scrolling through the latest version of the disassembly listing, stumbling upon something that looks interesting, and then adding comments/labels or replacing addresses with best guesses of variable names. It very rarely becomes stale, because most discoveries (however small) will have knock-on effects elsewhere in the source file that open up other avenues of investigation. Sometimes I did get stuck for a couple of days and lose the will to carry on, only to discover a new avenue of investigation next time I opened the disassembly. As time went on, I gained more of a high-level view of the code - I began to see execution patterns and regularly-used data structures amongst the sea of instructions.

One technique that came in useful (which I understand is quite commonly used in this type of task) was making modifications to the disassembled code, reassembling and then running within MAME to understand the impact of those changes. For example, if I wanted to know what the subroutine at address 0x1234 was doing without actually reading through it, I could place the first instruction in that subroutine with a 'ret' instruction (being careful to pad out with NOP instructions as appropriate) to disable the execution of that subroutine. Subsequently executing Pole Position in MAME using the modified code would hopefully result in some obvious change in behaviour that would help to identify the purpose of the modified subroutine. Unfortunately, my initial efforts at making use of this approach were hampered by the checksum code in the ROMs - the emulated Pole Position machine was detecting that it's own ROMs were malfunctioning! It took a couple of days to work out where the checksum code was and how to bypass it and I was once again on my way.

So over the course of 3-4 months, I established the role of each Z8002 and gained an understanding of the relationship between them. Broadly speaking, the roles of each CPU are as follows:

[*]The first Z80002 is responsible for maintaining the game state, updating scores, moving the player car in accordance with controller input and communicating witthe graphics hardware[/*]
[*]The second Z8002 is responsible for moving the AI cars around the track. It can also execute one of a series of numbered operations at the request of sub1, by means of a function queue held in shared memory. For example, function number 0x1 lays the cars out on the starting grid[/*]

I'd found the game logic code much more readable (and interesting) than the code to communicate with the graphics hardware. It was clear by this point in time that trying to unravel the graphics hardware management code was going to be a pain due to the code complexity resulting from limitations in the (admittedly impressive for the time) graphics hardware. For example, AI cars beyond a certain distance are drawn at low detail using two sprites, whereas cars close to the camera are drawn at a higher level of details using 6(?) sprites - the logic for spreading a single car across 2 or 6 sprites along with the code to choose between these two approaches is all embedded in the code in a not so readable fashion. I was questioning the benefit of making sense of this graphics hardware management code at this point in time, and this was part of the reason I chose to focus my efforts on building a brand-new 3D scene graph on top of the existing game logic.

I'll try to write some more on the next steps (conversion of the assembly code to C and the challenges of mapping the game logic to a 'true' 3D environment) at a later point, but this should be enough to go with for now! I'm aware that I've probably skimmed over some important matters so feel free to fire any questions over.
User avatar
Muttley
Sheepie Whore
Sheepie Whore
Posts: 13169
Joined: Thu Apr 25, 2002 12:30 pm 0
XBL Gamertag: Mu77ley
Location: In a maze of twisty passages, all alike.
Contact:

Re: Show us yer programming projects!

Post by Muttley »

@chicane Brilliant stuff chicane, I love stuff like this. See also CannonBall https://github.com/djyt/cannonball/wiki
Muttley's Blog: Rarely updated, occasionally interesting… maybe
RetroRemakes Framework: Open Source BlitzMax Game Framework
MuttMod: My Open Source BlitzMax Modules
User avatar
gravy
Sheepie Whore
Sheepie Whore
Posts: 19572
Joined: Wed May 01, 2002 1:28 pm 0
Location: The Other Place

Re: Show us yer programming projects!

Post by gravy »

Yeah great post and lovely project :) You should blog it, it's fascinating stuff
User avatar
gilesgoat
Sheepie Whore
Sheepie Whore
Posts: 27259
Joined: Wed Sep 03, 2003 7:54 am 0
Location: Monfalcone, Italy
Contact:

Re: Show us yer programming projects!

Post by gilesgoat »

I started off by observing that the arcade hardware was running on one Z80 and two Z8002s (as you've observed), and imagined that the game logic was likely to be driven by one or more of these three chips. Having identified the ROM holding the Z80 code through examination of the MAME source code, I had a poke around using commonly available Z80 disassembly tools and came to the conclusion that there was little or no game logic in the Z80 code. In fact, to this day, I'm not sure what the Z80 in Pole Position does
A very quick look at the Pole Position Technical Manual clearly in the schematics calls the Z80 as "Sound Microprocessor", the sounds too seems to be split in a few
modules, what is weird it seems ( never seen it made as that ) the Z80 drives a set of CMOS switches and some resistors that creates 4 x 4 bits D/A channels to
control the "gain" of something else.

There's some "sound memory" connected to it but it seems the sounds are actually generated by some custom PAL/chip, there's a "voice memory" as well.

There's a specific bit of HW called "Engine Sound Generator" that looks like it's dedicated only for that.

As I said I never seen before an HW so "modular" where "every single thing seems to have its own HW bit".
Giles the billy goat
"Baa"
Post Reply