Transport Tycoon Forums

Hi

I search a network programmer for windows to write a client/server system for ttdpatch to test various ideas and maybe someday have a stable internet multiplayer with TTDPatch.

Instead working in evil assembler you have to work in plain c which can be compiled with gcc mingw tools.


I need a server component application:
- Needs to handle up to 8 connects,
- Slave mode for a different main server.
- Send Data to the client dll and checks for data integrity.
- Buffers Data for a slow clients. (Latency)

A client dll component:
- gets loaded TTDPatch
- has to do the server connection
- handles network io to the server.
- does timestamping in the future of data packes
- has a buffers


If you think you have the necessary knowledge and want to help TTDPatch drop a line.

Thanks

-----
OpenTTD Code is not wanted... comments suggestion this will be removed without comment.

Maybe you guys could get some help from TrueLight, he did the network code of OTTD. TrueLight is retired though so I don't know if he wants to do it for TTDPatch too.

(Is this also suggesting to use OTTD code or not?).

Oskar, could you provide some more information on this?
What exactly are the limitations on the current method inside TTD?

I've written a few C++ TCP Client/Server applications... I'd imagine both client and server functionality be built into the one dll (if not incorporated into ttdpatch.dll) and then TTD init either the client or server on request.

Sounds pretty straightforward, but I can't see how exporting into a DLL with C/C++ will overcome the issues that TTD currently has?

Well surely there need to be some internal changes, the point useing c with a dll is, we can exactly know what it is doing. Ohh and I think the server component should be an external app that can be run on linux aswell.
(Ohh the gui for entering the ip to the server should be in TTD so we can work with fullscreen...)


My todo text has some more internals explained...
-edit-
If you have more questions you can reach me via email and the usally ways

Status Update.

Since r666 there is a branch for asynchronous multiplayer in TTDPatch (aka a internet multiplayer that works in TTD)

http://svn.ttdpatch.net/trac/browser/branch/asyncmp

So if you like to contribute or see any errors in my slopy C code I write, let me know...


How it should work.

Certain TTD(Patch) functions have to be rewritten to use a callback system. Every Player Action will get transmitted to the server.

At every 27ms the server will collect all data it has got, merge it to a package and send it to the client as heartbeat package, so the clients can advance to the next frame. If a client is in the 27ms wait loop and there is more then one heartbeat waiting, it stops waiting and start to delag it self automatical. Generally there is no need for resyncs, because every TCP package should go to the wire...

A big problem I need code that guarantee that the heartbeat package doesn't get lost somewhere, say the TCP package gets a timeout, on the other hand, the client would see a long time not a heartbeat, or a heartbeat package with wrong counter number it can say it got desynced.


What is a bit different then in any other games:
The timewarp: Not all players will see on time x the map at the same game time. It's servertime + transmit to player time. As the whole concept of it doesn't need to guess when a action should take place (the server does it) nothing really bad can happen...

However the server heartbeat time can be made slower so slow player can resync easier...

With TCP a packet can't timeout, they're always guaranteed to arrive at the application in the order they're sent. Only the whole connection can timeout which would cause the socket read/write to fail, so you know that it happened. I guess in that case you'd show an error message and either quit to the title screen or continue as a single player game.

*Bump*

We still need some kind of server component, if you have experience in writing client server It would be nice if you could help us

C would be good, C++ as server is possible aswell, Perl, Python to help us to get a start too... (However it needs to work on win32 and linux)

First off: I have no clue about programming in C or assembly, but maybe some input from a player's perspective is appreciated.

- Working internet play for TTDPatch would be great. It's the only thing OpenTTD has on the patch if you ask me.
- No matter how rare it is, I find it very annoying when I spent a lot of time on a game for it to desync without possibilities to continue the game multiplayer.
Maybe an option could be presented to the player, allowing him to either save the game at that point and continue the game later, multiplayer, or to continue right there and make it a single player game.
I hope I am making any sense, and not suggesting obvious features.

You can just rejoin to the server that just desynced you and continue in the company that you created.

Network-related code is my main programing interest, and I've spent a fair amount of time writing client/server/semi-p2p in both python (with twisted, so platform independant) and linux C. I've never done Win32 C or WinSock, but I have a fairly good understanding of networking in general, the linux sockets api and twisted. I've expressed interest in multiplayer TTD Patch before on #tycoon, but I seem to have neglected replying in this topic.

In other words, I'm interested.

Ok, very good. I have no further concerns

As I guess changing the code of the server while testing is very likely, a python server (that works on win32 with python) is the best way. However it needs to convert byte streams to meaning full data. Aka you get a bytestream of Data and need to decode necessary data, like a framecounter, change the data slightly and broadcast it back to all clients in a bytestream.
As we need guaranteed Data transfers, we can use TCP.

We need a bit logic in the server.
A map of newgrfs with md5sums as example that can be verified. (this can be done later)

If a new client connects, it needs to send a service package to random/all clients. A client or all sends then a savegame data (which can be quite big) back to the server with a timestamp and it has to be transmitted to the new client, + all the action packages after the time stamp.


I already have started creating a winsock c code for the client side into ttdpatch.dll, so I don't think you need to worry much.

Here is the current structure draft:

We have two types of packages, action packages, and service packages that aren't send back to the clients.

A package looks like this, for each frame timestamp there will be more then one package, the order isn't allowed to be changed when send back!

[uint32 frame timestamp] [counter*] [u8 type] [uint32 lenof data] [data]

* only if we can't gurantee the order, a timestamp, counter order...

Type:
0x01 = 52 bytes action data

Service Package:
0x30 = heartbeat

0x41 = random gen value
0x42 = setplayer
0x43 = newgrfmap with md5sums
0x44 = get sync package
0x45 = sync package (whole savegame data)

0x60 = change setting that affect others, like the gamespeed

When TTD normally would wait 27ms,
a framecounter is increased and waits on the network to recieve the next heartbeat (service) package,

Server creates every 27ms a heartbeat package, all clients wait for it so they can progress to the next frame,
the timeing may be alter to support gamespeed settings. If a clients doesn't get a heartbeat package it will consinders the connection lost after waiting x secs.

The server recieves packages from it's clients, the frame timestamp will be altered so tbey get the next heartbeat framecount timestand and send to all clients (includeing the source).

The server checks the transmitted random gen number with other clients, and looks if they are still in sync,
if a client isn't in sync anymore, it needs to be resynced. A savegame is requested from all clients, (later tested if the same) and transmitted



Planed addtion:
The server should have a way to record the data (for debugging purpose).


Some notes:
Every client will always be as fast as the Server.

Sending packets in the same order to all the clients shouldn't be a problem, as long as the relative ordering of packets from different clients doesn't matter.



These need more details, I know the heartbeat is what tells the client what frame to advance to, but the others need more clarification, I gather they are related to what happens when a new client connects?



From what I've understood, the clients buffer the DoActions, and when they get a hearbeat packet for a frame, they execute all DoActions up to and including that frame?



Again, these seem to be related to the service packets, need more info on those.



We discussed latency issues a long time ago on #tycoon, will the server still perform some magic to help with those?

If the package of one player come into a stream, and not like mixed between Player 1 3 4 2 1 3 then this should be ok.


For the beginning only really the heartbeat and randomgen values are really needed. To advance the game and to look if the games run out of sync. Currently. As we transfer all actions as old TTD we should have a synced game after we connect two players. We should start with some small steps and do the highlevel error fixing later. (We assume that both TTD(Patch) are the same, and that we keep the player settings until the TTDPatch core works async enough-


The hearbeat package is the clock signal. As OTTD do, we still think both games work in sync when there is no userinteraction. The userinteraction is passed to the network completly and not stored. Some frame later it will return on all machines the same frametime. (The differene here, we don't wait for the player action to really happen as TTD does, but the result will be somewhere in the future)

Yes, this still has some latency problem. Actually I assume that the latency doesn't change much between packages, so I assume if package needs 100ms to travel, the next won't be that different and need aswell 100ms +-27ms.
Then you have have a steady time offset to your server and the UI keeps responsive. Only all players work with a past TTD Version. To be exact their client will be always one traveltime of the heartbeat package away. And their actions, (build an rail as example) will need exactly transmit time x 2 to do really something in the game. But aswell OTTD can't change physic. A player action (you build something) have to be reached all clients before the timestamp to really do the action has been reached. This means a OTTD action needs to be [traveltime to server] + [slowest possible time to send from client to server] + [fixed value to compensate latency issues] in the future.




The game runs clocked by the server. While you will see latency differency changes outside the 27ms the game should run normally.
(As I said, it works by assumeing that your connection isn't highly changeing the latency.)

Heartbeat at Client 1: <##latency 1##> 1 2 3 4 5 6 
>------------------------------------- Timeline
Heartbeat at Client 2: <###latency 2###> 1 2 3 4 5 6 

As soon as latency changes, say Client 2 gets a lower latency, you will have one frame or more that are shorter as usally, until it has resolved the latency difference. Say the latency drops from 181 ms to 100, then you client has to fix 81ms, if we say a frame in TTD has a minium of 10ms we have synced again in around 9 frames.

If the latency gets higher, we have a problem, then we can only wait until the next heartbeat package (with maybe new player created actions) arrive. If the latency changes from 100ms to 181ms, the player has a stop of 81ms unresponsive UI.

(This can be little bit compensated if we cache say 2 frames always on the client side before TTD really starts doing something, a bit like a audio stream, you don't hear something really until the cache is full) If the cache gets empty, we need to throttle the client a bit, and say, we change the frame time to 30ms, so in 81 frames, we get an extra one in our cache)



Hmm, to make it for all people understandable, TTDPatch Multiplayer would work like a music or video stream, you have to wait a bit on start to get a cache, then if something bad happens on the line, you will eat the cache, But if the cache gets smaller, your TTD will get abit slower,a) you don't have something in the cache, you get a fullstop or b) your cache gets filled again, and your TTD will run in normal speed.

*edit*
Trying fixing the Broken quotes...

eis_os: Can I ask what do you plan to do with code that assumes that actions are already finished when DoAction returns? For example, in NewRailVehWindowHandler, at 00161F5D, the window handler buys a new rail vehicle via DoAction, and then starts to look for the vehicle it bought in the vehicle array. If you do some magic with DoAction that delays the actions, code parts like this will break. I don't know how many such code parts are there, but probably too many to patch manually.

Well we need aswell callbacks, so we get a DoAction by Players, and the usally internal DoActions.
Reconstruction the stack after a callback gets a bit tricky, we can find a bunch of code that do Player Actions with a fragment that looks for the sound for gui clicks. The other have to be created manually, as usally...


About the question on irc, yes, the gui would currently do a fullstop without a heartbeat packet and should show some kind of "waiting x secs for lost connection... "

We could try to put the GUI into some kind of extended pause mode so it still reacts on most on your clicks, but this is stuff that can be later decided.

ReSyncing and rejoining will be abit more complicate then as example in OTTD, the server ask for the gamestate at a tick. And records all actions until the savegame arrives and then it can send it to the new/synced client. So the server runs with quite low memory and is nothing more then a relay. Still it can generally record a full game if decided for debug purpose from the first.

So TTDPatch server would be a relay for data transmitting (savegames, and user changes) and a global time server, that throws every 27ms a tick on the line. To reduce the overhead we could aswell change it to actually only do transfers every second tick. This would reduce bandwidth usage..,

With regards to latency, with the heartbeat model, we can compensate simply by slowing the game if there's a bunch of really laggy clients.

Notes, sizes are described in bits with prefix for signed or unsigned.

Specs:

The server maintains an array of 8 player objects, each object has a field for the connection id,
a 16 byte password. These are the possible Player Slots.

The server waits for a connection on a tcp port until the maxplayer is reached.


After the first player arrives, a framecounter variable will be increased every 27ms and a heartbeat message will be send every 27ms to all player slots with a valid connection id.


Stage 1:
After a client connects, the server waits for a handshake, the client gets a connection id and is now in Stage 1.

The client sends an Hardcoded String TTDPATCH + u32 version of TTDPatch

If it doesn't send it, the server disconnects the client after a max timeout of 1min.
The Server returns TTDPATCHSERVER + u32 serverversion + u16 status, if the client doesn't work with the server version, it will disconnect now.


The status can be 0x00 for good connection, or any other error, like server full,
if there is an error, the client gets disconnected.

Status Codes:
00 = TTDPatch Version right, client will go in Stage2
81 = TTDPatch version wrong
82 = Server full


Stage 2:
The server now accepts service packets bigger 0xF0, the server sends a free slot message (0xF1)

As soon as the server recieves a Set Player Slot(0xF2) it checks the password if it matches,
and sends (0xF3) if it matches or (0xF4) if it's wrong.


If it matches, the server will look if there is already is atleast one player slot with connection id.


When there is one, the server does a resync with this player.
The connection id will be added to the player slot, the heartbeat will be enabled.


Client Stage 3:
As soon as a Client arrives in Stage 3, the server accepts all message types by the client.

The client may now send packets now,
if the server recieves Action data packets (0x01), it stores them into a new temporay buffer.
as soon the server will recieve an (0x02) the server may send the buffer together with the next heartbeat. (see below)

If the server recieves a random gen value 0x40 package,
it has to check them with all other players, if a player has a different random gen then the others, it will be disconnected or resynced.

Heartbeats:
If the heartbeat is enabled, every 27ms:
Server Framenumber is increased.
If there is buffered action date, in all buffered packets the framenumber will be set to the new server framenumber.
The buffered data is broadcasted to all player slots.

A heartbeat package will be send, every 100 heartbeat packets the Heartbeat type 0x31 is used, otherwise 0x30.


Client Disconnect:
The connection id will be removed from the player slot.
If the last player disconnects, the server will end.



Resync:
(only for Stage2/3 clients)
Hearbeat if it's enabled will be stopped

The server broadcasts a Slot(0x44) message to all other player slots with a connection id.
It waits until it gets a 0x45 Sync Package, this will be transmitted to the new/resynced client.
Hearbeat will be enabled again


Advance Servers Resyncing:
The connection id will removed from the player slot.

The server broadcasts a Slot(0x44) message to all other player slots with a framenumber.
All action Data a server transmits will be recorded now.
The server waits to get a 0x45 Sync Package, this will be transmitted to the new/resynced client.
Addtional all ActionData will be transmitted with sync heartbeats (0x46, no 27ms wait) up to the framenumber.
The connection id will be restored from the player slot.


Packets are defined as:


[u32 framenumber] [u8 type] [variable amount of data]

Types are:
0x01 = [u8 len] [len * u8 data] action data follows, the server doesn't need to know the content.
0x02 = End of Action Data. No extra data

Service Package Types:
0x30 = heartbeat No extra data
0x31 = heartbeat with random number request => zero bytes follow

0x40 = random gen value => [u32 random]

0x44 = get sync package
0x45 = Sync Package (whole savegame data). [u32 len] [u8*len of savegame data]
0x46 = sync heartbeat, no extra data, only send when a client gets resyced.


Server Control Messages:
0x80 = Quit Server [u8*16 server password]
0x81 = KickBan Playerslot [u8*16 server password]

Stage 2 Packets:

0xF1 = Free Slot Messages [u32 len of data package] [u8 free slot mask] [u8 password mask] [zero terminated strings follow for all player slots]
Free slot mask => For every free slot (no connection id) of player a bit set
Password mask => Bit set if this playerslot is password protected
0xF2 = Set Player Slot [u8 player slot to use] [16x u8 password]
Password is filled with zero to match 16 Bytes, all zero means no password.
0xF3 = Player Accepted.
0xF4 = Password wrong.
0xF5 = newgrfmap with md5sums

That's enough of a spec for me to start writing some code. There are some inconsistencies in your text, but I'll go through those once I've written some implementation (as I'll probably find more once I do that). In the meantime, it might be a good idea to move the specs to a wiki (the TTD Patch one, probably). Better formatting, editing and versioning capabilities.