UGS Protocols / Discussion

UGS Network Model

mdm: I believe there are serious misconceptions about the network-model, which I will elaborate upon below.

1. Use of the Jabber/XMPP Protocol

Jabber/XMPP is a protocol for instant messaging and presence information. That means it is used to tell you if people that you know are available for communication and if they are it transports your messages. It was made for applications like ICQ, MSN, Jabber (of course) or Skype, which are built around contact lists. If you want to use it for UGS a number of problems arise.

Evan: At it's heart, XMPP was designed as an open, flexible protocol for sending data across the internet in real time. There are several cases where it has been used as a protocol for game servers, indeed Jabber developers have said it is perfectly capable and suitable to use XMPP for this.

mdm: If you remember the names of any relevant cases, please link them here. I couldn't find anything with google. Thank you.

1.1 Multi-User Chat (MUC) Rooms

Usually on a game-server you do not want to limit yourself to play only people that you know. But how do you know who else is on the server, if they are not in your contact list? The suggested remedy for this problem was the Jabber/XMPP-MUC-Extension.

This extension allows you to create MUC-Rooms. These rooms are similar to those on KGS. However, there is a big difference. If you know IRC, you probably know already what I'm talking about. Anybody can create rooms and more importantly gains ownership to the rooms he created. That means he can kick and ban users from the room and give other users the right to do so. Apart from the ability to keep the “French Room” free of non-french-speaking people (sorry, just kidding ;), I don't know if that would be very useful.

Evan: It is up to the server to decide what privileges the different users have. In other words, we can control all of these things to suit our needs, blindly copying the way it is done on normal jabber servers was certainly not the plan.

mdm: Well, of course the server can do whatever it wants, but then it would not be following the protocol as it is defined in the specification. In other words, you just have defined a custom protocol.

Malweth: This is not true. A custom protocol would indicate that the actual communication between client and server has changed. Having different permissions for different users is outside of the scope of the protocol and inside the scope of the architecture.

1.2 Jabber/XMPP Resources

In the protocol clients are identified by an ID of the form <user@targetserver/resource> (e.g. <paul@jabber.org/Home>). The resource part enables multiple connections to the same server.

Example 1: <paul@jabber.org/Home> could log in without logging out <paul@jabber.org/Work> first.

Example 2: <paul@jabber.org/ComputerA> and <paul@jabber.org/ComputerB> could log in simultaneously.

Example 3: <paul@jabber.org/17kyu> and <paul@jabber.org/9dan> could play a game together as a team.
Correction: They cannot play two different games at the same time - as originally stated above (mdm).

Evan: Again, it is possible for the server to block each of these possibilities if need be. We could also only allow registered nicknames to enter chat rooms, I believe there is a JEP that concerns this.

mdm: If you have a link to that JEP you are talking about, I would be interested.

1.3 Custom Protocol

I am sure that the problems described above (I'm sure there are some more) could be worked around, but that would probably change the base protocol. So I suggest to implement a custom protocol. It's not much harder than fixing Jabber/XMPP. Jabber/XMPP- and IRC-protocol specifications are still good learning references, though. So you might want to study both of them thoroughly, before implementing your own protocol.
You might also want to look at IGSCP, although it seems insufficient to me.

Evan: No, changing the base protocol is not needed, it's just a case of creating or modifying the server software to suit our needs, this needn't involve breaking any of the JEPs that we plan to make use of. Creating a custom protocol really isn't needed, jabber was designed for this kind of flexibility.

Malweth: In fact, even if custom protocol is needed, Jabber extensibility allows for this. It's one of the reasons Jabber/XMPP was chosen as a base protocol.

2. The Master/Server/Client Model

A fully distributed network has been suggested as network model. In a distributed network widely spread machines compute partial information (e.g. game results), which is gathered by a master-server, who then combines them into complete information (e.g. ratings). And guess what – Yes, still more problems arise.

2.1 Master Server

A master-server is needed if the computed ratings should be consistent across the network. Servers could share game results without a master, but then they would be subject to netsplits (just like IRC networks are). If no master (or a similar sharing-mechanism) is used the ratings would be independent on each server. Having independent ratings means having independent servers, really. Thus the most prominent reason against having independent ratings probably is that creating a new server potentially splits the user-base of existing servers. They would use the same protocol, but that's all.

The problem is even with a master-server they become little more. They become independent servers with a shared rating. Some of you might argue that you can connect to many of those servers simultaneously, through the same user-interface and that this would make the individual servers actually appear as top-level chat-rooms. Well, there you have 50 shiny, new servers and each of them is used by about 200 people (assuming each users is logged into two servers at a time). Don't you think that a single server could handle the total of 5000 users easily and cost much less?

Evan: No one plans creating 20 servers just for the sake of it. Yet people should be able to create new servers if they are unhappy with the service provided by existing ones. And by this I include the possibility of setting up a "competing" rankings server. This is a philosophical point as much as a practical one. By creating an open network of servers I feel people/companies/organisations will be more inclined to join and support us. Of course this open network may only ever actually have 2 or even one server/s that are involved!

These points have though already been made at UltimateGoServer/Architecture, I really don't feel you've addressed the issue at all. Restricting ourselves to a single server just for the sake of simplicity, simply isn't good enough.

mdm: I agree with you. My basic point here was that no single component should be irreplaceable - for the exact same reason you stated above. Maybe I was not clear enough about that. But UltimateGoServer/Architecture states that the master should also be used for authentication (for the whole network), which implies that a different rating-server would also create a different network. That's why is was worried about that master-server.

Malweth: This is still a point of contention. There are problems with having a single master server just as there are problems with having multiple master servers (or its equivalent case, multiple servers with no master).

In the case of a single master server, there are the problems with having a single strike point and anti-opensource (e.g. the master either isn't open source or new contending servers can be opened but cannot communicate with the others).

In the case of multiple servers there's a very big problem with trustability. In other words, if multiple ratings engines are allowed, do the engines trust each other completely, less than games played on the same server, or not at all? Joe 5d on new server X may really be a KGS 20k. I think there are ways to resolve this (probably involving a partial trust scenario)

The other thing is that if such a master-server ever goes down, nobody will be able to play rated games, because the servers can't obtain the current ratings. And if you mirror the rating database on the servers, you can just as well have the servers share ratings directly.

2.2 Single Server Approach

I think a single-server approach is much more reasonable. Having lots of different Jabber servers works, because they do not have to share information (e.g. ratings). But we do. If the server-load should ever turn out to be too high, you could „secretly“ push the actual work onto slave-servers, while still appearing to the end-user as a single server (that's what google and a lot of other people do).

I would strongly discourage the creation of many new servers. I agree that KGS is still two steps away from perfect and I would prefer something that I can fix myself instead of waiting for wms to fix it. Since you are talking about open-source you can't really prevent people from setting up their own server (which is good), but you should not design for using lots of servers.

Malweth: This is the reason for the proposed /architecture. A single core server with multiple "rooms" (subservers) that can be "fixed" as their owners see fit. A single server would suffer from a few major problems:

Firstly, a single server would have some of the same problems as a single master server. For instance, an open source single server will give way to contending servers that cannot interact with one another. Additionally, there's the problem with having a single strike point.

Secondly, a single server is not easily updateable. If there's a new feature that you want to add... how do you load it onto the server? Either you get approval after testing and it goes into the next scheduled update (which is HARDLY faster than what KGS currently does) OR you start your own contending server! Having scheduled updates very often is not an appropriate solution for a server that may have traffic from 100+ people 24 hours a day. Even having a weekly update would disrupt many people's playing schedules.

Having subservers that are "owned" by others solves much of this problem. Scheduled updates for the master can trickle down to quarterly (or less often) once the major bugs are fixed. Scheduled updates by subservers can be done in off-hours as subservers are likely to be (somewhat) regional.

2.3 Interoperability With Other Servers

You could have an option to let users connect to other (independent) servers. If any existing server chooses to adopt your protocol (which I doubt), you could then connect to it using the same client. Learning from Jabber/XMPP you could even have server components that interface your protocol e.g. with the IGS protocol and let users play on foreign-protocol servers. Fixes in the protocol-translation code would then be server-side updates. Clients would remain untouched.

The lack of a master-server allows different servers to have different rating-systems on different servers. I know some of you think I am contradicting myself here. Earlier I said independent ratings were a bad idea. They are a nightmare in my opinion, but you can't really avoid them. You are go-players. You are supposed to think stuff through ;).

Assume you had developed a master-server to share ratings. Now, some of the users of your network think the rating system is utterly bad. They complain to the admin (they should complain to the developers, but they will probably complain to the admin). But the admin (or the developers) won't fix it because it's in the master-server code and it would affect all the users who are happy with the rating system as it is (or at least don't complain about it). However, it's open-source. So the disappointed users might just take the source-code and fix it. Et voilà – you have two incompatible master-servers running two separate networks. Tell me, what have you gained compared to a couple of servers with (possibly) independent rating systems?

Well, I hope this was helpful. Forgive me if it wasn't. Any feedback here or directly to me (mdm) is welcome.

Benjamin: Welcome to the team! Maybe discussion like this should better be done on our Mailing-List. The wiki should rather be used to give a summary of our results and to give others the possibility to say something without registering.