[MUD-Dev] Client-Server vs. Peer-to-Peer -- Implementing DIS on the Internet

[claw at kanga.nu]

A decent summary of an older thread here:

http://artoo.hemmet.s-hem.chalmers.se/~crozius/resources/dis/cs-vs-ptp.html

-<cut>-
                 Client-Server vs. Peer-to-Peer 

                Implementing DIS on the Internet 

            Niklas Elmqvist (d97elm at dtek.chalmers.se).
                      6 January 1999 


Introduction

The DIS standard has traditionally implied a so-called peer-to-peer
architecture -- in this setup, there is no central server and the
clients communicate with each other directly. An example of this is
that all clients are responsible for broadcasting their status to
the rest of the world at a regular basis. In addition, clients are
also responsible for reporting fired shots, and the targeted client
is then charged with computing whether or not the shot hit and the
extents of the damages. A peer-to-peer structure plainly allows much
greater flexibility when it comes to distribution -- all the
computing load is more or less evenly distributed across the
individual clients, at least in theory, thus avoiding the need for a
powerful number-crunching server.

However, peer-to-peer has some inherent problems which makes it
unsuitable for use on untrusted networks with high potential
latency, the main one being the all-important issue of security. In
addition, having all peers communicate with each other introduces a
lot of overhead on large and slow networks, and thus, this
architecture scales very poorly with an increasing number of
clients. In a client-server architecture, on the other hand, clients
are connected to a (or several) central server which handles the
brunt of all computation -- the clients only receive information
pertinent to their own situation, and are given a lot less leeway in
terms of responsibilities and decision-making. Clearly, this
remedies many of the earlier problems, but introduces a few new
ones.

It is the goal of this paper to discuss the advantages and
disadvantages of these two communication paradigms and weigh them
against each other to find the most suitable one.

Acknowledgements

Many people from the MUD-Dev mailing list contributed greatly to the
creation of this document. These honorable and knowledgable
individuals have been duly credited below. (Please note that I have
not even tried to attribute different portions of the text to
specific persons -- this paper is a mosaic of their and my opinions
and the lines are all blurred.) Thank you for your help.

     Greg Underwood - MUD-Dev denizen and DIS expert. 
     JC Lawrence - maintainer of the MUD-Dev mailing list. 
     Ling - MUD-Dev denizen. 
     Marc Hernandez - MUD-Dev denizen. 
     Raph Koster - MUD-Dev denizen and Ultima On-Line Designer Dragon 

Peer-to-Peer

As stated earlier, we are interested in investigating a suitable
communication paradigm for untrusted and high-latency networks such
as the Internet, FMIP (the Swedish Defense IP Network), or other
large-scale WANs.  A traditional DoD DIS system would use a
peer-to-peer architecture, treating all nodes as equal and avoiding
the use of a central server. This means that the clients themselves
are arbiters for resolving vital simulation issues in the world, in
essence making each node into its own server.

The benefits of this setup is clearly that the system is totally
distributed and does not depend on number-crunching central
servers. All simulators are treated the same, and may leave and
enter the simulation at will.

However, in order for the peer-to-peer paradigm to work well, the
following assumptions must be met:

     All nodes are equally trustworthy. 
     Bandwidth is free. 
     Latency is low. 
     Nodes have infinite resources. 

While these assumptions certainly are viable for use on
high-performance LANs such as those DIS software traditionally has
been run on (meaning high-budget government/military simulation
systems), they ring hollow in the context of large, untrusted,
high-latency networks such as the Internet. Here, not only are
client resources and bandwidth limited, but security aspects also
come into play as clients can no longer be trusted with vital
decisions.

Bandwidth

For one thing, the fact that every node in the simulation is
required to inform all other nodes about its status may actually
swamp even the fastest of networks if a sufficently large number of
clients connect. Add the fact that the Internet and similar networks
are, at best, moderately fast, and you have a recipe for
failure. Techniques such as IP broadcasting and multicasting would
certainly do a lot towards solving the bandwidth problem, but
existing routing hardware (mostly manufactured by Cisco) have
notoriously low performance with these methods, and this is a fact
that is not likely to change in the immediate future.

Scalability

Even if the bandwidth problem would be solved, the issue of
computation will quickly rear its ugly head. If each node is
informed of the status of all other nodes, the computation load for
the node will thus increase linearly with the number of connected
clients. This is distribution, yes, but distribution in extremis,
since there will be a tremendous amount of redundancy in the
calculations performed by the nodes. Granted, a publish/subscribe
messaging system as proposed in the new HLA (high-level
architecture) standard would help immensely with reducing this
problem, as well as, to a certain extent, the bandwith problem.

In other words, a peer-to-peer setup might very well be valid for
small virtual worlds with only a few connected clients (cf the
earliest Doom and Duke Nukem 3D PC multiplayer games which used a
peer-to-peer model), but does not scale at all for larger numbers of
clients.

Latency

Latency is not a big problem on high-end LANs such as those employed
by governments, but is, unfortunately, a fact of life on the
Internet. In a DIS context, latency will have a large impact on
"fairness" since the average latency for different nodes bears no
relation to each other. Without a server acting as a latency
mediator, this fairness problem will become a real issue.

Network Access

Another, slightly more minor, issue with peer-to-peer software is
that they might be hard to find and contact on large networks such
as the big wide Internet. One of the main reasons for putting up
servers in the first place is that they usually have well-known
network addresses which makes it trivial for clients to
connect. Peer-to-peer systems do not have this benefit, as there is
no server and the coming and goings of the individual clients is
completely haphazard.

Security

The final and most important drawback of the DIS peer-to-peer model
is the inherent lack of security. First of all, the DIS standard has
no explicit security mechanism, not even an entrance gatekeeper. In
other words, clients are allowed to enter and leave games at
will. Naturally, this is highly undesirable when simulation
administrators would like to limit access to a specific
session. Secondly, the fact that the clients themselves are
responsible for arbiting simulation events is a huge security hole
-- it should be a small task for a skilled programmer to hack the
client software, either by using a hex editor or by modifying the
source code (in case of an Open Source project), and make it
cheat. More specifically, in a naive DIS implementation, a tampered
simulator would, upon receiving a message from another simulation
that a shot has been fired upon it, always reply that the shot
missed, or that a very small amount of damage was inflicted. This is
an example of the saying "Never EVER trust the client -- it's in the
hands of the enemy." Experiences from naive multiplayer games such
as Interstate '76 and Diablo is plain proof of the problems which
arise otherwise.

In addition, the fact that each client is supposed to communicate
with its peers also means that the clients will retain the network
addresses of all participants (full IP addresses, in this
case). This kind of setup paves the road for denial-of-service (DoS)
attacks against other clients without having to tamper with the
client executable -- sniffing the destination of outgoing network
packets is sufficient. Many players of Ultime On-Line, Origin's
massively multiplayer on-line game, have been hit especially hard by
this kind of tactics (although the problem here, as the author
understands it, does not stem from the fact that the IP adresses to
other clients is handled by the local client -- they are not -- but
that other network connectivity tools such as ICQ displays the IP
address for others to see).

Conclusion

The conclusion must be that employing DIS across a peer-to-peer
communication model is highly unsuited for use on large, untrusted,
moderate-performance networks such as the Internet.

Client-Server

While not a panacea in its own right, the client-server paradigm is
much better suited than the peer-to-peer one to networks which do
not meet the assumptions listed above. This communication model
centers around the notion of one or several central computers acting
as hubs and game arbiters. Clients do not communicate directly with
each other -- all data is passed strictly between clients and
server. In addition, the server retains decision-making capabilities
for critical game mechanics to maximize security and minimize the
risk of cheating. Below, we will explore the benefits as well as the
disadvantages of using a client-server model for a DIS
implementation.

The first issue and potential disadvantage is that the DIS standard
was not really intended for a client-server approach. Most of the
methods and protocols discussed in the standard implies a
peer-to-peer relationship between nodes. However, it is beyond the
scope of this document to analyze any deviations from the DIS
standard a client-server implementation would entail -- this will be
handled in other documents.

Conclusion

Summary


Updated 1999-01-06 by Niklas Elmqvist (d97elm at dtek.chalmers.se). 

-<cut>-

--
J C Lawrence                              Internet: claw at kanga.nu
----------(*)                            Internet: coder at kanga.nu
...Honorary Member of Clan McFud -- Teamer's Avenging Monolith...


_______________________________________________
MUD-Dev maillist  -  MUD-Dev at kanga.nu
http://www.kanga.nu/lists/listinfo/mud-dev