Foenix index server_protocol
*** Foenix Server Protocol ***
Heading,
Foenix Server Common Protocol
Draft 20231223.aa
Craig Turner 2023
Project overview,
Goal: design and implement a common protocol approach that will work for
multiple games/applications.
The approach must scale down to MTU of 255, in order to support the 8-bit
systems.
Server side work will be done in python3 asyncio.
Deploy initially to songseed.org, later to a Ryzen box I have in London on
FTTH that I can dedicate to this.
The protocol is to be practical for client authors using C.
The protocol is to be practical for client authors using Basic.
The primary use-case should be for UDP clients, as conventional wisdom is
that this is a better foundation for games.
Protocols should be defined to support UDP fail-cases, such as lost
messages in-flight.
We need to keep options open for TCP support down the line. I cannot think
of any accomodations we would need to support. Porting from TCP to UDP is
problematic, but I cannot think of any hassles going in the other
direction.
Similarly, we need to keep the door open to being able to support
encrypted channels. It is not clear how current foenix hardware will
support this. Perhaps through cartridge additions.
Milestones
m1. Bootstrapped codebase and release arrangements presenting an echo
server (Done 20231006, see next section for evaluation services)
m2. Draft protocol document to Rene, Gadget, and rest of community. (Done
20231007)
m3. Basic chat server with TCP support. (Done early oct 2023)
m4. Chat server now also has a UDP server. (Done 20231111)
m5. Guess-the-number server with reference client in python (via solent
framework). The purpose of the work in this milestone is to develop an
approach to binary UDP protocol design. (Done around 23 December 2023)
m6. Multiplayer, arcade-based interaction.
Purpose. Evolve the binary UDP model and create a multiplayer arcade
interaction game that is fun to play. System should have a lobby, and
should be able to run multiple games in parallel on the same server.
Option. Driving game. Reluctance: a. I don't think per-frame would
give an acceptable experience. b. We should be able to come up with
something simpler, particularly something that is fun to play on a
grid rather than an open field. c. Collision detection seems complex
due to client/server sync concerns.
Option. Boom UDP. Bomberman-like. All movement is strict grid-based,
which could be represented on an ascii grid. Players enter a grid
square, exit a grid square. Players can only move once they are
settled on a grid square.
Option. Tails. Snakes with fixed tail length operate inside a plain.
Works for development purposes, but I don't think this would be fun.
Snake seems to have limited play.
First foenix client to games above.
Discussion,
Endianess. There will be big-endian and little-endian versions of each
server. We expect the 68k models will connect to BE servers and the 65x
models will connect to LE servers.
Form "something_id" names an integer enumeration.
Form "something_h" names a string enumeration ('handle').
There will be a standard set of datatypes, "dtype". This document
describes these.
There will be a simple DSL for describing protocols. This will be useful
to client authors.
Where you see cs, it means client-to-server.
Where you see sc, it means server-to-client.
It is up to each application to ensure their message size does not exceed
MTU. Messages should be kept within 256 bytes, as this is the maximum that
the 8/16-bit platforms can handle.
We will need to discuss this further once we are looking at streaming
music over the network. Perhaps we come up with a binary protocol
optimised for each platform set.
The protocol is designed for situations where there is a clear server and
a client. UDP can support other arrangements, but there are no
accomodations for those in this design.
In order for apps to support UDP communication, the onus is on each app to
design their message schema to support this. Some systems that communicate
by UDP have the server keep a full history of messages for each client (or
for a broadcast group) to allow clients to rerequest missed messages. This
protocol does not place such responsibilities on the server.
Server will be implemented in python3 asyncio ande deployed to linux. I
have personal (and strictly non-work) IP available in this, and the
platform allows me to move quickly and build sophisticated stuff.
The codebase includes a design for a deployment, for multiple environments
(uat, prod, etc) and port allocation to specific applications.
Outstanding matters,
License
There are deployment complications around making the code available,
to do with the keys of the ssh client. Those should not sit in github,
and I would rather move the server code forward than spend time
polishing deployment in this moment.
License of server code. My intent is to make this code available
either to Stefany for her business project, or else as open-source
code. I have not settled on a model for this.
License of this document. I have no problems with people building
derived works from this document, but at this time ask to be credited
in any derived works. (i.e. standard copyright assumptions, but with
no charge for use.)
Checksum algorithm,
I have not worked out how this works.
Third party application developers
It is not yet clear how other people will write game servers that can
deploy into this model. One option would be to have the code as a
monorepo, with defined administrators.
More advanced deployment considerations,
Want a substrate that allows me to deploy an array of server apps from
a single command.
Seek to design deployment so that players are not booted off their
existing processes at time of a restart, but get to finish their game,
whilst new clients connect to a new server. May need a dual host
blue/green arrangement to support this, with a proxy in front of it.
Steam-for-foenix
Should be possible to stream foenix application content to foenix
users via this server system. Have not put much thought into this yet.
Once we have the basics of this system in place, we could ask people
from other retro communities if they would like to implement a client.
Music
It would be /awesome/ to be able to stream music from the foenix
server out to clients. I also have some ideas about a role for jingles
in Foenix-Cup. I have formal musical training (theory, singing) but
low experience in audio programming. Could use a partner who could
work with me on this. Basically, I think we will just need a data
structure that we could stream out. Could start with a melody, and
then move to polyphony, custom samples.
Would appreciate someone I could work with one-on-one for this, in
parallel to the milestones given below.
Encryption
Once we are talking about sending authenticaiton details over the
internet, we will need an encryption story. It is straightforward for
me to implement on the server-side, but there will need to be some
story about this on the Foenix side. One option would be to have a
piece of bridging hardware between Foenix devices and the internet
that has a build processor that sorts this out. i.e. Foenix device
would think it has a TCP connection, but it would really be running
over something like stunnel between the bridging device and the
server. There is an option here for Stefany to get into the business
of selling stunnel devices to people in the retro community who want
to be able to talk to Foenix servers or webservers.
Available demonstrations,
HTTP server
http://songseed.org:2324/
Comments you place here will be retained in the linux process.
SSH server
ssh admin@songseed.org -p 2323
Password is 'admin'
This is a ssh shell into the same process as the HTTP server. If you
type ls you can see a "say" and "show" command. See the way the text
in the HTTP server is the same as the text you can interact with in
the ssh shell. It is my intent that this ssh shell should become the
command interface for the foenix server into the future, because it is
much faster to build administration functionality against a CLI like
this than having to create web pages for each new action. (Also, it is
cooler.) I suspect we will do other application funcitonality through
this system. For example, I can imagine implementing in ASCII a track
generator for Foenix-Cup, with the tracks them being available on the
server.
Note that foenix clients will never interact with this ssh server.
Rather, this is a control interface for server admins who want to
affect the foenix server.
If you are on Windows and have not used ssh before, download the
program Putty, and then use that to connect to the details above. If
you are on a mac, you will already have ssh configured. Open a
Terminal, and then type the syntax above.
Echo server
nc -vv songseed.org 2327
This simply echoes bytes back to you. May be useful for someone who is
playing with foenix serial-to-network equipment.
Emphasis: all the links above connect to a single linux process. It is not
separate software instances. Ssh is into the process, not into the Linux.
Types used in foenix server protocol
dtype_h size desc
-- -- --
u8 1 uint8
arr_u8 * leading byte gives length.
mask_u8 1 each bit is a flag
u16 2 uint16
arr_u16 * leading byte (u8) gives length.
u32 4 uint32
str 0-256 leading byte gives length ('pascal string')
arr_str * leading uint8 gives array length, then each string
within is a pascal string.
arena * untyped mem, can occur as last field in the packet.
If you care about perfection,
Strings should be encoded as UTF-8, with the length representing the
length of the UTF-8 string, not the number of bytes.
If you do not care about perfection,
Do the easy thing of treating the length given by the leading character as
the number of bytes, and unpacking characters as 7-bit ascii. You will
probably get away with it without ever suffering ill-effects.
If you get problems with strings in the future, ask us about this in the
forums and people who know about UTF-8 will be able to guide you to the
'perfection' solution above.
Structure of envelope,
type name size
-- -- --
u16 env_mtype_h 2
u16 env_server_secret 2
u16 env_client_secret 2
UDP protocol notes
UDP protocols,
Guess-the-number protocol
Be robust in handling duplicate message receive.
Be generous in performing duplicate message send.
Handshake,
The client opens a three-way handshake with the server. This
highlights routing problems between sides, and allows an exchange of
secrets which would frustate subsequent efforts to do spoofing.
Tha handshake also has both sides agree on the name of the protocol.
This acts as a defensive measure against situations where a client
connects to the port of a server that speaks a different protocol.
Handshake should never handle authentication. That design would make
it vulnerable to timing attacks that spoof logon. Where auth is
needed, handle it as a separate exchange of messages that happens
after connection/greet.
Equipment notes,
Whenever you order foenix units, ask Stefanie to install a feather with
the SLIP firmware loaded.
Separately, Gadget recommended these devices for connecting Foenix to
network in a way that uses the serial interface, and you should also
consider getting one of these. Note that you will probably need to do
firmware updates to that, which may steer you to install crappy software
on your machine. (I refuse to do this on computers I care about)
https://www.tindie.com/products/theoldnet/rs232-serial-wifi-modem-for-vintage-computers-v4/
https://www.cbmstuff.com/index.php?route=product/product&path=59_66&product_id=59
https://biosrhythm.com/?page_id=1453
https://electronicsisfun.com/gurumodem
https://www.tindie.com/products/retrodisks/wirsa-v2-wifi-rs232-serial-modem-adapter-with-sd/
You may also need a serial cable extender due to practical matters.
When I got my theoldnet unit it did not plug into the foenix because
both the computer and the plug have standoff screws that run into one
another. The solution to this is to get a serial extender cable. Be
careful not to order a null modem cable when you need a serial
extender. Be careful to get a unit with one male and one female
connector.
If you can get a null modem cable (plus gender-changer, probably) you
could hook your foenix directly to a linux box and supply connectivity via
SLIP from there.