********************************************************************
FTSC FIDONET TECHNICAL STANDARDS COMMITTEE
********************************************************************
Publication: FSP-1011
Revision: 3
Title: Binkp - a protocol for transferring FidoNet mail over
reliable connections
Authors: Dima Maloff
Nick Soveiko
Maxim Masiutin
Revision Date: 31 July 2000
Expiry Date: 31 July 2002
--------------------------------------------------------------------
This specification defines binkp - a protocol to handle a session between two Fidonet Technology systems over a reliable connection. Assumption that the connection is reliable makes possible to eliminate error-checking and unnecessary synchronization steps, achieving both ease of implementation and major performance improvement over connections with large unpredictable delays (e.g. Internet).
This document is a Fidonet Standards Proposal (FSP).
This document specifies an optional Fidonet standard protocol for the Fidonet community, and requests discussion and suggestions for improvements.
This document is released to the public domain, and may be used, copied or modified for any purpose whatever.
It's been a long time since a new Fidonet protocol has been developed, [EMSI] definitions being published last time in 1991, not speaking about basic standards, [FTS-0001] and [FTS-0006]. Fidonet is evolving everyday and new transport layers are being introduced into practice. This led to a situation when in certain Fidonet Regions a visible portion of traffic, especially long distance traffic generating high toll, is being carried by means of protocols that formally are not Fidonet standards. This creates an ambiguity for such systems in indicating their additional capabilities in Fidonet nodelist and in some instances, from being listed in the nodelist at all.
This document attempts to document the current practice for communication between two Fidonet systems via a reliable channel, provide technical reference for Fidonet software developers and eventually improve Fidonet connectivity.
Existing Fidonet Technical Standards and Fidonet Reference Library documents [FTS-0001], [FTS-0006], [EMSI] specify both session handshake procedures and transmission capabilities that imply:
This was commonplace a few years ago, when Fidonet systems were not using transport other than direct dial-up on a visible basis. Things have changed today, when other communication media becomes widely available on a day-to-day basis. This communication media typically provides implementation of Physical, Data Link, Network and Transport layers of the ISO/OSI Reference Model and facilitates relieving Session layer of inappropriate functions, such as error control, flow control, call management and data transparency [Halsall95]. Examples of such communication media are TCP/IP socket connection and HDLC family protocol connection.
New communication media can be generally characterized by the reliable transmission service offered by it to the Session layer protocol. Reliable transmission implies that:
Combination of these factors imposed the following requirements for the new Fidonet protocol:
The key words "MUST", "MUST NOT", "SHOULD", "SHOULD NOT" and "MAY" in this document are to be interpreted as specified in [FTA-0006]. However, for readability, these words may sometimes not appear in all uppercase letters in this specification. Although it should not impact minimal realization of binkp protocol, it must be noted that Protocol Extensions may override, update or obsolete requirement levels indicated by the above keywords in chapters from 3 to 6 inclusive.
Calling party in this document is referred to as the Originating side and called party is referred to as the Answering side. Originating side here is the party that initiates the connection between two systems.
Mailer in this document is a software that implements the protocol.
Words "frame", "packet", and "block" when used in this document refer to binkp's Frames, unless explicitly stated otherwise.
Other definitions that are not local to this document can be found in the Glossary.
This document is organized as following:
Frames section defines binkp's frames. Binkp/1.0 commands and their arguments section provides detailed description of all defined protocol commands together with recommendations for their usage. Actual binkp implementation may match it's own diagrams provided that such implementation remains fully compatible with current specification. Protocol states section gives rigorous state diagrams for the minimum realization of binkp. All mailers MUST support this minimum realization. Recommended Protocol Extensions section documents most important extensions to the basic protocol that are in use as of the time of this writing. The License, Glossary and References sections can be found at the end of this document.
Binkp is a Fidonet session layer protocol intended for use over data transparent bi-directional channels with reliable transmission. There are no other requirements for the service provided by the underlying protocol suite. Presentation and application layer protocols are not discussed here. Whenever TCP/IP socket is used, IANA registered port number for binkp 24554 SHOULD be used (as registered with the Internet Assigned Numbers Authority).
Functionality of the minimum protocol realization makes provision for:
Binkp uses only one synchronization point during session startup, that is password exchange. This feature facilitates fast session startup for high latency links. Sliding window flow control is incorporated on the file level. This ensures that a batch of small files is transmitted with the same efficiency as a one large file.
Binkp is defined in terms of sending and receiving specifically formatted data blocks. We call them frames.
Command frames carry protocol commands and may change protocol state. Data frames are usually appended to files being received by mailers or may be discarded, depending on the protocol state.
The particular way of mapping an octet stream or a datagram stream of the transport layer into binkp frames may depend on the underlying protocol suite. At this time, we define such mapping for TCP/IP socket connection which can also be used for similar transports as well.
The socket stream is being split into binkp frames in the following manner:
7 6543210 76543210
+-+-------+--------+--- ................ ---+
|T| SIZE | DATA |
+-+-------+--------+--- ................ ---+
|<- 2 octets ->|<- up to 32767 octets ->|
(frame header) (frame data)
If T bit is 0, this is a data frame.
If T bit is 1, this is a command frame.
15 bits marked SIZE carry the size of the DATA part of the frame in octets (with the bit marked 0 being the least significant). That is, the actual length of a binkp frame is SIZE+2.
The size of the DATA part may vary between 1 and 32767 octets. A correct realization should never set SIZE to 0. Upon receiving of a packet header with the SIZE field set to 0, the total length of the incoming packet must be treated as 2, this packet must be dropped, and the event should be logged.
The first octet of a command frame data is the command ID. The ID must be between 0 and 127 inclusive.
Other octets carry command arguments. Command arguments are an arbitrary symbol string that may be null-terminated. Treating of a null character in the middle of a command depends on realization (with the options being "treat as whitespace" or "treat as end-of-line"). The terminating null character (if any) is either stripped or used by mailers internally as an end-of-line marker.
As stated before, command ID is a small number between 0 and 127. Every binkp command defined in this document has a symbolic name in the form M_XXX. Symbolic names are defined in binkp commands section. We will use symbolic names and not numeric command IDs to refer to commands everywhere in this document.
The following notation is used to describe binkp's command frames:
M_XXX "Data string"
The actual numeric command ID for the command with the symbolic name of M_XXX should be written into the first octet of the DATA area of a binkp frame. "Data string" is a string to be copied into DATA area starting at second octet. SIZE should be set to the total length of "Data string" plus one for the octet to store the command number. T bit should be set to 1.
M_OK "":
7 6543210 76543210 76543210
+-+-------+--------+--------+
|1| 0 1| 4|
+-+-------+--------+--------+
| | +----- command ID (no arguments)
| +-------- frame length
+- command frame flag
M_NUL "TEST":
+-+-------+--------+--------+-------+--------+--------+--------+ |1| 0 5| 0| T E S T | +-+-------+--------+--------+-------+--------+--------+--------+
Protocol commands may be classified the following way:
In Mailer-parseable commands that contain a file name, the file name MUST NOT include a whitespace (ASCII value 20 hex). The file name SHOULD NOT include symbols other than alphanumeric (A-Z,a-z,0-9) and safe characters as defined below in BNF. All other symbols are to be considered unsafe and SHOULD be escaped in the form of two hexadecimal digits preceded by a backslash (e.g. a whitespace must be transmitted as "\20").
filename = *pchar
pchar = unreserved | escape
unreserved = ALPHA | DIGIT | safe
safe = "@" | "&" | "=" | "+" | "%" | "$" | "-" | "_" |
"." | "!" | "(" | ")" | "#" | "|"
escape = "\" HEX HEX
National characters should not be escaped, but rather transmitted using [UTF8] encoding (see section discussing non-ASCII characters below).
The best current practice is that Mailer does not alter a file name without sysop's intention. If the mailer does provide such a mechanism, it MUST BE optional and it SHOULD BE off by default.
The protocol does not impose limitations on the file name length other than those arising from the finite length of the binkp frame itself.
Generally, mailer SHOULD use only characters from the ASCII range [32...126] in the symbol strings for command arguments. In case when there is a necessity to use non-ASCII characters, mailer SHOULD use the [UTF8] format of the multioctet Universal Character Set [ISO10646]. Mailer SHOULD use non-ASCII characters only if the other side have indicated it's support by transmitting M_NUL "OPT UTF8" frame during the session setup stage. Otherwise, mailer SHOULD assume that the remote does not support non-ASCII characters and SHOULD NOT use them in command arguments.
Format: symbolic_command_name command_ID
Command arguments contain human-readable information, such as nodelist info, sysop name, etc. This frame can also be used by some Mailers to exchange protocol options. Mailer MAY ignore and/or log arguments of M_NUL.
e.g. "ZYZ Dima Maloff"
The following format of M_NUL argument is recommended for compatibility purposes:
List of 4D/5D addresses (space separated).
e.g. "2:5047/13@fidonet 2:5047/0@fidonet"
Session password, case sensitive. After successful password authentication of the remote, originating side proceeds to the file transfer stage. This command MUST never be sent by the Answering side.
e.g. "pAsSwOrD"
Acknowledgement for a correct password. Upon receiving of this command, originating side goes to file transfer stage. This command MUST never be sent by the Originating side. Arguments may be ignored.
e.g. ""
Space separated list of parameters for the next file to be transmitted: filename; size in bytes; unixtime; file transmission offset.
In protocol extensions, negative values for the offset may have special meaning (see non-reliable mode for an example of such usage), basic implementation may treat negative values as an error.
Size, time and offset parameters are decimal. Until the next M_FILE command is received, all data frames must carry data from this file in consecutive manner. There is no end of file identifier as the file size is known beforehand. If there are "extra" data frames, Mailer may append this data to the file. By default, transmission of each file should be started from offset 0. M_GET command sent by the remote MUST force the mailer to start transmission from the specified offset.
e.g. "config.sys 125 2476327846 0"
or, answering to M_GET with offset 100:
"config.sys 125 2476327846 100"
End-of-Batch. M_EOB command must be transmitted after all the files have been sent.
Arguments of the command may be ignored.
e.g. ""
File acknowledgement, that must be transmitted upon receiving of the last data frame for current file. Arguments for this command shall be the same as for the M_FILE sent by remote, excluding the last argument, file offset, which is not transmitted back to the system which have sent M_FILE. M_GOT can also be transmitted while receiving a file, in which case transmitting party may interpret it as a destructive skip.
e.g. "config.sys 125 2476327846"
This command indicates a fatal error. A party sending M_ERR should abort the session. Argument should contain an error explanation and may be logged. Mailer sends M_ERR in response for an incorrect password. Mailer NUST NOT abort a session without sending a M_ERR or a M_BSY frame (though state machine tables, for simplicity, may not include "transmit M_ERR" instructions).
e.g. "Incorrect password"
M_BSY command is transmitted when the system encounters a non-fatal error typically due to temporary lack of resources to proceed with the session. The argument should contain an explanation of the situation and may be logged by remote. M_BSY may be sent at any time during the session (including session setup stage), not only the stages explicitly indicated in the finite state machine. The side, which have sent M_BSY, is in legal position to abort the session. Mailer MUST be able to accept M_BSY at any time. Though state machine tables, for simplicity, may not include handling of M_BSY command, Mailer MUST NOT be confused by reception of M_BSY command.
e.g. "Too many servers are running already"
If a mailer wishes to suggest the remote a time interval before the next session attempt, it may choose to transmit it in the following format:
M_BSY "RETRY NNNN: explanation"
where NNNN is interval in seconds (decimal string) and explanation is an arbitrary string containing explanation of the matter (optional).
M_GET command is a request to (re)send files. Arguments of the command are the same as for the M_FILE command and refer to a file which we'd like to receive from the remote.
Mailer may send M_GET when it doesn't like transmission file offset (e.g. file was partially received during one of the previous sessions).
e.g. "config.sys 125 2476327846 100"
Mailer reacts to this command as follows: according to the first three arguments (filename/size/unixtime), it determines whether the M_GET argument is the current file being transmitted to the remote (or a file that have been transmitted, but we are still waiting an M_GOT ack for it). If this is the case, it should
For the example above, corresponding M_FILE will have the following arguments: "config.sys 125 2476327846 100"
When the mailer is finished with transmitting data of the requested file it may proceed with transmission of other files it has for the remote.
Non destructive skip. Parameter is a space separated list of filename, size and unixtime. This command indicates that the remote should postpone sending the file until next session.
e.g. "config.sys 125 2476327846"
| Originating side | Answering side |
|---|---|
| M_NUL "SYS ..."
M_NUL "ZYZ ..." M_NUL "LOC ..." M_NUL "VER ..." M_ADR "2:2/2.2@fidonet" M_PWD "password" |
M_NUL "SYS ..."
M_NUL "ZYZ ..." M_NUL "LOC ..." M_NUL "VER ..." M_ADR "3:3/3.3@fidonet" (waiting for a password from remote) |
| (waiting for password acknowledgement) | M_OK "" (or M_ERR "Bad password") |
| (got M_OK) | M_FILE "file2 200 42342434 0" |
| M_FILE "file1 100 423424244 0" | data |
| data | data |
| data | data |
| M_EOB | (got file1, acknowledging it) |
| (got file2, acknowledging it) | M_GOT "file1 100 423424244" |
| M_GOT "file2 200 42342434" | data |
| M_EOB |
The protocol has two major stages: session setup (different for originating side and answering side) and file transfer (where state machined for both sides are the same). Methods for initiating connection as well as numerical values for particular timeouts are dependent on the underlying layer's protocol suite and are not considered here. Mailer MAY allow configuration of timeouts in reasonably wide range to cover all supported transport protocols.
The Finite State Machine notation is used throughout this section as defined by [FTS-0001].
Originating side should initiate a binkp session according to Table 1. Answering side should be able to act according to Table 2. Any optional extensions of the handshake procedure MUST NOT confuse the other side, which may choose at it's discretion to follow this minimal implementation. Upon successful handshake, both sides follow Table 3 (file transfer stage). That's why terms Answering side and Originating side were chosen for this specification instead of Client and Server - both sides play the same roles, and their state machines differ in session setup stage only.
Session setup stage has the following roles
Originating side sends M_ADR and M_PWD frames, waits for successful authentication acknowledgement from the Answering side (M_OK frame) and goes to file transfer stage. Originating side MUST NOT wait before sending M_ADR frame, i.e. this frame should be send just after setting up a connection on underlying layer. Originating side MUST NOT wait before sending M_PWD except after reception of M_ADR frame. The term wait in this paragraph means do not send anything while expecting data from remote.
| # | Name | Predicate(s) | Action(s) | Next |
|---|---|---|---|---|
| S0 | ConnInit | Attempt to establish connection | S1 | |
| S1 | WaitConn | Connection established | Send M_NUL frames with system
info (at least one M_NUL "SYS ..." frame should be
sent before M_ADR) Send M_ADR frame with system addresses Set Timer See if we have password for the remote |
S2 |
| Connection refused | Report no connection | exit | ||
| S2 | SendPasswd | Yes, we have a password | Send M_PWD "password" frame Reset Timer |
S3 |
| No, there's no password | Send M_PWD "-" frame | S3 | ||
| S3 | WaitAddr | M_ADR frame received | See if answering side presented the address we've called | S4 |
| M_BSY frame received | Report remote is busy | exit | ||
| M_ERR frame received | Report error | exit | ||
| M_NUL frame received | Ignore (optionally, log frame argument) | S3 | ||
| Other known frame received | Report unexpected frame | exit | ||
| Unknown frame received | Ignore | S3 | ||
| Nothing happens | Wait | S3 | ||
| Timer Expired | Report timeout | exit | ||
| S4 | AuthRemote | Yes, the address was presented | See if we've sent a password for this address | S5 |
| No, the address was not presented | Report we called the wrong system | exit | ||
| S5 | IfSecure | Yes, we've sent a password | Wait for M_OK frame | S6 |
| No, there was no password | Report non-secure session | T0 | ||
| S6 | WaitOk | M_OK frame received | report secure session | T0 |
| M_BSY frame received | Report remote is busy (Answering size MAY report busy after reception of caller's address) | exit | ||
| M_ERR frame received | Report error | exit | ||
| M_NUL frame received | Ignore (optionally, log arguments) | S6 | ||
| Other known frame received | Report unexpected frame | exit | ||
| Unknown frame received | Ignore | S6 | ||
| Nothing happens | Wait | S6 | ||
| Timer Expired | Report timeout | exit |
Originating side sends M_ADR and waits for M_ADR and M_PWD frames from remote. Upon receptions of these frames, it decides whether the password really matches the list of presented addresses, and either acknowledges it by sending M_OK frame (and goes to file transfer stage) or rejects by sending M_ERR frame (and disconnects). The term wait in this paragraph means do not send anything while expecting data from remote.
| # | Name | Predicate(s) | Action(s) | Next |
|---|---|---|---|---|
| R0 | WaitConn | Incoming connection established | Send M_NUL frames with system
info (at least one M_NUL "SYS ..." frame should be
sent before M_ADR) Send M_ADR frame with system addresses Set Timer | R1 |
| Nothing happens | Wait | R0 | ||
| R1 | WaitAddr | M_ADR frame received | See if we have a password for any of the remote addresses | R2 |
| M_ERR frame received | Report error | exit | ||
| M_NUL frame received | Log | R1 | ||
| Other known frame received | Report unexpected frame | exit | ||
| Unknown frame received | Ignore | R1 | ||
| Nothing happens | Wait | R1 | ||
| Timer expired | Report timeout | exit | ||
| R2 | IsPasswd | Yes, we have a password | Set Timer | R3 |
| Yes, but we have several different passwords for different addresses of the remote | Send M_ERR
frame Report inconsistent password settings | exit | ||
| No, there's no password | Report non-secure session | T0 | ||
| R3 | WaitPwd | M_PWD frame received | See if the password matches | R4 |
| M_ERR frame received | Report error | exit | ||
| M_NUL frame received | Log | R4 | ||
| Other known frame received | Report unexpected frame | exit | ||
| Unknown frame received | Ignore | R4 | ||
| Nothing happens | Wait | R3 | ||
| Timer Expired | Report timeout | exit | ||
| R4 | PwdAck | Yes, the password matches | Send M_OK frame Report secure session | T0 |
| No, password does not match | Report password error | exit |
File transfer stage is based on two major routines. We call them Receive Routine and Transmit Routine. These routines perform some actions depending on their state variables. State variables are RxState for Receive Routine and TxState for Transmit Routine.
RxState := { RxWaitF | RxAccF | RxReceD | RxWriteD | RxEOB | RxDone }
TxState := { TxGNF | TxTryR | TxReadS | TxWLA | TxDone }
| # | Name | Predicate(s) | Action(s) | Next |
|---|---|---|---|---|
| T0 | InitTransfer | none | Set Timer Set RxState to RxWaitF Set TxState to TxGNF | T1 |
| T1 | Switch | RxState is RxDone and TxState is TxDone | Report session complete | exit |
| Data Available in Input Buffer | call Receive routine | T2 | ||
| Free space exists in output buffer | call Transmit routine | T3 | ||
| Nothing happens | Wait | T1 | ||
| Timer Expired | Report Timeout | exit | ||
| T2 | Receive | Receive routine returned OK | Set Timer | T1 |
| Receive routine returned Failure | Close all opened files | exit | ||
| Receive routine returned Continue | Call Receive routine again | T2 | ||
| T3 | Transmit | Transmit routine returned OK | Set Timer | T1 |
| Transmit routine returned Failure | Close all opened files | exit | ||
| Transmit routine returned Continue | Call Transmit routine again | T3 |
Tables 4-6 are not actually state machines, but routines called during file transfer stage
We define here a FIFO queue called "TheQueue", which is used to pass incoming M_GET / M_GOT / M_SKIP frames from Receive Routine to Transmit Routine. Receive routine itself does not react to these frames.
| RxState | Predicate(s) | Condition(s) | Actions(s) | Next | Return |
|---|---|---|---|---|---|
| RxWaitF | Get a frame from Input Buffer | Haven't got a complete frame yet | none | RxWaitF | OK |
| Got Data frame | ignore | RxWaitF | OK | ||
| Got M_ERR | Report Error | RxDone | Failure | ||
| Got M_GET / M_GOT / M_SKIP | Add frame to The Queue | RxWaitF | OK | ||
| Got M_NUL | Log | RxWaitF | OK | ||
| Got M_EOB | Report End of Batch | RxEOB | OK | ||
| Got M_FILE | none | RxAccF | continue | ||
| Got other known frame | Report unexpected frame | RxDone | Failure | ||
| Got unknown frame | ignore | RxWaitF | OK | ||
| RxAccF | Decide how to accept Incoming File | Accept from beginning | Report receiving file | RxReceD | OK |
| Accept from offset (we do already have a part of file) | Send M_GET Report receiving file, requested offest | RxReceD | OK | ||
| Accept later (or failed to create file) | Send M_SKIP Report we will accept file later, not in current session | RxWaitF | OK | ||
| Refuse (delete on remote) | Send M_GOT Report we do not accept file | RxWaitF | OK | ||
| RxReceD | Get a frame from Input Buffer | Didn't got a complete frame yet | none | RxReceD | OK |
| Got Data frame | none | RxWriteD | continue | ||
| Got M_ERR | Report Error | RxDone | Failure | ||
| Got M_GET / M_GOT / M_SKIP | Add frame to The Queue | RxReceD | OK | ||
| Got M_NUL | Log | RxReceD | OK | ||
| Got M_FILE | Report partially received file | RxAccF | Continue | ||
| Got other known frame | Report unexpected frame | RxDone | Failure | ||
| Got unknown frame | ignore | RxReceD | OK | ||
| RxWriteD | Write data to file | Write Failed | Report error | RxDone | Failure |
| File Pos > Reported | Report write beyond EOF | RxDone | Failure | ||
| File Pos = Reported | Close File Send M_GOT Report File Received | RxWaitF | OK | ||
| File Pos < Reported | none | RxReceD | OK | ||
| RxEOB | Get a frame from Input Buffer | Didn't get a complete frame yet or TxState is not TxDone | none | RxEOB | OK |
| Got M_ERR | Report Error | RxDone | Failure | ||
| Got M_GET / M_GOT / M_SKIP | Add frame to The Queue | RxEOB | OK | ||
| Got M_NUL | Log | RxEOB | OK | ||
| Got other known frame or data frame | Report unexpected frame | RxDone | Failure | ||
| Got unknown frame | ignore | RxEOB | OK | ||
| RxDone | none | none | none | RxDone | OK |
We define the list called "PendingFiles". After we put the last byte of file into output buffer, we cannot yet consider the file as being successfully transmitted, thus we have to add the file to this list and then look for corresponding incoming M_GET / M_GOT / M_SKIP frames to remove the file from the list and decide whether the file was indeed received by remote or remote will accept this file later, or something else. After we have sent M_EOB frame, we must wait until PendingFiles list gets empty before disconnecting.
If the connection accidentally breaks, all the files left in PendingFiles are considered unsent and will be re-transmitted in the next session. If the connection breaks when the remote did actually receive the file (but the corresponded confirmation frame (M_GOT) didn't came back to us) and we are resending this file again in the next session, remote may get two copies of the same file (file dupe). Binkp allows to reduce or totally suppress such dupes (at a cost of performance, of course), see Non-Reliable mode and "No Dupes" protocol extension (to be found in a separate document at a later date).
| TxState | Predicate(s) | Condition(s) | Actions(s) | Next | Return |
|---|---|---|---|---|---|
| TxGNF | Open next file from outgoing queue | File opened OK | Send M_FILE Report sending file | TxTryR | continue |
| Failed to open file | Report failure | TxDone | Failure | ||
| No more files | Send M_EOB Report end of batch | TxWLA | continue | ||
| TxTryR | Check TheQueue | TheQueue is empty | none | TxReadS | continue |
| TheQueue is not empty | call ProcessTheQueue | continue | |||
| TxReadS | Read data block from file | Read failed | Report Error | TxDone | Failure |
| Read OK, Reached EOF | Send data block frame Close current file Add current file to PendingFiles | TxGNF | OK | ||
| Read OK, not reached EOF | Send data block frame | TxTryR | OK | ||
| TxWLA | Check TheQueue | TheQueue is empty and RxState >= RxEOB | none | TxDone | OK |
| TheQueue is empty and RxState < RxEOB | none | TxWLA | OK | ||
| TheQueue is not empty | call ProcessTheQueue | continue | |||
| TxDone | none | none | none | TxDone | OK |
We define a list called KnownFiles. This list contains files that can be requested by the remote using M_GET command. This list shall at least contain all the files that are part of the PendingFiles list.
| Predicate(s) | Condition(s) | Actions(s) |
|---|---|---|
| M_GET received | requested file is not in the KnownFiles list | Report unknown file |
| M_GET received for a known file | Requested pos is FileSize | Close and finalize file. Report that remote refused file being transmitted. Set TxState to TxGetNextFile. |
| Requested pos is less than FileSize | Set file pointer to requested pos. Report that remote requested offset. Set TxState to TxReadSend. |
|
| Requested pos is greater than FileSize | Ignore frame | |
| M_GOT file that is currently transmitting | none | Close and finalize file Report Remote refused file being transmitted Set TxState to TxGNF |
| M_GOT file that is not currently transmitting | File is in PendingFiles list | Finalize file Report file has been sent Remove file from the PendingFiles list |
| File is not in PendingFiles | Ignore frame | |
| M_SKIP file that is currently transmitting | none | Close file (do not finalize, we will send it later, not in current session) Report remote will accept this file later Set TxState to TxGNF |
| M_SKIP file that is not currently transmitting | none | Report remote will accept this file later Remove file from PendingPiles, if exists there |
A session may be terminated in any of the following cases:
A session termination itself is not a protocol stage. Mailer may terminate a session at any time simply by issuing disconnect (shutdown) command to the underlying transport layer, provided any of the three conditions above are met. Mailer MUST take all proper steps to provide a graceful shutdown of the transport layer, as it is the transport layer that is responsible for all the data transmitted by one side to be received by another before disconnection, provided that shutdown of the transport layer protocol was successful.
This section documents already implemented and proposed extensions for the binkp/1.0. These extensions are purely optional and are included here for the sake of compatibility with future implementations.
Sides indicate supported protocol extensions by sending M_NUL frame(s) with "OPT list_of_extensions" string, where list_of_extensions is a space separated list of supported protocol extensions. Whenever multiple M_NUL "OPT ..." frames are received during the session, they SHOULD augment the current list of extensions rather than replace it, unless specifically stated otherwise for a particular option.
Mailer SHOULD NOT use any extension unless exactly sure that this extension is supported by the remote. Mailer SHOULD use M_NUL "OPT ..." to indicate supported options. Other methods for indicating supported extensions are allowed as long as the provide full backwards compatibility.
Non-reliable mode solves the problem with frequently aborted connections when the sides can not successfully complete file transfer before connection is broken. In this case, if the transmitting side starts retransmission from offset 0, performance degrades as by the time it receives M_GET from the remote, network buffers are already full and by the time they are freed for retransmission from requested offset, the connection might go down again.
In order to circumvent this problem, a mailer can request the remote to enter non-reliable mode by sending a M_NUL "OPT NR" frame at any time during the session. After the remote acknowledges it by sending an M_NUL "OPT NR" frame indicating that the option is supported, both sides can assume that they are in non-reliable mode.
When session is in non-reliable mode, the transmitting side may send -1 for the offset value in M_FILE command. If it does so, it should wait for the M_GET frame from the receiving side that explicitly specifies file offset and start transmitting file data from this offset. If the receiving side has indicated that it supports non-reliable mode by sending M_NUL "OPT NR" frame, it must recognize -1 as the file offset in M_FILE command as an explicit request for the file offset and transmit an appropriate M_GET frame as soon as possible.
It should be understood that this option degrades performance over regular quality connections and it should be used only if absolutely necessary.
The session is in MB mode if both sides set "MB" flag in any of M_NUL "OPT" packets exchanged before sending of M_OK/M_PWD packets.
In MB mode both sides restart session from RxDone into InitTransfer state if there were any command packets sent or received by any side between starting at InitTransfer and exchanging of M_EOB by the sides (RxDone state). Otherwise, the session terminates as usual.
Multiple batches mode is intended to handle WaZOO [FTS-0006] file requests. If there were any WaZOO request files transferred in a batch, sides MAY process them and send resulting files in the next batch. Mailers MAY also generate list of files to send in additional batches by other techniques -- including rescanning of their spools or processing of other magic files transferred before in the same session.
Multiple password mode allows to specify different passwords for the different addresses of the remote.
Originating side identifies it's multipassword capabilities by sending M_NUL "OPT MPWD" during session setup stage before sending any M_ADR commands and waits for response from the answering side.
If answering side responds with the M_NUL "OPT MPWD", then it supports multiply passwords too. Answering side also always responds with it's own address list: M_ADR "adr1 adr2 adr3 ...". If M_NUL "OPT MPWD" was not received prior to the first M_ADR command, originating side should assume that the remote does not support multiple password mode and send a single password (if any) for one of the addresses of the remote.
If the MPWD option was indicated by the answering side, originating side now may send M_PWD "pwd1 pwd2 pwd3 ..." with the number of entries in space separated password list equivalent to the number of addresses presented by the answering side. If there is no password for a particular address, it must send '-' character as a placeholder.
If the passwords presented are consistent, answering side must acknowledge successful authentication by sending M_OK command.
Challenge-Response Authentication Mechanism (CRAM) allows to avoid passing cleartext, reusable passwords across the network. Since it utilizes Keyed-Hashing digests [Keyed], it does not require password to be stored in the clear on the Mailer's media, allowing storage of the intermediate results which are known as "contexts".
Providing binkp-mailer is capable of [Keyed] digest calculation and conversion of a byte array to a hexadecimal string and back, implementation of CRAM is easily achieved by slightly modifying the state machine.
CRAM adds an additional synchronization step to binkp protocol. The description of this step follows:
Similar technique is used in [IMAP-AUTH].
Size and contents of challenge data are implementation-dependent, but it SHOULD be no smaller than 8 bytes and no bigger than 64 bytes. Answering side SHOULD never generate the same challenge data.
Instead of generating a long challenge data, answering side MAY use a hash function to shorten it. In calculation of a challenge data answering side MAY also use connection/line number, caller's IP address, current time, etc.
Answering side transmits challenge data in the very first M_NUL message, in the following way:
M_NUL "OPT [othropt] CRAM-lsthf-cde [othropt]"lsthf is a list of aliases of supported hash functions, delimited by slash characters. The list begins with alias of the most preferred and ends with alias of the least preferred hash function.
Currently defined aliases are: MD5 for [MD5] and SHA1 for [SHA-1].
cde is the challenge data encoded to hexadecimal string, Lower-case ASCII characters MUST be used for encoding, but Mailer SHOULD also accept upper-case characters. The length of the string MUST be even, and the leading zeros MUST NOT be trimmed.
Originating side responds with:
M_PWD "CRAM-chosenhf-khde [othropt]"where chosenhf is the alias of the chosen hash function and khde is the keyed hashed digest, encoded to a hexadecimal string.
According to [IMAP-AUTH], keyed hashed digest is produced by calculating
HASH((secret XOR opad), HASH((secret XOR ipad), challengedata))where HASH is chosen hash function, ipad and opad are 36 hex and 5C hex (as defined in [Keyed]) and secret is a password null-padded to a length of 64 bytes. If the password is longer than 64 bytes, the hash-function digest of the password is used as an input (16-byte for [MD5] and 20-byte for [SHA-1]) to the keyed hashed calculation.
Answering side MUST send
M_NUL "OPT [othropt] CRAM-lsthf-cde [othropt]"as a very first M_NUL message if it supports CRAM.
It MAY send other non-M_NUL messages before though. Current specification doesn't define any such non-M_NUL message, they are reserved for protocol extension.
Originating side MUST be ready to receive non-M_NUL before M_NUL in a CRAM session. Binkp state machine MUST ignore any received message of unknown type in order to be compatible with future extensions.
If an originating side receives a first message that is a M_ADR or a M_NUL message that is not
M_NUL "OPT [othropt] CRAM-lsthf-cde [othropt]"it MUST decide that the answering side doesn't support CRAM and MAY either disconnect or use old password exchange. If the sides have no any compatible hash function, originator may also either disconnect or use old password exchange. If an originating side decides to disconnect, it SHOULD send M_ERR frame with a proper explanation before disconnecting.
When parsing M_NUL "OPT ..." string (coming from the answering side), originating side first splits it by using space delimiter to get a list of options, and then if an option begins with "CRAM-lsthf-", takes the remaining substring as a hexadecimal-encoded challenge data.
Originating :
send M_NUL messages
and M_ADR
wait for first M_NUL message
Answering :
send M_NUL "OPT ND CRAM-SHA1/MD5-f0315b074d728d483d6887d0182fc328"
and other messages
wait for M_PWD
Originating :
M_PWD "CRAM-MD5-56be002162a4a15ba7a9064f0c93fd00"
Answering :
M_OK and continue session
[MD5] and [SHA-1] are the most widely used cryptographic hash functions. [MD5] has been shown to be vulnerable to collision search attacks [Dobb]. This attack and other currently known weaknesses of [MD5] do not compromise the use of [MD5] within CRAM as specified in this document (see [Dobb]); however, [SHA-1] appears to be a cryptographically stronger function. To this date, [MD5] can be considered for use in CRAM for applications where the superior performance of [MD5] is critical. In any case, implementors and users need to be aware of possible cryptanalytic developments regarding any of these cryptographic hash functions, and the eventual need to replace the underlying hash function.
You can implement binkp protocol in your software as long as you agree to the following conditions:
Remember that you may use, implement or utilize binkp, it's description or any other associated texts or documentations at your own risk, without any warranty, without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE
Binkp author: Dima Maloff.
Many entries in this glossary are provided courtesy of Butterfly Glossary of Internet and Data Communication terms and RFC-1983.
This document is partially based on extracts from RFCs and FTSC publications too numerous to be acknowledged individually.
The authors would like to thank Joaquim Homrighausen, Kim 'B' Heino, Rune Johansen and many others for fruitful discussions and suggestions regarding protocol design and specifications.
Dima Maloff
Fidonet: 2:5020/128
E-mail: maloff@corbina.net
WWW: http://www.corbina.net/~maloff/
Maxim Masiutin
Fidonet: 2:469/84
E-mail: MAX at RITLABS dot COM
WWW: http://www.ritlabs.com/
Nick Soveiko
Fidonet: 2:5030/23.101
E-mail: nsoveiko@doe.carleton.ca
WWW: http://www.doe.carleton.ca/~nsoveiko/