izaya/OC-Copper
1
0
Fork 0
mirror of https://github.com/XeonSquared/OC-Copper.git synced 2024-11-23 01:48:06 +11:00
Code Issues Releases Wiki Activity

Remove the fragmentation part of the reliablility layer

Browse Source 
In practice it is unlikely that data that's too big for the reliability layer won't need splitting
 anyway by the application at some point, and the libraries are already too open to OOM DoS.
This commit is contained in:
gamemanj 2017-03-18 15:31:25 +00:00
parent e77c203514
commit aabbb554c6
8 changed files with 420 additions and 111 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
LICENSE
View File

@ -1 +1,2 @@
I, 20kdc, release this into the public domain.
No warranty is provided, implied or otherwise.

9
culib.lua
View File

@ -1,3 +1,6 @@
-- I, 20kdc, release this into the public domain.
-- No warranty is provided, implied or otherwise.
-- 'Copper' networking test implementation.
-- This is meant as a portable (even into OC) library for networking.
-- This 'outer function' is the instantiator.
@ -36,8 +39,6 @@ return function (hostname, transmit, onReceive, time)
-- This is not a complete clear.
local tuningFlushLoopDetector = 60
local tuningRandomPathwarming = 0.1
-- Do not change this value unless protocol has changed accordingly.
local tuningAutorejectLen = 1506
@ -80,10 +81,6 @@ return function (hostname, transmit, onReceive, time)
for k, v in pairs(lastKnownReceiver) do
if t >= v[2] then
--print("It was decided LKV[" .. k .. "] was out of date @ " .. v[2] .. " by " .. hostname)
-- Keep the transmission path 'warm' with a null packet
if math.random() < tuningRandomPathwarming then
transmit(nil, "\xFF" .. encodeName(hostname) .. encodeName(k))
end
lastKnownReceiver[k] = nil
end
end

14
gennet.lua
View File

@ -1,3 +1,5 @@
-- I, 20kdc, release this into the public domain.
-- No warranty is provided, implied or otherwise.
-- Generate connected network where all nodes are connected.
-- Saves graph to "netref.dot", outputs lua tables to stdout.
@ -8,7 +10,11 @@ local wordsA = {
"intriguing",
"bright",
"solitudial",
"nuanced"
"nuanced",
"confused",
"confuzzled",
"inspiring",
"dark",
}
local wordsB = {
"fontaine",
@ -16,7 +22,11 @@ local wordsB = {
"poirot",
"pinkie",
"sparks",
"twi"
"twi",
"edgar",
"edith",
"author",
"entity",
}
for i = 1, #wordsA do
for j = 1, #wordsB do

33
protocol.0
View File

@ -43,6 +43,8 @@ Should a situation be dire enough,
The Broadcast Address is a possible feature which may or may not be actually used.
For now it is not implemented.
The idea is that if a name is directly equal to "*", it should be broadcast around the local network.
Notably, it is completely incompatible with reliability mechanisms,
but can still be used in networks running the Reliability Layer via the 'unreliable' escape-hatch.
Hierarchial gateways do not need modification on the from-child rules
("*" is local to them there,
@ -50,3 +52,34 @@ Hierarchial gateways do not need modification on the from-child rules
but in the from-parent rules it may be desirable to forward "*" to child networks.
Or not.
--- OpenComputers Usage
Copper should be used on port 4957.
Messages should be two part.
The first part should always be "copper",
the second part should always be the actual data.
All available methods of communication should be open to Copper unless explicitly noted otherwise -
for example, Linked Cards are one of the methods that could be used for purpose-built wireless links.
--- Overlay Network Usage
In this case, Copper messages may be relayed however the users see fit - but port 4957 UDP messages
are a good fit on the basis that they are a parallel.
To prevent annoyed ISPs, the best networking method is a looped daisy-chain.
This provides a bit of redundancy while avoiding ISP-ire-causing mass fanout.
Network messages should be accepted from any IP address, as spoofing is perfectly doable anyway.
--- Serial Port/TCP Usage
Copper packets should just be written out on the connection, prefixed with a big-endian two-byte
unsigned integer specifying the size.
--- Hybrid Usage
The connection between OpenComputers and Copper can be performed with a TCP gateway between the
Minecraft server and Copper server on the same computer, or via designated Copper HTTP event feeds
of some form, or whatever.
It doesn't really matter how the data is transmitted here, since it's just a gateway.
Possibly consider integrating a hierarchial gateway into the OpenComputers/world bridge.

64
protocol.2
View File

@ -1,4 +1,4 @@
Copper Protocol :: Reliability/Fragmentation Layer
Copper Protocol :: Reliability Layer
20kdc, 2017
The Copper Protocol as described in files 1 and 2 does not have any
@ -12,13 +12,12 @@ For applications, however, a protocol must be layered on top.
This document on the Reliability Layer describes how that should work.
All implementations of Copper that synthesize their own packets SHOULD
follow this protocol when doing so, unless they are a custom system
that will not be connected to any global network.
Firstly, note that, to the application, a Reliability Layer packet can
be up to 22,500 bytes in size, though a fragment can only be up to 1500 bytes.
only be up to 1500 bytes precisely. This value does not change.
Secondly, note that an application should be able to ask to be notified
when a packet is received successfully or when the implementation gives up,
@ -35,58 +34,25 @@ The next byte is the 'attempt number' - the amount of attempts by this
This can be achieved serially or otherwise, but should have a random base.
Combined with correctly-forgetting packet caches, this should prevent
any packets lost by data collision.
The final header byte is the actual indicator of what is in the packet.
The upper nibble indicates the amount of fragments in the packet - 0
indicates an acknowledgement.
The lower nibble indicates which fragment this is, or if this is an
acknowledgement, which fragment was acknowledged.
0x00 indicates that this is an unreliable packet.
0x01 indicates that this is a reliable packet, expecting acknowledgement.
0x02 indicates that this is an acknowledgement for a reliable packet.
0x0F indicates that this is a *deliberately* unreliable packet.
(These packets cannot be fragmented or acknowledged, and thus have the
per-fragment limit of 3993 bytes.
The attempt number and primary packet number still have meaning.)
Two example scenarioes will now be presented:
An example scenario will now be presented:
1.
ARCHWAYS sends a 0x10 'First fragment of a one fragment packet' to
IWAKURA on port 8080, twice (the first attempt being dropped).
1F 90 | F4 21 B9 | 00/01 | 10 | (...)
port packetID Attempt CC Data
Alice sends a 0x01 reliable packet to Bob on port 8080,
twice (the first attempt being dropped).
1F 90 | F4 21 B9 | 00/01 | 01 | (...)
port packetID Attempt PT Data
IWAKURA receives it successfully on the second time, and sends back a
Bob receives it successfully on the second time, and sends back a
response, three times.
1F 90 | F4 21 B9 | 00/01/02 | 00
port packetID Attempt CC
1F 90 | F4 21 B9 | 00/01/02 | 02
port packetID Attempt PT
ARCHWAYS receives the response and does not send a third packet.
2.
IWAKURA, having parsed the packet, sends back a long response on the same port.
The response is two packets long.
1F 90 | 91 19 28 | 00 | 20 | (...)
1F 90 | 91 19 28 | 00 | 21 | (...)
ARCHWAYS receives both packets, in the wrong order
(but it reassembles it anyway), and ACKs three times...
...but the packets are dropped due to a crow getting in the way of the
satellite dish at the wrong point. Blasted crow.
1F 90 | 91 19 28 | 00/01/02 | 21
1F 90 | 91 19 28 | 00/01/02 | 20
IWAKURA, waiting, say, 6 seconds
(assuming ACKs are sent a second and a half apart) sends a retransmission.
1F 90 | 91 19 28 | 01 | 20 | (...)
1F 90 | 91 19 28 | 01 | 21 | (...)
ARCHWAYS ACKs the retransmission, just in case - this works.
1F 90 | 91 19 28 | 00/01/02 | 21
1F 90 | 91 19 28 | 00/01/02 | 20
IWAKURA's application knows the message got through.
Alice receives the response and does not send a third packet.

50
relib.lua Normal file
View File

@ -0,0 +1,50 @@
-- I, 20kdc, release this into the public domain.
-- No warranty is provided, implied or otherwise.
-- Copper Reliability Layer
-- Notably, this should be instantiated rather than the normal Copper instance.
local culib = require("culib")
-- onRReceive is now: (from, to, port, data, unreliablePacket)
-- where to can be anything for unreliable packets, but otherwise is the current hostname.
return function (hostname, transmit, onRReceive, time)
-- node.hostname should be used for hostname generally.
local node
local onReceive = function (nfrom, nto, data)
if data:len() < 6 then return end
local port = data:byte(2) + (data:byte(1) * 256)
if data:byte(7) == 0x0F then
onRReceive(nfrom, nto, port, data, true)
return
end
if nto ~= node.hostname then
return
end
end
node = culib(hostname, transmit, onReceive, time)
-- Just an array, no special index.
-- Contents : {
-- trigger = function(),
-- expiry = time,
-- }
local timers = {}
local relib = {}
relib.refresh = function ()
node.refresh()
local i = 1
local t = time()
while i <= #timers do
if timers[i].expiry < t then
table.remove(timers, i)
else
i = i + 1
end
end
end
relib.input = node.input
relib.output = function ()
end
end

11
runtest.lua
View File

@ -1,3 +1,6 @@
-- I, 20kdc, release this into the public domain.
-- No warranty is provided, implied or otherwise.
-- Load testnet
local culib = require("culib")
@ -99,9 +102,9 @@ end
print(#nodes, statSD, statPT)
local maxMisses, maxHits = 0, 0
local allMisses, allHits = 0, 0
for i = 1, #nodes do
maxMisses = math.max(maxMisses, nodes[i].lkrCacheMisses)
maxHits = math.max(maxHits, nodes[i].lkrCacheHits)
allMisses = allMisses + nodes[i].lkrCacheMisses
allHits = allHits + nodes[i].lkrCacheHits
end
print(maxMisses, maxHits)
print(allMisses, allHits)

349
testnet.lua
View File

@ -5,89 +5,338 @@ return function (declare, connect)
declare("changing_pinkie")
declare("changing_sparks")
declare("changing_twi")
declare("changing_edgar")
declare("changing_edith")
declare("changing_author")
declare("changing_entity")
declare("ponderous_fontaine")
declare("ponderous_marple")
declare("ponderous_poirot")
declare("ponderous_pinkie")
declare("ponderous_sparks")
declare("ponderous_twi")
declare("ponderous_edgar")
declare("ponderous_edith")
declare("ponderous_author")
declare("ponderous_entity")
declare("intriguing_fontaine")
declare("intriguing_marple")
declare("intriguing_poirot")
declare("intriguing_pinkie")
declare("intriguing_sparks")
declare("intriguing_twi")
declare("intriguing_edgar")
declare("intriguing_edith")
declare("intriguing_author")
declare("intriguing_entity")
declare("bright_fontaine")
declare("bright_marple")
declare("bright_poirot")
declare("bright_pinkie")
declare("bright_sparks")
declare("bright_twi")
declare("bright_edgar")
declare("bright_edith")
declare("bright_author")
declare("bright_entity")
declare("solitudial_fontaine")
declare("solitudial_marple")
declare("solitudial_poirot")
declare("solitudial_pinkie")
declare("solitudial_sparks")
declare("solitudial_twi")
declare("solitudial_edgar")
declare("solitudial_edith")
declare("solitudial_author")
declare("solitudial_entity")
declare("nuanced_fontaine")
declare("nuanced_marple")
declare("nuanced_poirot")
declare("nuanced_pinkie")
declare("nuanced_sparks")
declare("nuanced_twi")
connect(1, 3)
connect(1, 9)
connect(2, 20)
connect(3, 17)
connect(3, 35)
connect(4, 5)
connect(4, 19)
connect(5, 29)
connect(6, 15)
connect(6, 22)
connect(6, 25)
connect(6, 32)
connect(7, 10)
connect(7, 24)
connect(8, 18)
connect(8, 33)
connect(8, 36)
connect(9, 17)
connect(9, 33)
connect(9, 35)
connect(10, 20)
declare("nuanced_edgar")
declare("nuanced_edith")
declare("nuanced_author")
declare("nuanced_entity")
declare("confused_fontaine")
declare("confused_marple")
declare("confused_poirot")
declare("confused_pinkie")
declare("confused_sparks")
declare("confused_twi")
declare("confused_edgar")
declare("confused_edith")
declare("confused_author")
declare("confused_entity")
declare("confuzzled_fontaine")
declare("confuzzled_marple")
declare("confuzzled_poirot")
declare("confuzzled_pinkie")
declare("confuzzled_sparks")
declare("confuzzled_twi")
declare("confuzzled_edgar")
declare("confuzzled_edith")
declare("confuzzled_author")
declare("confuzzled_entity")
declare("inspiring_fontaine")
declare("inspiring_marple")
declare("inspiring_poirot")
declare("inspiring_pinkie")
declare("inspiring_sparks")
declare("inspiring_twi")
declare("inspiring_edgar")
declare("inspiring_edith")
declare("inspiring_author")
declare("inspiring_entity")
declare("dark_fontaine")
declare("dark_marple")
declare("dark_poirot")
declare("dark_pinkie")
declare("dark_sparks")
declare("dark_twi")
declare("dark_edgar")
declare("dark_edith")
declare("dark_author")
declare("dark_entity")
connect(1, 42)
connect(1, 44)
connect(1, 65)
connect(1, 93)
connect(1, 99)
connect(2, 25)
connect(2, 56)
connect(3, 7)
connect(4, 16)
connect(4, 54)
connect(4, 66)
connect(4, 75)
connect(5, 45)
connect(5, 63)
connect(6, 16)
connect(6, 53)
connect(6, 100)
connect(7, 21)
connect(7, 46)
connect(7, 69)
connect(7, 95)
connect(8, 10)
connect(8, 16)
connect(8, 53)
connect(8, 64)
connect(8, 88)
connect(8, 89)
connect(8, 96)
connect(9, 53)
connect(9, 54)
connect(10, 14)
connect(10, 34)
connect(10, 39)
connect(10, 69)
connect(10, 87)
connect(11, 13)
connect(11, 23)
connect(11, 27)
connect(11, 28)
connect(12, 29)
connect(11, 65)
connect(11, 68)
connect(12, 30)
connect(13, 24)
connect(13, 25)
connect(13, 28)
connect(13, 33)
connect(14, 19)
connect(14, 28)
connect(15, 21)
connect(15, 30)
connect(15, 31)
connect(15, 33)
connect(16, 24)
connect(16, 32)
connect(16, 34)
connect(12, 37)
connect(12, 59)
connect(12, 72)
connect(13, 50)
connect(13, 69)
connect(13, 80)
connect(13, 89)
connect(14, 37)
connect(14, 73)
connect(14, 81)
connect(14, 90)
connect(15, 61)
connect(15, 89)
connect(16, 41)
connect(16, 49)
connect(16, 51)
connect(16, 54)
connect(16, 74)
connect(17, 28)
connect(17, 40)
connect(17, 48)
connect(17, 69)
connect(17, 75)
connect(17, 84)
connect(17, 91)
connect(17, 93)
connect(18, 23)
connect(18, 36)
connect(19, 23)
connect(19, 25)
connect(18, 25)
connect(19, 28)
connect(19, 31)
connect(22, 35)
connect(23, 26)
connect(24, 31)
connect(19, 33)
connect(19, 35)
connect(19, 44)
connect(19, 60)
connect(19, 62)
connect(19, 69)
connect(19, 72)
connect(19, 89)
connect(20, 62)
connect(20, 67)
connect(20, 92)
connect(21, 41)
connect(21, 52)
connect(21, 75)
connect(21, 87)
connect(21, 100)
connect(22, 48)
connect(22, 49)
connect(22, 100)
connect(23, 31)
connect(23, 45)
connect(23, 49)
connect(23, 63)
connect(23, 66)
connect(23, 90)
connect(24, 30)
connect(24, 33)
connect(24, 39)
connect(24, 83)
connect(24, 84)
connect(24, 98)
connect(25, 33)
connect(26, 34)
connect(30, 34)
connect(31, 33)
connect(33, 35)
connect(34, 35)
connect(25, 59)
connect(25, 65)
connect(26, 61)
connect(26, 66)
connect(26, 78)
connect(27, 31)
connect(27, 54)
connect(27, 59)
connect(27, 65)
connect(27, 88)
connect(28, 56)
connect(29, 36)
connect(29, 37)
connect(29, 55)
connect(29, 62)
connect(29, 67)
connect(29, 74)
connect(29, 78)
connect(30, 50)
connect(30, 62)
connect(30, 78)
connect(30, 83)
connect(30, 91)
connect(31, 49)
connect(31, 66)
connect(31, 74)
connect(31, 79)
connect(31, 89)
connect(31, 100)
connect(32, 44)
connect(32, 66)
connect(32, 70)
connect(32, 71)
connect(33, 74)
connect(33, 94)
connect(34, 50)
connect(34, 77)
connect(34, 83)
connect(34, 85)
connect(34, 99)
connect(35, 60)
connect(35, 82)
connect(35, 90)
connect(36, 56)
connect(36, 64)
connect(36, 69)
connect(36, 88)
connect(37, 52)
connect(37, 60)
connect(37, 72)
connect(37, 75)
connect(38, 77)
connect(39, 69)
connect(39, 78)
connect(40, 66)
connect(40, 82)
connect(40, 83)
connect(40, 85)
connect(41, 52)
connect(41, 90)
connect(42, 56)
connect(42, 70)
connect(43, 56)
connect(43, 94)
connect(44, 65)
connect(44, 93)
connect(44, 94)
connect(45, 70)
connect(45, 89)
connect(46, 58)
connect(46, 79)
connect(46, 95)
connect(47, 82)
connect(47, 85)
connect(48, 60)
connect(48, 63)
connect(49, 68)
connect(50, 58)
connect(50, 61)
connect(50, 67)
connect(50, 77)
connect(50, 87)
connect(50, 98)
connect(52, 67)
connect(52, 84)
connect(52, 98)
connect(53, 58)
connect(53, 61)
connect(53, 64)
connect(53, 77)
connect(54, 76)
connect(54, 85)
connect(55, 80)
connect(56, 92)
connect(57, 71)
connect(58, 60)
connect(58, 76)
connect(58, 88)
connect(59, 96)
connect(60, 62)
connect(61, 63)
connect(62, 65)
connect(63, 75)
connect(63, 84)
connect(63, 89)
connect(63, 91)
connect(64, 68)
connect(64, 72)
connect(64, 73)
connect(64, 76)
connect(66, 77)
connect(66, 86)
connect(66, 97)
connect(67, 88)
connect(68, 81)
connect(69, 81)
connect(69, 92)
connect(69, 93)
connect(70, 77)
connect(71, 99)
connect(72, 79)
connect(73, 84)
connect(73, 87)
connect(73, 94)
connect(74, 80)
connect(75, 98)
connect(78, 79)
connect(79, 80)
connect(81, 92)
connect(82, 84)
connect(82, 92)
connect(84, 88)
connect(84, 93)
connect(86, 91)
connect(88, 91)
connect(91, 99)
connect(92, 95)
connect(92, 96)
connect(93, 96)
connect(94, 99)
end