Node¶

class Node.Node(container, node_address)¶

class Successor(finger_table_ref)¶

List manager for successor references.

It is responsible that entries in the successor list and first finger are consistent.

Node.check_predecessor()¶

Verifies this node’s immediate predecessor’s live.

If it is lost, remove reference to give new nodes a chance to repair it.

Node.find_successor(node_id, with_neighbors=False)¶

Wrapper for find_successor_rec() to clean responses.

Parameters:	node_id – Key node_id whose responsible successor is interesting. with_neighbors – If True, the immediate successor and predecessor nodes augment the result of the responsible successor.
Returns:	Responsible successor node for given key node_id.
Return type:	dict or None

Node.find_successor_rec(node_id, with_neighbors=False, tracing=False)¶

Recursively locate the responsible node for a given node_id (key).

This function is the heart of the Chord DHT. It is used locally and by remote peers.

Parameters:	node_id – Key node_id whose responsible successor is interesting. with_neighbors – If True, the immediate successor and predecessor nodes augment the result of the responsible successor. This is useful if the predecessor of the responsible node is needed.
Returns:	Responsible successor node for given key node_id.

Node.find_successor_trace(node_id)¶

Wrapper for find_successor_rec() with trace log enabled for intermediate hops.

Parameters:	node_id – Key node_id whose responsible successor is interesting.
Returns:	Responsible successor node for given key node_id.
Return type:	dict or None

Node.fix_finger(finger_id=-1)¶

Resolves the responsible node for the given finger and updates it accordingly.

Parameters:	finger_id – index of the finger table to update. The value should be between 0 and length of the finger table.

static Node.generate_key(address)¶

Generates a node identifier (key) based on the network address for this instance.

Parameters:	address – The network address as string
Returns:	Generated node id
Return type:	int

Node.get_closest_preceding_finger(node_id, fall_back=0, start_offset=255)¶

Find closest preceding finger within m -> 0 fingers.

Parameters:	node_id – node ID as an integer. fall_back – chooses less optimal finger nodes if value increases. This allows to find a slower, but still working lookup although the best matching finger is not responding anymore. In the worst case, this function falls back to this node itself. For example, this is the case if our immediate successor is responsible for all of our fingers, but does not respond to requests done previously.
Returns:	returns the interesting node descriptor as a dictionary with successor and predecessor.
Return type:	dict

Node.get_trace(key)¶

Information about the hops involved in the path for the lookup of the given key.

The list is in reverse order: The target peer is at index 0. The node that started the request, is at the last position.

Parameters:	key – Node ID to lookup.
Returns:	Array with dicts containing the address information of all involved hops.

Node.init_finger_table()¶

Generates a basic finger table for this node joining an existing Chord network.

Node.init_successor_list(successor)¶

Fetch successor list from our immediate successor when joining a network.

Node.join(node_id=None, node_address=None, bootstrap_address=None, additional_data=None)¶

Joins an existing Chord network or creates a new one.

It set ups all internal state variables needed for operation. Needs to be called previously to any other function or RPC call.

Parameters:

node_id – Optional node ID. If not supplied, it will be generated automatically. node_address – Optional node address formatted as an aiomas agent address (IPv4 or IPv6 address) bootstrap_address – If not given, a new Chord network is created. Otherwise, the new node will gather the required information to integrate into the Chord network. additional_data – Optional additional data as dict that is added to the trace log if a node calls find_successor_rec() with tracing enabled.

Node.stabilize()¶

Stabilize routine

Node.update_neighbors(initialization=False)¶

Update immediate neighbors.

Update our successor’s pointer to reference us as immediate predecessor (according to default Chord specification). Notify our direct predecessor about our presence. This allows to early stabilize its immediate successor finger[0].

Requires that finger[0] is set properly.

Node.update_others()¶

Update peers’ finger table that should refer to our node and notify them.

Node.update_successor(new_node)¶

Updates the reference to our immediate successor triggered by other peer’s hint.

A neighboring successor uses this function to notify us about its presence. This ensures that the Chord ring is correct. The parameter new_node gives a hint about the new successor in this case. To verify this hint, this node contacts its old successor.

Parameters:	new_node – Successor hint.

Node.update_successor_list()¶

Periodically checks availability of our successor peer, maintains a list of possible successors and swaps to another successor if the first fails.

storage¶

The Storage class manages the data structure for DHT_PUT and DHT_GET operations. It automatically deletes items when its timeToLive has expired and supports multiple entrys per key.

class storage.Storage¶

Manage PUT and GET operations

clean_old()¶

Clean old items where the time to live has expired.

delete_storage_data_between(keyOldPredecessor, keyNewPredecessor)¶

Delete a set of storage items between two nodes. Ususally called if a node has sent some of its content to another node

Parameters:	keyOldPredecessor – the key of the old predecessor keyNewPredecessor – the key of the new predecessor

get(key)¶

Returns a list of elements for the given key.”

Parameters:	key – the dht key
Returns:	list of values
Return type:	list
Todo:	Add support for binary keys and values
Example:	storage = Storage() storage.put("penguin", "tux") storage.put("penguin", "linus") storage.get("penguin") # returns ["tux", "linus"]

get_storage_data_between(keyOldPredecessor, keyNewPredecessor)¶

Returns a set of storage items between two nodes.

Parameters:	keyOldPredecessor – the key of the old predecessor keyNewPredecessor – the key of the new predecessor
Returns:	list of data storage items
Return type:	list

merge(dataToMerge)¶

Merge another storage to this storage

Parameters:	dataToMerge (list) – the dht key
Example:

put(key, value, ttl=43200, timeOfInsert=None)¶

Add data to the storage. You can also provide a time to live and specify the time of insert.

Parameters:	key – the dht key value – the dht value ttl – time to live in seconds. after this period of seconds the item will be deleted timeOfInsert – datetime when the item was inserted. If none is set, we will use the current date. Will be used later to check outdated items.
Todo:	Add support for binary keys and values
Example:	See example of get() method.

timeDiff(timeStart, timeStop, ttl)¶

Checks if an item inserted at timeStart with a given ttl is still valid at timestop (ttl has not expired at timeStop)

Parameters:	timeStart – time when the item was inserted timeStop – time when the item is checked ttl (int) – time to live in seconds
Type:	timeStart: datetime
Type:	timeStop: datetime
Returns:	True if valid or false if invalid
Return type:	Boolean

messageParser¶

The message parser parses and generates binary messages to communicate with other modules like the KX module.

class messageParser.DHTHop(peerId, kxPort, IPv4Address, IPv6Address)¶

A DHT Hop for MSG_DHT_TRACE_REPLY message

Parameters:	peerId (Int) – the peer id with a maximum length of 32 bytes kxPort (Int) – the kx port with a maxmimum length of 2 bytes IPv4Address (String) – For example 192.168.0.1 IPv6Address (String) – For example FE80:0000:0000:0000:0202:B3FF:FE1E:8329

class messageParser.DHTMessage¶

Base class for other classes representing incoming data such as as DHTMessagePUT

getSize()¶

Returns the size of the message

Returns:	Message Size in bytes
Return type:	int

parse()¶

Parse the message

self.message will automatically become the type of message specified with the command number (DHTMessageGET, DHTMessagePUT etc.)

read_binary(data)¶

Read and parse binary data

Parameters:	data (bytearray) – The data

read_file(filename)¶

Read a binary file representing a message. The message is automatically parsed afterwards.

Parameters:	filename – the location of the file

class messageParser.DHTMessageERROR(requestType, requestKey)¶

Generates an error message

Parameters:	requestType (int) – The type of request requestKey (int) – The key

get_data()¶

Return the binary representation

Returns:	Message in binary format
Return type:	bytearray

class messageParser.DHTMessageGET(data, size)¶

Provides additional parameters for a DHTMessage which is a GET message.

get_key()¶

Returns the key as integer

Return type:	int

class messageParser.DHTMessageGET_REPLY(key, content)¶

Initializes a MSG_DHT_GET_REPLY message to send later.

Parameters:	key – the key as integer content – the content of the get query in binary format.

get_data()¶

Returns the data in binary format

Return type:	bytearray

class messageParser.DHTMessagePUT(data, size)¶

Provides additional parameters for a DHTMessage which is a PUT message

get_content()¶

Returns the content

Returns:	content
Return type:	bytearray

get_key()¶

Returns the key as integer

Return type:	int

get_replication()¶

Returns the replication

Return type:	int

get_ttl()¶

Returns the time to live (ttl) in seconds

Return type:	int

class messageParser.DHTMessageTRACE(data, size)¶

Provides additional parameters for a DHTMessage which is a TRACE message.

get_key()¶

Returns the key as integer

Return type:	int

class messageParser.MAKE_MSG_DHT_GET(key)¶

Initializes a MSG_DHT_GET` message to send later.

Parameters:	key – the key as integer hops – a list of DHTHop() objects

class messageParser.MAKE_MSG_DHT_PUT(key, content, ttl=43200, replication=3)¶

Initializes a MSG_DHT_PUT message to send later.

Parameters:	key (int) – key as integer content (bytearray) – The content to be stored ttl (int) – time the content is available in seconds (43200 per default) replication (int) – The amount of replication. A replication degree of three means a tripple redundancy. If one node crashes, there are still two nodes available for example.

class messageParser.MAKE_MSG_DHT_TRACE_REPLY(key, hops)¶

Initializes a MSG_DHT_TRACE_REPLY message to send later.

Parameters:	key – the key as integer hops – a list of DHTHop() objects

get_data()¶

Return the binary representation

Returns:	Message in binary format
Return type:	bytearray

iniParser¶

Helper class to parse the ini files.

Please note that there is also a python module named configparser. However, as the example ini module contanined no section header for the first entry (HOSTKEY), we wrote our own

class iniParser.IniParser(filename)¶

Initializes a DHT_TRACE_REPLY message to send later.

Parameters:	filename (string) – The filename. Example: “config.ini”

get(attribute, section='')¶

get an attribute of an ini file

Parameters:	attribute (string) – The attribute section – The section. Default is no section
Example:	#test.ini: #[DHT] #PORT = 123 inip = IniParser("test.ini") inip.get("PORT", "DHT") # returns 123

read_file(filename)¶

Open a new file and parse it :param filename: The filename :type filename: string

replica¶

class replica.Replica(chordRingSize, replicationCount=3)¶

Parameters:	chordRingSize – The Size of the Chord ring. Usually something like 2^256

get_key(key, replicaIndex=1)¶

Set default attribute values only

Parameters:	key – the key to be hased as byte array
Returns:	a hash value which is the location in the chord ring
Return type:	int

get_key_list(key, replicationCount=-1)¶

Get a list of replica keys for a given replica key

Parameters:	key (bytearray) – the key to be hased as byte array
Returns:	a list of integer replica keys as
Return type:	list of type int

ipc¶

class ipc.ApiServer(dht_node)¶

Class to connect to the Chord Node

data_received(message)¶

Parses and executes API calls received from an external client. A dedicated asyncio task will be spawned for each incoming request.

Parameters:	message – raw message received.

dhtQuery¶

This is a console tool made to put and get data from the DHT. You can start and use it with ./dhtQuery.py. Keys need to be provided as integers and are converted to big endian. Values are converted to strings using utf8 encoding.