Difference between revisions of "Protocol specification"

From Bitmessage Wiki
Jump to navigation Jump to search
m (→‎msg: Removed todo and added link to Encrypted payload description)
(→‎Pubkey bitfield features: Reassign bit 29 from extended_encoding to chat)
 
(53 intermediate revisions by 8 users not shown)
Line 1: Line 1:
 +
<span style="color:red">All objects sent on the network should support protocol v3 starting on Sun, 16 Nov 2014 22:00:00 GMT. </span>
 +
 
==Common standards==
 
==Common standards==
  
Line 14: Line 16:
  
 
  hello
 
  hello
−
  9b71d224bd62f3785d96d46ad3ea3d73319bfbc2890caadae2dff72519673ca72323c3d99ba5c11d7c7acc6e14b8c5da0c4663475c2e5c3adef46f73bcdec043(first round is sha-256)
+
  9b71d224bd62f3785d96d46ad3ea3d73319bfbc2890caadae2dff72519673ca72323c3d99ba5c11d7c7acc6e14b8c5da0c4663475c2e5c3adef46f73bcdec043(first round is sha-512)
 
  79a324faeebcbf9849f310545ed531556882487e (with ripemd-160)
 
  79a324faeebcbf9849f310545ed531556882487e (with ripemd-160)
  
Line 30: Line 32:
 
| 12 || command || char[12] || ASCII string identifying the packet content, NULL padded (non-NULL padding results in packet rejected)
 
| 12 || command || char[12] || ASCII string identifying the packet content, NULL padded (non-NULL padding results in packet rejected)
 
|-
 
|-
−
| 4 || length || uint32_t || Length of payload in number of bytes
+
| 4 || length || uint32_t || Length of payload in number of bytes. Because of other restrictions, there is no reason why this length would ever be larger than 1600003 bytes. Some clients include a sanity-check to avoid processing messages which are larger than this.
 
|-
 
|-
 
| 4 || checksum || uint32_t || First 4 bytes of sha512(payload)
 
| 4 || checksum || uint32_t || First 4 bytes of sha512(payload)
 
|-
 
|-
−
| ? || payload || uchar[] || The actual data, a [[#Message_types|message]] or an [[#Object_types|object]]
+
| ? || message_payload || uchar[] || The actual data, a [[#Message_types|message]] or an [[#object|object]]. Not to be confused with objectPayload.
 
|}
 
|}
  
Line 47: Line 49:
 
=== Variable length integer ===
 
=== Variable length integer ===
  
−
Integer can be encoded depending on the represented value to save space.  Variable length integers always precede an array/vector of a type of data that may vary in length.
+
Integer can be encoded depending on the represented value to save space.  Variable length integers always precede an array/vector of a type of data that may vary in length. Varints MUST use the minimum possible number of bytes to encode a value. For example; the value 6 can be encoded with one byte therefore a varint that uses three bytes to encode the value 6 is malformed and the decoding task must be aborted.
  
 
{|class="wikitable"
 
{|class="wikitable"
Line 54: Line 56:
 
| < 0xfd || 1 || uint8_t
 
| < 0xfd || 1 || uint8_t
 
|-
 
|-
−
| <= 0xffff || 3 || 0xfd followed by the length as uint16_t
+
| <= 0xffff || 3 || 0xfd followed by the integer as uint16_t
 
|-
 
|-
−
| <= 0xffffffff || 5 || 0xfe followed by the length as uint32_t
+
| <= 0xffffffff || 5 || 0xfe followed by the integer as uint32_t
 
|-
 
|-
−
| - || 9 || 0xff followed by the length as uint64_t
+
| - || 9 || 0xff followed by the integer as uint64_t
 
|}
 
|}
  
Line 91: Line 93:
 
=== Network address ===
 
=== Network address ===
  
−
When a network address is needed somewhere, this structure is used.  This protocol and structure supports IPv6, '''but note that the original client currently only supports IPv4 networking'''. Network addresses are not prefixed with a timestamp or stream in the version message.
+
When a network address is needed somewhere, this structure is used.  Network addresses are not prefixed with a timestamp or stream in the version message.
  
 
{|class="wikitable"
 
{|class="wikitable"
 
! Field Size !! Description !! Data type !! Comments
 
! Field Size !! Description !! Data type !! Comments
 
|-
 
|-
−
| 4 (or 8)|| time || uint32 || the Time. Protocol version 1 clients use 4 byte time while protocol version 2 clients use 8 byte time.
+
| 8 || time || uint64 || the Time.
 
|-
 
|-
 
| 4 || stream|| uint32 || Stream number for this node
 
| 4 || stream|| uint32 || Stream number for this node
Line 102: Line 104:
 
| 8 || services || uint64_t || same service(s) listed in [[#version|version]]
 
| 8 || services || uint64_t || same service(s) listed in [[#version|version]]
 
|-
 
|-
−
| 16 || IPv6/4 || char[16] || IPv6 address. The original client only supports IPv4 and only reads the last 4 bytes to get the IPv4 address. However, the IPv4 address is written into the message as a 16 byte [http://en.wikipedia.org/wiki/IPv6#IPv4-mapped_IPv6_addresses IPv4-mapped IPv6 address]
+
| 16 || IPv6/4 || char[16] || IPv6 address. IPv4 addresses are written into the message as a 16 byte [http://en.wikipedia.org/wiki/IPv6#IPv4-mapped_IPv6_addresses IPv4-mapped IPv6 address]
 
(12 bytes ''00 00 00 00  00 00 00 00  00 00 FF FF'', followed by the 4 bytes of the IPv4 address).
 
(12 bytes ''00 00 00 00  00 00 00 00  00 00 FF FF'', followed by the 4 bytes of the IPv4 address).
 
|-
 
|-
Line 118: Line 120:
 
|-
 
|-
 
| 32 || hash || char[32] || Hash of the object
 
| 32 || hash || char[32] || Hash of the object
 +
|}
 +
 +
=== Encrypted payload ===
 +
 +
Bitmessage uses [https://en.wikipedia.org/wiki/Integrated_Encryption_Scheme ECIES] to encrypt its messages. For more information see [[Encryption]]
 +
 +
{|class="wikitable"
 +
! Field Size !! Description !! Data type !! Comments
 +
|-
 +
| 16||IV||uchar[]||Initialization Vector used for AES-256-CBC
 +
|-
 +
| 2||uint16_t||Curve type||Elliptic Curve type 0x02CA (714)
 +
|-
 +
| 2||uint16_t||X length||Length of X component of public key R
 +
|-
 +
| X length||uchar[]||X||X component of public key R
 +
|-
 +
| 2||uint16_t||Y length||Length of Y component of public key R
 +
|-
 +
| Y length||uchar[]||Y||Y component of public key R
 +
|-
 +
| ?||encrypted||uchar[]||Cipher text
 +
|-
 +
| 32||MAC|| uchar[]||HMACSHA256 Message Authentication Code
 
|}
 
|}
  
Line 125: Line 151:
 
! Field Size !! Description !! Data type !! Comments
 
! Field Size !! Description !! Data type !! Comments
 
|-
 
|-
−
| 1+||msg_version||var_int||Message format version
+
| 1+||msg_version||var_int||Message format version. <span style="color:red">This field is not included after the protocol v3 upgrade period.</span>
 
|-
 
|-
 
| 1+||address_version||var_int||Sender's address version number. This is needed in order to calculate the sender's address to show in the UI, and also to allow for forwards compatible changes to the public-key data included below.
 
| 1+||address_version||var_int||Sender's address version number. This is needed in order to calculate the sender's address to show in the UI, and also to allow for forwards compatible changes to the public-key data included below.
Line 137: Line 163:
 
| 64||public encryption key||uchar[]||The ECC public key used for encryption (uncompressed format; normally prepended with \x04 )
 
| 64||public encryption key||uchar[]||The ECC public key used for encryption (uncompressed format; normally prepended with \x04 )
 
|-
 
|-
−
| 1+||nonce_trials_per_byte||var_int||Used to calculate the difficulty target of messages accepted by this node. The higher this value, the more difficult the Proof of Work must be before this individual will accept the message. This number is the average number of nonce trials a node will have to perform to meet the Proof of Work requirement. 320 is the network minimum so any lower values will be automatically raised to 320. '''This field is new and is only included when the address_version >= 3.'''
+
| 1+||nonce_trials_per_byte||var_int||Used to calculate the difficulty target of messages accepted by this node. The higher this value, the more difficult the Proof of Work must be before this individual will accept the message. This number is the average number of nonce trials a node will have to perform to meet the Proof of Work requirement. 1000 is the network minimum so any lower values will be automatically raised to 1000. '''This field is new and is only included when the address_version >= 3.'''
 
|-
 
|-
−
| 1+||extra_bytes||var_int||Used to calculate the difficulty target of messages accepted by this node. The higher this value, the more difficult the Proof of Work must be before this individual will accept the message. This number is added to the data length to make sending small messages more difficult. 14000 is the network minimum so any lower values will be automatically raised to 14000. '''This field is new and is only included when the address_version >= 3.'''
+
| 1+||extra_bytes||var_int||Used to calculate the difficulty target of messages accepted by this node. The higher this value, the more difficult the Proof of Work must be before this individual will accept the message. This number is added to the data length to make sending small messages more difficult. 1000 is the network minimum so any lower values will be automatically raised to 1000. '''This field is new and is only included when the address_version >= 3.'''
 
|-
 
|-
 
| 20||destination ripe||uchar[]||The ripe hash of the public key of the receiver of the message
 
| 20||destination ripe||uchar[]||The ripe hash of the public key of the receiver of the message
Line 155: Line 181:
 
| 1+||sig_length||var_int||Length of the signature
 
| 1+||sig_length||var_int||Length of the signature
 
|-
 
|-
−
| sig_length||signature||uchar[]||The ECDSA signature which covers everything from the msg_version to the ack_data.  
+
| sig_length||signature||uchar[]||The ECDSA signature which covers the object header starting with the time, appended with the data described in this table down to the ack_data.  
 
|}
 
|}
  
Line 168: Line 194:
 
| 2 || SIMPLE || UTF-8. Uses 'Subject' and 'Body' sections. No MIME is used.  
 
| 2 || SIMPLE || UTF-8. Uses 'Subject' and 'Body' sections. No MIME is used.  
 
<code>messageToTransmit = 'Subject:' + subject + '\n' + 'Body:' + message</code>
 
<code>messageToTransmit = 'Subject:' + subject + '\n' + 'Body:' + message</code>
 +
|-
 +
| 3 || EXTENDED || See [[Extended encoding]]
 
|}
 
|}
  
Line 182: Line 210:
 
| ... || ... || ...
 
| ... || ... || ...
 
|-
 
|-
−
| 30 || include_destination || Receiving node expects that the RIPE hash encoded in their address preceedes the encrypted message data of msg messages bound for them.  
+
| 27 || onion_router || ('''Proposal''') Node can be used to onion-route messages. In theory any node can onion route, but since it requires more resources, they may have the functionality disabled. This field will be used to indicate that the node is willing to do this.
 +
|-
 +
| 28 || forward_secrecy || ('''Proposal''') Receiving node supports a forward secrecy encryption extension. The exact design is pending.
 +
|-
 +
| 29 || chat || ('''Proposal''') Address if for chatting rather than messaging.
 +
|-
 +
| 30 || include_destination || ('''Proposal''') Receiving node expects that the RIPE hash encoded in their address preceedes the encrypted message data of msg messages bound for them.
 +
'''NOTE:''' since hardly anyone implements this, this will be redesigned as simple recipient verification: https://github.com/Bitmessage/PyBitmessage/pull/808#issuecomment-170189856
 
|-
 
|-
 
| 31 || does_ack|| If true, the receiving node does send acknowledgements (rather than dropping them).
 
| 31 || does_ack|| If true, the receiving node does send acknowledgements (rather than dropping them).
Line 188: Line 223:
  
 
== Message types ==
 
== Message types ==
 +
Undefined messages received on the wire must be ignored.
  
 
=== version ===
 
=== version ===
Line 198: Line 234:
 
! Field Size !! Description !! Data type !! Comments
 
! Field Size !! Description !! Data type !! Comments
 
|-
 
|-
−
| 4 || version || int32_t || Identifies protocol version being used by the node
+
| 4 || version || int32_t || Identifies protocol version being used by the node. Should equal 3. Nodes should disconnect if the remote node's version is lower but continue with the connection if it is higher.
 
|-
 
|-
 
| 8 || services || uint64_t || bitfield of features to be enabled for this connection
 
| 8 || services || uint64_t || bitfield of features to be enabled for this connection
Line 210: Line 246:
 
| 8 || nonce || uint64_t || Random nonce used to detect connections to self.
 
| 8 || nonce || uint64_t || Random nonce used to detect connections to self.
 
|-
 
|-
−
| 1+ || user_agent || [[#Variable length string|var_str]] || [[User Agent]] (0x00 if string is 0 bytes long)
+
| 1+ || user_agent || [[#Variable length string|var_str]] || [[User Agent]] (0x00 if string is 0 bytes long). Sending nodes must not include a user_agent longer than 5000 bytes.
 
|-
 
|-
−
| 1+ || stream numbers || [[#Variable length list of integers|var_int_list]] || The stream numbers that the emitting node is interested in.
+
| 1+ || stream_numbers || [[#Variable length list of integers|var_int_list]] || The stream numbers that the emitting node is interested in. Sending nodes must not include more than 160000 stream numbers.
 
|}
 
|}
  
Line 223: Line 259:
 
|-
 
|-
 
| 1 || NODE_NETWORK || This is a normal network node.
 
| 1 || NODE_NETWORK || This is a normal network node.
 +
|-
 +
| 2 || NODE_SSL    || This node supports SSL/TLS in the current connect (python < 2.7.9 only supports a SSL client, so in that case it would only have this on when the connection is a client).
 +
|-
 +
| 3 || NODE_POW    || '''(Proposal)''' This node may do PoW on behalf of some its peers (PoW offloading/delegating), but it doesn't have to. Clients may have to meet additional requirements (e.g. TLS authentication)
 +
|-
 +
| 4 || NODE_DANDELION || Node supports dandelion (https://github.com/gfanti/bips/blob/master/bip-dandelion.mediawiki)
 
|}
 
|}
  
 
=== verack ===
 
=== verack ===
  
−
The ''verack'' message is sent in reply to ''version''.  This message consists of only a [[#Message structure|message header]] with the command string "verack".
+
The ''verack'' message is sent in reply to ''version''.  This message consists of only a [[#Message structure|message header]] with the command string "verack". The TCP timeout starts out at 20 seconds; after verack messages are exchanged, the timeout is raised to 10 minutes.
 +
 
 +
If both sides announce that they support SSL, they MUST perform a SSL handshake immediately after they both send and receive verack. During this SSL handshake, the TCP client acts as a SSL client, and the TCP server acts as a SSL server. The current implementation (v0.5.4 or later) requires the AECDH-AES256-SHA cipher over TLSv1 protocol, and prefers the secp256k1 curve (but other curves may be accepted, depending on the version of python and OpenSSL used).
  
 
=== addr ===
 
=== addr ===
Line 239: Line 283:
 
| 1+ || count || [[Protocol_specification#Variable_length_integer|var_int]] || Number of address entries (max: 1000)
 
| 1+ || count || [[Protocol_specification#Variable_length_integer|var_int]] || Number of address entries (max: 1000)
 
|-
 
|-
−
| 34x? || addr_list || [[Protocol_specification#Network_address|net_addr]] || Address of other nodes on the network.  
+
| 38 || addr_list || [[Protocol_specification#Network_address|net_addr]] || Address of other nodes on the network.  
 
|}
 
|}
  
Line 267: Line 311:
 
|-
 
|-
 
| 32x? || inventory || [[Protocol specification#Inventory Vectors|inv_vect]][] || Inventory vectors
 
| 32x? || inventory || [[Protocol specification#Inventory Vectors|inv_vect]][] || Inventory vectors
 +
|}
 +
 +
=== object ===
 +
 +
An object is a message which is shared throughout a stream. It is the only message which propagates; all others are only between two nodes. Objects have a type, like 'msg', or 'broadcast'. To be a valid object, the [[Proof of work|Proof Of Work]] must be done. The maximum allowable length of an object (not to be confused with the objectPayload) is 2<sup>18</sup> bytes.
 +
 +
{|class="wikitable"
 +
! Field Size !! Description !! Data type !! Comments
 +
|-
 +
| 8||nonce||uint64_t||
 +
Random nonce used for the [[Proof of work|Proof Of Work]]
 +
|-
 +
| 8||expiresTime||uint64_t||
 +
The "end of life" time of this object (be aware, in version 2 of the protocol this was the generation time). Objects shall be shared with peers until its end-of-life time has been reached. The node should store the inventory vector of that object for some extra period of time to avoid reloading it from another node with a small time delay. The time may be no further than 28 days + 3 hours in the future.
 +
|-
 +
| 4||objectType||uint32_t||
 +
Four values are currently defined: 0-"getpubkey", 1-"pubkey", 2-"msg", 3-"broadcast". All other values are reserved. Nodes should relay objects even if they use an undefined object type.
 +
|-
 +
| 1+||version||var_int||The object's version. Note that msg objects won't contain a version until Sun, 16 Nov 2014 22:00:00 GMT.
 +
|-
 +
| 1+||stream number||var_int||The stream number in which this object may propagate
 +
|-
 +
| ?||objectPayload||uchar[]||
 +
This field varies depending on the object type; see below.
 +
|-
 
|}
 
|}
  
 
== Object types ==
 
== Object types ==
−
Objects are a subset of network messages. They are shared throughout a stream.  A client should advertise objects that are not older than 2.5 days. To be a valid object, the [[Proof of work|Proof Of Work]] has to be done.
+
Here are the payloads for various object types.
  
 
=== getpubkey ===  
 
=== getpubkey ===  
Line 279: Line 348:
 
! Field Size !! Description !! Data type !! Comments
 
! Field Size !! Description !! Data type !! Comments
 
|-
 
|-
−
| 8||POW nonce||uint64_t||Random nonce used for the [[Proof of work|Proof Of Work]]
+
| <font color="#666666">20||<font color="#666666">ripe||<font color="#666666">uchar[]||<font color="#666666">The ripemd hash of the public key. This field is only included when the address version is <= 3.</font>
−
|-
 
−
| 4 (or 8)||time||uint32_t||The time that this message was generated and broadcast. We are transitioning to 8 byte time.
 
 
|-
 
|-
−
| 1+||address version||var_int||The address' version
+
| 32||tag||uchar[]||The tag derived from the address version, stream number, and ripe. This field is only included when the address version is >= 4.
−
|-
 
−
| 1+||stream number||var_int||The address' stream number
 
−
|-
 
−
| 20||pub key hash||uchar[]||The ripemd hash of the public key
 
 
|}
 
|}
  
 
=== pubkey ===  
 
=== pubkey ===  
  
−
A version 2 public key. This is still in use and supported by current clients but ''new'' v2 addresses are not generated by clients.
+
A version 2 pubkey. This is still in use and supported by current clients but ''new'' v2 addresses are not generated by clients.
 +
 
 
{|class="wikitable"
 
{|class="wikitable"
 
! Field Size !! Description !! Data type !! Comments
 
! Field Size !! Description !! Data type !! Comments
−
|-
 
−
|-
 
−
| 8||POW nonce||uint64_t||Random nonce used for the [[Proof of work|Proof Of Work]]
 
−
|-
 
−
| 4 (or 8)||time||uint32_t||The time that this message was generated and broadcast. We are transitioning to 8 byte time.
 
−
|-
 
−
| 1+||address version||var_int||The address' version which is set to 2.
 
−
|-
 
−
| 1+||stream number||var_int||The address' stream number
 
 
|-
 
|-
 
| 4||[[Protocol_specification#Pubkey_bitfield_features|behavior bitfield]]||uint32_t||A bitfield of optional behaviors and features that can be expected from the node receiving the message.  
 
| 4||[[Protocol_specification#Pubkey_bitfield_features|behavior bitfield]]||uint32_t||A bitfield of optional behaviors and features that can be expected from the node receiving the message.  
Line 316: Line 371:
 
{|class="wikitable"
 
{|class="wikitable"
 
! Field Size !! Description !! Data type !! Comments
 
! Field Size !! Description !! Data type !! Comments
−
|-
 
−
|-
 
−
| 8||POW nonce||uint64_t||Random nonce used for the [[Proof of work|Proof Of Work]]
 
−
|-
 
−
| 4 (or 8)||time||uint32_t||The time that this message was generated and broadcast. We are transitioning to 8 byte time.
 
−
|-
 
−
| 1+||address version||var_int||The address' version which is set to 3.
 
−
|-
 
−
| 1+||stream number||var_int||The address' stream number
 
 
|-
 
|-
 
| 4||[[Protocol_specification#Pubkey_bitfield_features|behavior bitfield]]||uint32_t||A bitfield of optional behaviors and features that can be expected from the node receiving the message.  
 
| 4||[[Protocol_specification#Pubkey_bitfield_features|behavior bitfield]]||uint32_t||A bitfield of optional behaviors and features that can be expected from the node receiving the message.  
Line 332: Line 378:
 
| 64||public encryption key||uchar[]||The ECC public key used for encryption (uncompressed format; normally prepended with \x04 )
 
| 64||public encryption key||uchar[]||The ECC public key used for encryption (uncompressed format; normally prepended with \x04 )
 
|-
 
|-
−
| 1+||nonce_trials_per_byte||var_int||Used to calculate the difficulty target of messages accepted by this node. The higher this value, the more difficult the Proof of Work must be before this individual will accept the message. This number is the average number of nonce trials a node will have to perform to meet the Proof of Work requirement. 320 is the network minimum so any lower values will be automatically raised to 320.  
+
| 1+||nonce_trials_per_byte||var_int||Used to calculate the difficulty target of messages accepted by this node. The higher this value, the more difficult the Proof of Work must be before this individual will accept the message. This number is the average number of nonce trials a node will have to perform to meet the Proof of Work requirement. 1000 is the network minimum so any lower values will be automatically raised to 1000.  
 
|-
 
|-
−
| 1+||extra_bytes||var_int||Used to calculate the difficulty target of messages accepted by this node. The higher this value, the more difficult the Proof of Work must be before this individual will accept the message. This number is added to the data length to make sending small messages more difficult. 14000 is the network minimum so any lower values will be automatically raised to 14000.
+
| 1+||extra_bytes||var_int||Used to calculate the difficulty target of messages accepted by this node. The higher this value, the more difficult the Proof of Work must be before this individual will accept the message. This number is added to the data length to make sending small messages more difficult. 1000 is the network minimum so any lower values will be automatically raised to 1000.
 
|-
 
|-
 
| 1+||sig_length||var_int||Length of the signature
 
| 1+||sig_length||var_int||Length of the signature
 
|-
 
|-
−
| sig_length||signature||uchar[]||The ECDSA signature which covers everything from the time to the extra_bytes.  
+
| sig_length||signature||uchar[]||The ECDSA signature which, <span style="color:red">as of protocol v3, covers the object header starting with the time, appended with the data described in this table down to the extra_bytes.</span>
 
|}
 
|}
  
−
=== msg ===
+
A version 4 pubkey
−
Used for person-to-person messages.
+
 
 
{|class="wikitable"
 
{|class="wikitable"
 
! Field Size !! Description !! Data type !! Comments
 
! Field Size !! Description !! Data type !! Comments
 
|-
 
|-
−
| 8||POW nonce||uint64_t||Random nonce used for the [[Proof of work|Proof Of Work]]
+
| 32||tag||uchar[]||The tag, made up of bytes 32-64 of the double hash of the address data (see example python code below)
 
|-
 
|-
−
| 4 (or 8)||time||uint32_t||The time that this message was generated and broadcast. We are transitioning to 8 byte time.
+
| ?||encrypted||uchar[]||Encrypted pubkey data.
 
|-
 
|-
−
| 1+ ||streamNumber||var_int||The stream number of the destination address.
 
−
|-
 
−
| ?||encrypted||uchar[]||Encrypted data. See [[Protocol_specification#Encrypted_payload|Encrypted payload]]. See also [[Protocol_specification#Unencrypted_Message_Data|Unencrypted Message Data Format]]
 
 
|}
 
|}
  
−
=== broadcast ===
+
When version 4 pubkeys are created, most of the data in the pubkey is encrypted. This is done in such a way that only someone who has the Bitmessage address which corresponds to a pubkey can decrypt and use that pubkey. This prevents people from gathering pubkeys sent around the network and using the data from them to create messages to be used in spam or in flooding attacks.
−
Version 1 broadcast messages are sent in-the-clear. Version 2 are encrypted. Users who are subscribed to the sending address will see the message appear in their inbox.
+
 
 +
In order to encrypt the pubkey data, a double SHA-512 hash is calculated from the address version number, stream number, and ripe hash of the Bitmessage address that the pubkey corresponds to. The first 32 bytes of this hash are used to create a public and private key pair with which to encrypt and decrypt the pubkey data, using the same algorithm as message encryption (see [[Encryption]]). The remaining 32 bytes of this hash are added to the unencrypted part of the pubkey and used as a tag, as above. This allows nodes to determine which pubkey to decrypt when they wish to send a message.
 +
 
 +
In PyBitmessage, the double hash of the address data is calculated using the python code below:
 +
 
 +
doubleHashOfAddressData = hashlib.sha512(hashlib.sha512(encodeVarint(addressVersionNumber) + encodeVarint(streamNumber) + hash).digest()).digest()
 +
 
 +
Encrypted data in version 4 pubkeys:
  
−
Version 1 broadcast format:
 
 
{|class="wikitable"
 
{|class="wikitable"
 
! Field Size !! Description !! Data type !! Comments
 
! Field Size !! Description !! Data type !! Comments
 
|-
 
|-
−
| 8 || POW nonce|| uint64_t|| The [[Proof of work|Proof Of Work]] nonce
 
 
|-
 
|-
−
| 4 (or 8) || time|| uint32_t|| The time that the message was broadcast. We are transitioning to 8 byte time.
+
| 4||[[Protocol_specification#Pubkey_bitfield_features|behavior bitfield]]||uint32_t||A bitfield of optional behaviors and features that can be expected from the node receiving the message.  
−
|-
 
−
| 1+|| broadcast version|| var_int||The version number of this broadcast protocol message which is equal to 1 in this case.
 
−
|-
 
−
| 1+ || address version|| var_int||The sender's address version
 
−
|-
 
−
| 1+ || stream number|| var_int||The sender's stream number
 
−
|-
 
−
| 4||[[Protocol_specification#Pubkey_bitfield_features|behavior bitfield]]||uint32_t||A bitfield of optional behaviors and features that can be expected from the owner of this pubkey.  
 
 
|-
 
|-
 
| 64||public signing key||uchar[]||The ECC public key used for signing (uncompressed format; normally prepended with \x04 )
 
| 64||public signing key||uchar[]||The ECC public key used for signing (uncompressed format; normally prepended with \x04 )
Line 378: Line 418:
 
| 64||public encryption key||uchar[]||The ECC public key used for encryption (uncompressed format; normally prepended with \x04 )
 
| 64||public encryption key||uchar[]||The ECC public key used for encryption (uncompressed format; normally prepended with \x04 )
 
|-
 
|-
−
| 20 || address hash|| uchar[]||The sender's address hash. This is included so that nodes can more cheaply detect whether this is a broadcast message for which they are listening, although it must be verified with the public key above.
+
| 1+||nonce_trials_per_byte||var_int||Used to calculate the difficulty target of messages accepted by this node. The higher this value, the more difficult the Proof of Work must be before this individual will accept the message. This number is the average number of nonce trials a node will have to perform to meet the Proof of Work requirement. 1000 is the network minimum so any lower values will be automatically raised to 1000.  
 
|-
 
|-
−
| 1+ || [[Protocol_specification#Message_Encodings|encoding]]|| var_int||The [[Protocol_specification#Message_Encodings|encoding type]] of the message
+
| 1+||extra_bytes||var_int||Used to calculate the difficulty target of messages accepted by this node. The higher this value, the more difficult the Proof of Work must be before this individual will accept the message. This number is added to the data length to make sending small messages more difficult. 1000 is the network minimum so any lower values will be automatically raised to 1000.
 
|-
 
|-
−
| 1+ ||messageLength || var_int||The message length in bytes
+
| 1+||sig_length||var_int||Length of the signature
 
|-
 
|-
−
| messageLength || message||uchar[] ||The message
+
| sig_length||signature||uchar[]||The ECDSA signature which covers everything from the object header starting with the time, then appended with the decrypted data down to the extra_bytes. <span style="color:red">This was changed in protocol v3. </span>
 +
|}
 +
 
 +
=== msg ===
 +
Used for person-to-person messages. Note that msg objects won't contain a version in the object header until Sun, 16 Nov 2014 22:00:00 GMT.
 +
{|class="wikitable"
 +
! Field Size !! Description !! Data type !! Comments
 
|-
 
|-
−
| 1+||sig_length||var_int||Length of the signature
+
| ?||encrypted||uchar[]||Encrypted data. See [[Protocol_specification#Encrypted_payload|Encrypted payload]]. See also [[Protocol_specification#Unencrypted_Message_Data|Unencrypted Message Data Format]]
−
|-
 
−
| sig_length|| signature||uchar[] ||The signature which covers everything from the broadcast version down through the message.
 
 
|}
 
|}
  
−
Version 2 broadcasts:
+
=== broadcast ===
 +
Users who are subscribed to the sending address will see the message appear in their inbox. Broadcasts are <span style="color:red">version 4 or 5.</span>
 +
 
 +
Pubkey objects and v5 broadcast objects are encrypted the same way: The data encoded in the sender's Bitmessage address is hashed twice. The first 32 bytes of the resulting hash constitutes the "private" encryption key and the last 32 bytes constitute a '''tag''' so that anyone listening can easily decide if this particular message is interesting. The sender calculates the public key from the private key and then encrypts the object with this public key. Thus anyone who knows the Bitmessage address of the sender of a broadcast or pubkey object can decrypt it.
 +
 
 +
The version of broadcast objects was previously 2 or 3 <span style="color:red">but was changed to 4 or 5 for protocol v3. </span> Having a broadcast version of 5 indicates that a tag is used which, in turn, is used when the sender's address version is >=4.
  
 
{|class="wikitable"
 
{|class="wikitable"
 
! Field Size !! Description !! Data type !! Comments
 
! Field Size !! Description !! Data type !! Comments
 
|-
 
|-
−
| 8 || POW nonce|| uint64_t|| The [[Proof of work|Proof Of Work]] nonce
+
| 32 || tag|| uchar[]||The tag. This field is new and only included when the broadcast version is <span style="color:red">>= 5. Changed in protocol v3</span>
−
|-
 
−
| 4 (or 8) || time|| uint32_t|| The time that the message was broadcast. We are transitioning to 8 byte time.
 
−
|-
 
−
| 1+|| broadcast version|| var_int||The version number of this broadcast protocol message which is equal to 2 in this case.
 
−
|-
 
−
| 1+ || stream number|| var_int||The sender's stream number
 
 
|-
 
|-
−
| ?||encrypted||uchar[]||Encrypted broadcast data. See [[Protocol_specification#Encrypted_payload|Encrypted payload]]. See also [[Protocol_specification#Unencrypted_Broadcast_Data|Unencrypted Broadcast Data Format]]
+
| ?||encrypted||uchar[]||Encrypted broadcast data. The keys are derived as described in the paragraph above. See [[Protocol_specification#Encrypted_payload|Encrypted payload]] for details about the encryption algorithm itself.
 
|}
 
|}
  
Line 411: Line 454:
 
! Field Size !! Description !! Data type !! Comments
 
! Field Size !! Description !! Data type !! Comments
 
|-
 
|-
−
| 1+|| broadcast version|| var_int||The version number of this broadcast protocol message which is equal to 2 in this case. This is included here so that it can be signed.  
+
| 1+|| broadcast version|| var_int||<span style="color:grey">The version number of this broadcast protocol message which is equal to 2 or 3. This is included here so that it can be signed.</span> <span style="color:red">This is no longer included in protocol v3</span>
 
|-
 
|-
 
| 1+ || address version|| var_int||The sender's address version
 
| 1+ || address version|| var_int||The sender's address version
Line 423: Line 466:
 
| 64||public encryption key||uchar[]||The ECC public key used for encryption (uncompressed format; normally prepended with \x04 )
 
| 64||public encryption key||uchar[]||The ECC public key used for encryption (uncompressed format; normally prepended with \x04 )
 
|-
 
|-
−
| 1+||nonce_trials_per_byte||var_int||Used to calculate the difficulty target of messages accepted by this node. The higher this value, the more difficult the Proof of Work must be before this individual will accept the message. This number is the average number of nonce trials a node will have to perform to meet the Proof of Work requirement. 320 is the network minimum so any lower values will be automatically raised to 320. '''This field is new and is only included when the address_version >= 3.'''
+
| 1+||nonce_trials_per_byte||var_int||Used to calculate the difficulty target of messages accepted by this node. The higher this value, the more difficult the Proof of Work must be before this individual will accept the message. This number is the average number of nonce trials a node will have to perform to meet the Proof of Work requirement. 1000 is the network minimum so any lower values will be automatically raised to 1000. This field is new and is only included when the address_version >= 3.
 
|-
 
|-
−
| 1+||extra_bytes||var_int||Used to calculate the difficulty target of messages accepted by this node. The higher this value, the more difficult the Proof of Work must be before this individual will accept the message. This number is added to the data length to make sending small messages more difficult. 14000 is the network minimum so any lower values will be automatically raised to 14000. '''This field is new and is only included when the address_version >= 3.'''
+
| 1+||extra_bytes||var_int||Used to calculate the difficulty target of messages accepted by this node. The higher this value, the more difficult the Proof of Work must be before this individual will accept the message. This number is added to the data length to make sending small messages more difficult. 1000 is the network minimum so any lower values will be automatically raised to 1000. This field is new and is only included when the address_version >= 3.
 
|-
 
|-
 
| 1+ || [[Protocol_specification#Message_Encodings|encoding]]|| var_int||The [[Protocol_specification#Message_Encodings|encoding type]] of the message
 
| 1+ || [[Protocol_specification#Message_Encodings|encoding]]|| var_int||The [[Protocol_specification#Message_Encodings|encoding type]] of the message
Line 435: Line 478:
 
| 1+||sig_length||var_int||Length of the signature
 
| 1+||sig_length||var_int||Length of the signature
 
|-
 
|-
−
| sig_length|| signature||uchar[] ||The signature which covers everything from the broadcast version down through the message.
+
| sig_length|| signature||uchar[] ||The signature which <span style="color:red">did</span> cover the unencrypted data from the broadcast version down through the message. <span style="color:red">In protocol v3, it covers the unencrypted object header starting with the time, all appended with the decrypted data.</span>
−
|}
 
−
 
 
−
=== Encrypted payload ===
 
−
 
 
−
Bitmessage uses the Elliptic Curve Integrated Encryption Scheme (ECIES)[http://en.wikipedia.org/wiki/Integrated_Encryption_Scheme] to encrypt the payload of the Message and Broadcast objects.
 
−
 
 
−
The scheme uses Elliptic Curve Diffie-Hellman (ECDH)[http://en.wikipedia.org/wiki/ECDH] to generate a shared secret used to generate the encryption parameters for Advanced Encryption Standard with 256bit key and Cipher-Block Chaining (AES-256-CBC)[http://en.wikipedia.org/wiki/Advanced_Encryption_Standard]. The encrypted data will be padded to a 16 byte boundary in accordance to PKCS7[http://en.wikipedia.org/wiki/Cryptographic_Message_Syntax]. This means that the data is padded with N bytes of value N.
 
−
 
 
−
The Key Derivation Function (KDF)[http://en.wikipedia.org/wiki/Key_derivation_function] used to generate the key material for AES is SHA512[http://en.wikipedia.org/wiki/Sha512]. The Message Authentication Code (MAC) scheme used is HMACSHA256[http://en.wikipedia.org/wiki/Hmac].
 
−
 
 
−
==== Format ====
 
−
 
 
−
{|class="wikitable"
 
−
! Field Size !! Description !! Data type !! Comments
 
−
|-
 
−
| 16||IV||uchar[]||Initialization Vector used for AES-256-CBC
 
−
|-
 
−
| 2||uint16_t||Curve type||Elliptic Curve type 0x02CA (714)
 
−
|-
 
−
| 2||uint16_t||X length||Length of X component of public key R
 
−
|-
 
−
| X length||uchar[]||X||X component of public key R
 
−
|-
 
−
| 2||uint16_t||Y length||Length of Y component of public key R
 
−
|-
 
−
| Y length||uchar[]||Y||Y component of public key R
 
−
|-
 
−
| ?||encrypted||uchar[]||Cipher text
 
−
|-
 
−
| 32||MAC|| uchar[]||HMACSHA256 Message Authentication Code
 
−
|}
 
−
 
 
−
In order to reconstitute a usable (65 byte) public key (starting with 0x04), the X and Y components need to be expanded by prepending them with 0x00 bytes until the individual component lengths are 32 bytes.
 
−
 
 
−
 
 
−
==== Encryption ====
 
−
 
 
−
# The destination public key is called K.
 
−
# Generate 16 random bytes using a secure random number generator. Call them IV.
 
−
# Generate a new random EC key pair with private key called r and public key called R.
 
−
# Do an EC point multiply with public key K and private key r. This gives you public key P.
 
−
# Use the X component of public key P and calculate the SHA512 hash H.
 
−
# The first 32 bytes of H are called key_e and the last 32 bytes are called key_m.
 
−
# Pad the input text to a multiple of 16 bytes, in accordance to PKCS7.
 
−
# Encrypt the data with AES-256-CBC, using IV as initialization vector, key_e as encryption key and the padded input text as payload. Call the output cipher text.
 
−
# Calculate a 32 byte MAC with HMACSHA256, using key_m as salt and cipher text as data. Call the output MAC.
 
−
 
 
−
The resulting data is: IV + R + cipher text + MAC
 
−
 
 
−
==== Decryption ====
 
−
 
 
−
# The private key used to decrypt is called k.
 
−
# Do an EC point multiply with private key k and public key R. This gives you public key P.
 
−
# Use the X component of public key P and calculate the SHA512 hash H.
 
−
# The first 32 bytes of H are called key_e and the last 32 bytes are called key_m.
 
−
# Calculate MAC' with HMACSHA256, using key_m as salt and cipher text as data.
 
−
# Compare MAC with MAC'. If not equal, decryption will fail.
 
−
# Decrypt the cipher text with AES-256-CBC, using IV as initialization vector, key_e as decryption key and the cipher text as payload. The output is the padded input text.
 
−
 
 
−
==== Example ====
 
−
 
 
−
Public key K:
 
−
 
 
−
{|class="wikitable"
 
−
! Data !! Comments
 
−
|-
 
−
|<pre>
 
−
04 09 d4 e5  c0 ab 3d 25
 
−
fe 04 8c 64  c9 da 1a 24
 
−
2c 7f 19 41  7e 95 17 cd
 
−
26 69 50 d7  2c 75 57 13
 
−
58 5c 61 78  e9 7f e0 92
 
−
fc 89 7c 9a  1f 17 20 d5
 
−
77 0a e8 ea  ad 2f a8 fc
 
−
bd 08 e9 32  4a 5d de 18
 
−
57</pre>||Public key, 0x04 prefix, then 32 bytes X and 32 bytes Y.
 
−
|}
 
−
 
 
−
 
 
−
Initialization Vector IV:
 
−
 
 
−
{|class="wikitable"
 
−
! Data !! Comments
 
−
|-
 
−
|<pre>bd db 7c 28  29 b0 80 38
 
−
75 30 84 a2  f3 99 16 81</pre>||16 bytes generated with a secure random number generator.
 
−
|}
 
−
 
 
−
Randomly generated key pair with private key r and public key R:
 
−
 
 
−
{|class="wikitable"
 
−
! Data !! Comments
 
−
|-
 
−
|<pre>5b e6 fa cd  94 1b 76 e9
 
−
d3 ea d0 30  29 fb db 6b
 
−
6e 08 09 29  3f 7f b1 97
 
−
d0 c5 1f 84  e9 6b 8b a4</pre>||Private key r
 
−
|-
 
−
|<pre>04 02 93 21  3d cf 13 88
 
−
b6 1c 2a e5  cf 80 fe e6
 
−
ff ff c0 49  a2 f9 fe 73
 
−
65 fe 38 67  81 3c a8 12
 
−
92 df 94 68  6c 6a fb 56
 
−
5a c6 14 9b  15 3d 61 b3
 
−
b2 87 ee 2c  7f 99 7c 14
 
−
23 87 96 c1  2b 43 a3 86
 
−
5a</pre>||Public key R
 
−
|}
 
−
 
 
−
Derived public key P (point multiply r with K):
 
−
 
 
−
{|class="wikitable"
 
−
! Data !! Comments
 
−
|-
 
−
|<pre>04 0d b8 e3  ad 8c 0c d7
 
−
3f a2 b3 46  71 b7 b2 47
 
−
72 9b 10 11  41 57 9d 19
 
−
9e 0d c0 bd  02 4e ae fd
 
−
89 ca c8 f5  28 dc 90 b6
 
−
68 11 ab ac  51 7d 74 97
 
−
be 52 92 93  12 29 be 0b
 
−
74 3e 05 03  f4 43 c3 d2
 
−
96</pre>||Public key P
 
−
|-
 
−
|<pre>0d b8 e3 ad  8c 0c d7 3f
 
−
a2 b3 46 71  b7 b2 47 72
 
−
9b 10 11 41  57 9d 19 9e
 
−
0d c0 bd 02  4e ae fd 89</pre>||X component of public key P
 
−
|}
 
−
 
 
−
SHA512 of public key P X component (H):
 
−
 
 
−
{|class="wikitable"
 
−
! Data !! Comments
 
−
|-
 
−
|<pre>17 05 43 82  82 67 86 71
 
−
05 26 3d 48  28 ef ff 82
 
−
d9 d5 9c bf  08 74 3b 69
 
−
6b cc 5d 69  fa 18 97 b4</pre>||First 32 bytes of H called key_e
 
−
|-
 
−
|<pre>f8 3f 1e 9c  c5 d6 b8 44
 
−
8d 39 dc 6a  9d 5f 5b 7f
 
−
46 0e 4a 78  e9 28 6e e8
 
−
d9 1c e1 66  0a 53 ea cd</pre>||Last 32 bytes of H called key_m
 
−
|}
 
−
 
 
−
Padded input:
 
−
 
 
−
{|class="wikitable"
 
−
! Data !! Comments
 
−
|-
 
−
|<pre>54 68 65 20  71 75 69 63
 
−
6b 20 62 72  6f 77 6e 20
 
−
66 6f 78 20  6a 75 6d 70
 
−
73 20 6f 76  65 72 20 74
 
−
68 65 20 6c  61 7a 79 20
 
−
64 6f 67 2e  04 04 04 04</pre>||The quick brown fox jumps over the lazy dog.0x04,0x04,0x04,0x04
 
−
|}
 
−
 
 
−
Cipher text:
 
−
 
 
−
{|class="wikitable"
 
−
! Data !! Comments
 
−
|-
 
−
|<pre>64 20 3d 5b  24 68 8e 25
 
−
47 bb a3 45  fa 13 9a 5a
 
−
1d 96 22 20  d4 d4 8a 0c
 
−
f3 b1 57 2c  0d 95 b6 16
 
−
43 a6 f9 a0  d7 5a f7 ea
 
−
cc 1b d9 57  14 7b f7 23</pre>||3 blocks of 16 bytes of encrypted data.
 
−
|}
 
−
 
 
−
MAC:
 
−
 
 
−
{|class="wikitable"
 
−
! Data !! Comments
 
−
|-
 
−
|<pre>4c 08 ac 6c  93 c7 37 7b
 
−
ac 5a 2e 87  3d d3 51 1b
 
−
12 7a ff 6d  0d 16 38 cd
 
−
ae 49 89 c4  d2 fe 7d e1</pre>||HMACSHA256 with key_m as salt and cipher text as input.
 
−
|}
 
−
 
 
−
Resulting encrypted data:
 
−
 
 
−
{|class="wikitable"
 
−
! Data !! Comments
 
−
|-
 
−
|<pre>bd db 7c 28  29 b0 80 38
 
−
75 30 84 a2  f3 99 16 81</pre>||IV
 
−
|-
 
−
|<pre>02 ca</pre>||Curve Type
 
−
|-
 
−
|<pre>00 20</pre>||X Length
 
−
|-
 
−
|<pre>02 93 21 3d  cf 13 88 b6
 
−
1c 2a e5 cf  80 fe e6 ff
 
−
ff c0 49 a2  f9 fe 73 65
 
−
fe 38 67 81  3c a8 12 92</pre>||X
 
−
|-
 
−
|<pre>00 20</pre>||Y Length
 
−
|-
 
−
|<pre>df 94 68 6c  6a fb 56 5a
 
−
c6 14 9b 15  3d 61 b3 b2
 
−
87 ee 2c 7f  99 7c 14 23
 
−
87 96 c1 2b  43 a3 86 5a</pre>||Y
 
−
|-
 
−
|<pre>64 20 3d 5b  24 68 8e 25
 
−
47 bb a3 45  fa 13 9a 5a
 
−
1d 96 22 20  d4 d4 8a 0c
 
−
f3 b1 57 2c  0d 95 b6 16
 
−
43 a6 f9 a0  d7 5a f7 ea
 
−
cc 1b d9 57  14 7b f7 23</pre>||Cipher text
 
−
|-
 
−
|<pre>4c 08 ac 6c  93 c7 37 7b
 
−
ac 5a 2e 87  3d d3 51 1b
 
−
12 7a ff 6d  0d 16 38 cd
 
−
ae 49 89 c4  d2 fe 7d e1</pre>||MAC
 
−
|}
 
−
 
 
−
 
 
−
Private key k:
 
−
 
 
−
{|class="wikitable"
 
−
! Data !! Comments
 
−
|-
 
−
|<pre>02 ba 27 44  e6 5c cd 7b
 
−
19 54 b0 a3  3b 80 d7 5e
 
−
16 ca b4 7f  2b 33 1f f0
 
−
b6 d1 84 b7  19 83 da 85</pre>||Private key k used to decrypt the above encrypted data.
 
 
|}
 
|}

Latest revision as of 02:40, 29 April 2020

All objects sent on the network should support protocol v3 starting on Sun, 16 Nov 2014 22:00:00 GMT.

Common standards

Hashes

Most of the time SHA-512 hashes are used, however RIPEMD-160 is also used when creating an address.

A double-round of SHA-512 is used for the Proof Of Work. Example of double-SHA-512 encoding of string "hello": 

hello
9b71d224bd62f3785d96d46ad3ea3d73319bfbc2890caadae2dff72519673ca72323c3d99ba5c11d7c7acc6e14b8c5da0c4663475c2e5c3adef46f73bcdec043(first round of sha-512)
0592a10584ffabf96539f3d780d776828c67da1ab5b169e9e8aed838aaecc9ed36d49ff1423c55f019e050c66c6324f53588be88894fef4dcffdb74b98e2b200(second round of sha-512)

For Bitmessage addresses (RIPEMD-160) this would give: 

hello
9b71d224bd62f3785d96d46ad3ea3d73319bfbc2890caadae2dff72519673ca72323c3d99ba5c11d7c7acc6e14b8c5da0c4663475c2e5c3adef46f73bcdec043(first round is sha-512)
79a324faeebcbf9849f310545ed531556882487e (with ripemd-160)

Common structures

All integers are encoded in big endian. (This is different from Bitcoin).

Message structure

Field Size Description Data type Comments
4 magic uint32_t Magic value indicating message origin network, and used to seek to next message when stream state is unknown
12 command char[12] ASCII string identifying the packet content, NULL padded (non-NULL padding results in packet rejected)
4 length uint32_t Length of payload in number of bytes. Because of other restrictions, there is no reason why this length would ever be larger than 1600003 bytes. Some clients include a sanity-check to avoid processing messages which are larger than this.
4 checksum uint32_t First 4 bytes of sha512(payload)
? message_payload uchar[] The actual data, a message or an object. Not to be confused with objectPayload.

Known magic values:

Magic value Sent over wire as
0xE9BEB4D9 E9 BE B4 D9

Variable length integer

Integer can be encoded depending on the represented value to save space. Variable length integers always precede an array/vector of a type of data that may vary in length. Varints MUST use the minimum possible number of bytes to encode a value. For example; the value 6 can be encoded with one byte therefore a varint that uses three bytes to encode the value 6 is malformed and the decoding task must be aborted.

Value Storage length Format
< 0xfd 1 uint8_t
<= 0xffff 3 0xfd followed by the integer as uint16_t
<= 0xffffffff 5 0xfe followed by the integer as uint32_t
- 9 0xff followed by the integer as uint64_t

Variable length string

Variable length string can be stored using a variable length integer followed by the string itself.

Field Size Description Data type Comments
1+ length var_int Length of the string
? string char[] The string itself (can be empty)

Variable length list of integers

n integers can be stored using n+1 variable length integers where the first var_int equals n.

Field Size Description Data type Comments
1+ count var_int Number of var_ints below
1+ var_int The first value stored
1+ var_int The second value stored...
1+ var_int etc...

Network address

When a network address is needed somewhere, this structure is used. Network addresses are not prefixed with a timestamp or stream in the version message.

Field Size Description Data type Comments
8 time uint64 the Time.
4 stream uint32 Stream number for this node
8 services uint64_t same service(s) listed in version
16 IPv6/4 char[16] IPv6 address. IPv4 addresses are written into the message as a 16 byte IPv4-mapped IPv6 address

(12 bytes 00 00 00 00 00 00 00 00 00 00 FF FF, followed by the 4 bytes of the IPv4 address).

2 port uint16_t port number

Inventory Vectors

Inventory vectors are used for notifying other nodes about objects they have or data which is being requested. Two rounds of SHA-512 are used, resulting in a 64 byte hash. Only the first 32 bytes are used; the later 32 bytes are ignored.

Inventory vectors consist of the following data format:

Field Size Description Data type Comments
32 hash char[32] Hash of the object

Encrypted payload

Bitmessage uses ECIES to encrypt its messages. For more information see Encryption

Field Size Description Data type Comments
16 IV uchar[] Initialization Vector used for AES-256-CBC
2 uint16_t Curve type Elliptic Curve type 0x02CA (714)
2 uint16_t X length Length of X component of public key R
X length uchar[] X X component of public key R
2 uint16_t Y length Length of Y component of public key R
Y length uchar[] Y Y component of public key R
? encrypted uchar[] Cipher text
32 MAC uchar[] HMACSHA256 Message Authentication Code

Unencrypted Message Data

Field Size Description Data type Comments
1+ msg_version var_int Message format version. This field is not included after the protocol v3 upgrade period.
1+ address_version var_int Sender's address version number. This is needed in order to calculate the sender's address to show in the UI, and also to allow for forwards compatible changes to the public-key data included below.
1+ stream var_int Sender's stream number
4 behavior bitfield uint32_t A bitfield of optional behaviors and features that can be expected from the node with this pubkey included in this msg message (the sender's pubkey).
64 public signing key uchar[] The ECC public key used for signing (uncompressed format; normally prepended with \x04 )
64 public encryption key uchar[] The ECC public key used for encryption (uncompressed format; normally prepended with \x04 )
1+ nonce_trials_per_byte var_int Used to calculate the difficulty target of messages accepted by this node. The higher this value, the more difficult the Proof of Work must be before this individual will accept the message. This number is the average number of nonce trials a node will have to perform to meet the Proof of Work requirement. 1000 is the network minimum so any lower values will be automatically raised to 1000. This field is new and is only included when the address_version >= 3.
1+ extra_bytes var_int Used to calculate the difficulty target of messages accepted by this node. The higher this value, the more difficult the Proof of Work must be before this individual will accept the message. This number is added to the data length to make sending small messages more difficult. 1000 is the network minimum so any lower values will be automatically raised to 1000. This field is new and is only included when the address_version >= 3.
20 destination ripe uchar[] The ripe hash of the public key of the receiver of the message
1+ encoding var_int Message Encoding type
1+ message_length var_int Message Length
message_length message uchar[] The message.
1+ ack_length var_int Length of the acknowledgement data
ack_length ack_data uchar[] The acknowledgement data to be transmitted. This takes the form of a Bitmessage protocol message, like another msg message. The POW therein must already be completed.
1+ sig_length var_int Length of the signature
sig_length signature uchar[] The ECDSA signature which covers the object header starting with the time, appended with the data described in this table down to the ack_data.

Message Encodings

Value Name Description
0 IGNORE Any data with this number may be ignored. The sending node might simply be sharing its public key with you.
1 TRIVIAL UTF-8. No 'Subject' or 'Body' sections. Useful for simple strings of data, like URIs or magnet links.
2 SIMPLE UTF-8. Uses 'Subject' and 'Body' sections. No MIME is used.

messageToTransmit = 'Subject:' + subject + '\n' + 'Body:' + message

3 EXTENDED See Extended encoding

Further values for the message encodings can be decided upon by the community. Any MIME or MIME-like encoding format, should they be used, should make use of Bitmessage's 8-bit bytes.

Pubkey bitfield features

Bit Name Description
0 undefined The most significant bit at the beginning of the structure. Undefined
1 undefined The next most significant bit. Undefined
... ... ...
27 onion_router (Proposal) Node can be used to onion-route messages. In theory any node can onion route, but since it requires more resources, they may have the functionality disabled. This field will be used to indicate that the node is willing to do this.
28 forward_secrecy (Proposal) Receiving node supports a forward secrecy encryption extension. The exact design is pending.
29 chat (Proposal) Address if for chatting rather than messaging.
30 include_destination (Proposal) Receiving node expects that the RIPE hash encoded in their address preceedes the encrypted message data of msg messages bound for them.

NOTE: since hardly anyone implements this, this will be redesigned as simple recipient verification: https://github.com/Bitmessage/PyBitmessage/pull/808#issuecomment-170189856

31 does_ack If true, the receiving node does send acknowledgements (rather than dropping them).

Message types

Undefined messages received on the wire must be ignored.

version

When a node creates an outgoing connection, it will immediately advertise its version. The remote node will respond with its version. No futher communication is possible until both peers have exchanged their version.

Payload:

Field Size Description Data type Comments
4 version int32_t Identifies protocol version being used by the node. Should equal 3. Nodes should disconnect if the remote node's version is lower but continue with the connection if it is higher.
8 services uint64_t bitfield of features to be enabled for this connection
8 timestamp int64_t standard UNIX timestamp in seconds
26 addr_recv net_addr The network address of the node receiving this message (not including the time or stream number)
26 addr_from net_addr The network address of the node emitting this message (not including the time or stream number and the ip itself is ignored by the receiver)
8 nonce uint64_t Random nonce used to detect connections to self.
1+ user_agent var_str User Agent (0x00 if string is 0 bytes long). Sending nodes must not include a user_agent longer than 5000 bytes.
1+ stream_numbers var_int_list The stream numbers that the emitting node is interested in. Sending nodes must not include more than 160000 stream numbers.

A "verack" packet shall be sent if the version packet was accepted. Once you have sent and received a verack messages with the remote node, send an addr message advertising up to 1000 peers of which you are aware, and one or more inv messages advertising all of the valid objects of which you are aware.

The following services are currently assigned:

Value Name Description
1 NODE_NETWORK This is a normal network node.
2 NODE_SSL This node supports SSL/TLS in the current connect (python < 2.7.9 only supports a SSL client, so in that case it would only have this on when the connection is a client).
3 NODE_POW (Proposal) This node may do PoW on behalf of some its peers (PoW offloading/delegating), but it doesn't have to. Clients may have to meet additional requirements (e.g. TLS authentication)
4 NODE_DANDELION Node supports dandelion (https://github.com/gfanti/bips/blob/master/bip-dandelion.mediawiki)

verack

The verack message is sent in reply to version. This message consists of only a message header with the command string "verack". The TCP timeout starts out at 20 seconds; after verack messages are exchanged, the timeout is raised to 10 minutes.

If both sides announce that they support SSL, they MUST perform a SSL handshake immediately after they both send and receive verack. During this SSL handshake, the TCP client acts as a SSL client, and the TCP server acts as a SSL server. The current implementation (v0.5.4 or later) requires the AECDH-AES256-SHA cipher over TLSv1 protocol, and prefers the secp256k1 curve (but other curves may be accepted, depending on the version of python and OpenSSL used).

addr

Provide information on known nodes of the network. Non-advertised nodes should be forgotten after typically 3 hours

Payload:

Field Size Description Data type Comments
1+ count var_int Number of address entries (max: 1000)
38 addr_list net_addr Address of other nodes on the network.

inv

Allows a node to advertise its knowledge of one or more objects. Payload (maximum payload length: 50000 items):

Field Size Description Data type Comments
? count var_int Number of inventory entries
32x? inventory inv_vect[] Inventory vectors

getdata

getdata is used in response to an inv message to retrieve the content of a specific object after filtering known elements.

Payload (maximum payload length: 50000 entries):

Field Size Description Data type Comments
? count var_int Number of inventory entries
32x? inventory inv_vect[] Inventory vectors

object

An object is a message which is shared throughout a stream. It is the only message which propagates; all others are only between two nodes. Objects have a type, like 'msg', or 'broadcast'. To be a valid object, the Proof Of Work must be done. The maximum allowable length of an object (not to be confused with the objectPayload) is 218 bytes.

Field Size Description Data type Comments
8 nonce uint64_t

Random nonce used for the Proof Of Work

8 expiresTime uint64_t

The "end of life" time of this object (be aware, in version 2 of the protocol this was the generation time). Objects shall be shared with peers until its end-of-life time has been reached. The node should store the inventory vector of that object for some extra period of time to avoid reloading it from another node with a small time delay. The time may be no further than 28 days + 3 hours in the future.

4 objectType uint32_t

Four values are currently defined: 0-"getpubkey", 1-"pubkey", 2-"msg", 3-"broadcast". All other values are reserved. Nodes should relay objects even if they use an undefined object type.

1+ version var_int The object's version. Note that msg objects won't contain a version until Sun, 16 Nov 2014 22:00:00 GMT.
1+ stream number var_int The stream number in which this object may propagate
? objectPayload uchar[]

This field varies depending on the object type; see below.

Object types

Here are the payloads for various object types.

getpubkey

When a node has the hash of a public key (from an address) but not the public key itself, it must send out a request for the public key.

Field Size Description Data type Comments
20 ripe uchar[] The ripemd hash of the public key. This field is only included when the address version is <= 3.
32 tag uchar[] The tag derived from the address version, stream number, and ripe. This field is only included when the address version is >= 4.

pubkey

A version 2 pubkey. This is still in use and supported by current clients but new v2 addresses are not generated by clients.

Field Size Description Data type Comments
4 behavior bitfield uint32_t A bitfield of optional behaviors and features that can be expected from the node receiving the message.
64 public signing key uchar[] The ECC public key used for signing (uncompressed format; normally prepended with \x04 )
64 public encryption key uchar[] The ECC public key used for encryption (uncompressed format; normally prepended with \x04 )

A version 3 pubkey

Field Size Description Data type Comments
4 behavior bitfield uint32_t A bitfield of optional behaviors and features that can be expected from the node receiving the message.
64 public signing key uchar[] The ECC public key used for signing (uncompressed format; normally prepended with \x04 )
64 public encryption key uchar[] The ECC public key used for encryption (uncompressed format; normally prepended with \x04 )
1+ nonce_trials_per_byte var_int Used to calculate the difficulty target of messages accepted by this node. The higher this value, the more difficult the Proof of Work must be before this individual will accept the message. This number is the average number of nonce trials a node will have to perform to meet the Proof of Work requirement. 1000 is the network minimum so any lower values will be automatically raised to 1000.
1+ extra_bytes var_int Used to calculate the difficulty target of messages accepted by this node. The higher this value, the more difficult the Proof of Work must be before this individual will accept the message. This number is added to the data length to make sending small messages more difficult. 1000 is the network minimum so any lower values will be automatically raised to 1000.
1+ sig_length var_int Length of the signature
sig_length signature uchar[] The ECDSA signature which, as of protocol v3, covers the object header starting with the time, appended with the data described in this table down to the extra_bytes.

A version 4 pubkey

Field Size Description Data type Comments
32 tag uchar[] The tag, made up of bytes 32-64 of the double hash of the address data (see example python code below)
? encrypted uchar[] Encrypted pubkey data.

When version 4 pubkeys are created, most of the data in the pubkey is encrypted. This is done in such a way that only someone who has the Bitmessage address which corresponds to a pubkey can decrypt and use that pubkey. This prevents people from gathering pubkeys sent around the network and using the data from them to create messages to be used in spam or in flooding attacks.

In order to encrypt the pubkey data, a double SHA-512 hash is calculated from the address version number, stream number, and ripe hash of the Bitmessage address that the pubkey corresponds to. The first 32 bytes of this hash are used to create a public and private key pair with which to encrypt and decrypt the pubkey data, using the same algorithm as message encryption (see Encryption). The remaining 32 bytes of this hash are added to the unencrypted part of the pubkey and used as a tag, as above. This allows nodes to determine which pubkey to decrypt when they wish to send a message.

In PyBitmessage, the double hash of the address data is calculated using the python code below:

doubleHashOfAddressData = hashlib.sha512(hashlib.sha512(encodeVarint(addressVersionNumber) + encodeVarint(streamNumber) + hash).digest()).digest()

Encrypted data in version 4 pubkeys:

Field Size Description Data type Comments
4 behavior bitfield uint32_t A bitfield of optional behaviors and features that can be expected from the node receiving the message.
64 public signing key uchar[] The ECC public key used for signing (uncompressed format; normally prepended with \x04 )
64 public encryption key uchar[] The ECC public key used for encryption (uncompressed format; normally prepended with \x04 )
1+ nonce_trials_per_byte var_int Used to calculate the difficulty target of messages accepted by this node. The higher this value, the more difficult the Proof of Work must be before this individual will accept the message. This number is the average number of nonce trials a node will have to perform to meet the Proof of Work requirement. 1000 is the network minimum so any lower values will be automatically raised to 1000.
1+ extra_bytes var_int Used to calculate the difficulty target of messages accepted by this node. The higher this value, the more difficult the Proof of Work must be before this individual will accept the message. This number is added to the data length to make sending small messages more difficult. 1000 is the network minimum so any lower values will be automatically raised to 1000.
1+ sig_length var_int Length of the signature
sig_length signature uchar[] The ECDSA signature which covers everything from the object header starting with the time, then appended with the decrypted data down to the extra_bytes. This was changed in protocol v3.

msg

Used for person-to-person messages. Note that msg objects won't contain a version in the object header until Sun, 16 Nov 2014 22:00:00 GMT.

Field Size Description Data type Comments
? encrypted uchar[] Encrypted data. See Encrypted payload. See also Unencrypted Message Data Format

broadcast

Users who are subscribed to the sending address will see the message appear in their inbox. Broadcasts are version 4 or 5.

Pubkey objects and v5 broadcast objects are encrypted the same way: The data encoded in the sender's Bitmessage address is hashed twice. The first 32 bytes of the resulting hash constitutes the "private" encryption key and the last 32 bytes constitute a tag so that anyone listening can easily decide if this particular message is interesting. The sender calculates the public key from the private key and then encrypts the object with this public key. Thus anyone who knows the Bitmessage address of the sender of a broadcast or pubkey object can decrypt it.

The version of broadcast objects was previously 2 or 3 but was changed to 4 or 5 for protocol v3. Having a broadcast version of 5 indicates that a tag is used which, in turn, is used when the sender's address version is >=4.

Field Size Description Data type Comments
32 tag uchar[] The tag. This field is new and only included when the broadcast version is >= 5. Changed in protocol v3
? encrypted uchar[] Encrypted broadcast data. The keys are derived as described in the paragraph above. See Encrypted payload for details about the encryption algorithm itself.

Unencrypted data format:

Field Size Description Data type Comments
1+ broadcast version var_int The version number of this broadcast protocol message which is equal to 2 or 3. This is included here so that it can be signed. This is no longer included in protocol v3
1+ address version var_int The sender's address version
1+ stream number var_int The sender's stream number
4 behavior bitfield uint32_t A bitfield of optional behaviors and features that can be expected from the owner of this pubkey.
64 public signing key uchar[] The ECC public key used for signing (uncompressed format; normally prepended with \x04 )
64 public encryption key uchar[] The ECC public key used for encryption (uncompressed format; normally prepended with \x04 )
1+ nonce_trials_per_byte var_int Used to calculate the difficulty target of messages accepted by this node. The higher this value, the more difficult the Proof of Work must be before this individual will accept the message. This number is the average number of nonce trials a node will have to perform to meet the Proof of Work requirement. 1000 is the network minimum so any lower values will be automatically raised to 1000. This field is new and is only included when the address_version >= 3.
1+ extra_bytes var_int Used to calculate the difficulty target of messages accepted by this node. The higher this value, the more difficult the Proof of Work must be before this individual will accept the message. This number is added to the data length to make sending small messages more difficult. 1000 is the network minimum so any lower values will be automatically raised to 1000. This field is new and is only included when the address_version >= 3.
1+ encoding var_int The encoding type of the message
1+ messageLength var_int The message length in bytes
messageLength message uchar[] The message
1+ sig_length var_int Length of the signature
sig_length signature uchar[] The signature which did cover the unencrypted data from the broadcast version down through the message. In protocol v3, it covers the unencrypted object header starting with the time, all appended with the decrypted data.
Retrieved from "https://wiki.bitmessage.org/index.php?title=Protocol_specification&oldid=48663"