Inside BitComet

Inside BitComet

From BitComet Wiki

Jump to: navigation, search

Contents

Inside BitComet(Technical Infomation)

PeerID Format

  • PeerID Format -- will be applied in v0.59 or later.

The format of peer_id in v0.59 or later follows the Azureus-style which is stated here. It uses the following encoding: '-', two characters for client id, four ascii digits for version number, '-', followed by random numbers.

For example, the peer id of BitComet v0.59 will be '-BC0059-'...

  • PeerID Format before v0.59 -- old format before v0.59

if ( memcmp(peerid, "exbc", 4) == 0 ) { version = "BitComet "; version += (unsigned char)buf[4]+(unsigned char)'0'; version += '.'; version += ((unsigned char)buf[5])/10+(unsigned char)'0'; version += ((unsigned char)buf[5])%10+(unsigned char)'0'; }


Torrent File Format

1. Torrent File Format -- will be applied in v0.59 or later, contents important notice with regards to encoding

 The torrent file format below is used in BitComet v0.59 and later.

The content of a metainfo file (the file ending in ".torrent") is a bencoded dictionary, containing the keys listed below. Keys not marked 'optional' are always generated in BitComet:

  "info": dictionary that describes the file(s) of the torrent. There are two possible forms: one for the case of a 'single-file' torrent with no directory structure, and one for the case of a 'multi-file' torrent, which may contain subdirectory trees.

In the case of the single-file mode:

   "name": the filename of the file. (string)

   "length": length of the file in bytes (integer)

In the case of the multi-file mode:

   "name": the name of the top-most directory in the structure -- the directory which contains all of the files listed in the above files list. (character string)

   "files": a list of dictionaries, one for each file. Each dictionary in this list contains the following keys:

    a)"length": length of the file in bytes (integer)

    b)"path": a list containing one or more string elements that, together, represent the path and filename. Each element in the list corresponds to either a directory name or (in the case of the final element) the filename. For example, a the file "dir1/dir2/file.ext" would consist of three string elements: "dir1", "dir2", and "file.ext". This is encoded as a bencoded list of strings such as l4:dir14:dir28:file.exte Important Notice: BitComet also inlcude empty directory when creating torrent, that is the file name here may be zero-length string.

In both cases:

   "piece length": number of bytes in each piece (integer)

   "pieces": string consisting of the chaining together of all 20-byte SHA1 hash values, one per piece (byte string)

   "publisher": (optional) the name of the torrent creator, if do not allow to be changed(string)

   "publisher-url": (optional) the url of the torrent creator if do not allow to be changed (string)


"encoding": the string encoding (string), in v0.59 or above is always "UTF-8"

"creation date": the creation time of the torrent, in standard Unix epoch format (integer seconds since 1-Jan-1970 00:00:00 UTC)

"created by": name and version of the program used to create the .torrent (string), like this: "BitComet/0.59"


"announce": (optional) The announce URL of the tracker (string)

"announce-list": (optional) multitracker list (a list of string)

"nodes": (optional) A list of DHT Network nodes. Each node is encoded as a 2-element list of host (string) and port (integer)

"private": (optional) if the torrent is Private Torrent (SecureTorrent), that is to announce only in tracker or tracker-list (integer, 0 or 1)


"publisher": (optional) the name of the torrent creator (string)

"publisher-url": (optional) the url of the torrent creator (string)

"comment": (optional) free-form textual comments by the author (string)


Important Notice: In BitComet v0.58 or prior, the string was encoded using MSCB (user's code page), and a ".utf-8" key was added for UTF-8 encoded string. In v0.59, the default encoding was changed to UTF-8, if the user's code page was neither Chinese nor Japanese. BitComet will still keep adding a ".utf-8" key for all string for backward-compatibility (e.g. add a "name.utf-8" key to store utf-8 file name). If most of the users upgrade their client in the near future, BitComet may stop adding a ".utf-8" key.

The reason why Chinese or Japanese torrents maintian the old format is because most of the torrent sites are not prepared for UTF-8 encoding. This situation may last for a rather long time. If you think it is better to move to all UTF-8 string torrent, please email us.

See also:

  http://wiki.theory.org/BitTorrentSpecification


2. Torrent File Format before v0.59 -- old format before v0.59

Please note: The torrent file format was changed in v0.59 and posterior!

Unicode Support, add ".utf-8" for those utf-8 encoding strings. UTF-8 values have higher priority than MBCS strings. Example:

"name" "name.utf-8" "path" "path.uft-8" "comment" "comment.utf-8" "publisher" "publisher.utf-8" "publisher-url" "publisher-url.utf-8" "announce-list" In addition to the standard "announce" key (in the main area of the metadata file and not part of the "info" section), there will be a new key, " announce-list". This key will refer to a list(s) of URLs, and will contain a list of tiers of announces. If the client is compatible with the multitracker specification, and if the "announce-list" key is present, the client will ignore the "announce" key and only use the URLs in "announce-list".

Message Stream Encryption (aka PHE) specification - new in v0.63

Message Stream Encryption BitComet v0.63 removed the old protocol header encryption, and support new Message Stream Encryption protocol which is compatible with Azureus and uTorrent.

The specification can be found here: http://azureus.aelitis.com/wiki/index.php/Message_Stream_Encryption

Tracker HTTP Protocol

1. Tracker HTTP Protocol -- will be applied in v0.59 or later, contents important notice regarding to SecureTorrent

 The Tracker HTTP Protocol below is used in BitComet v0.59 and later. 
  • Client->Tracker The tracker is an HTTP service which responds to HTTP GET requests. The base URL consists of the "announce URL", as defined in the metadata (.torrent) file. The parameters are then added to this URL, using standard CGI methods (i.e. a '?' after the announce URL, followed by 'param=value' sequences separated by '&') Note that all binary data in the URL (particularly info_hash and peer_id) must be properly escaped. This means that any byte not in the set 0-9, a-z, A-Z, and $-_.+!*'(), must be encoded using the "%nn" format, where nn is the hexadecimal value of the byte. (See RFC1738 for details.) The parameters used in the client->tracker GET request are as follows:
  • info_hash: 20-byte SHA1 hash of the value of the info key from the Metainfo file.
  • peer_id: 20-byte string used as a unique ID for the client, generated by the client at startup. See PeerID Format for the peer_id BitComet will use in v0.59 or later.
  • port: The port number that BitComet is listening on.
  • uploaded: The total number of bytes uploaded (since the client sent the 'started' event to the tracker) in base ten ASCII.
  • downloaded: The total number of bytes downloaded (since the client sent the 'started' event to the tracker) in base ten ASCII.
  • left: The number of bytes this client still has to download, encoded in base ten ASCII.
  • no_peer_id: Indicates that the client accepts a no_peer_id response. The peer_id key is then not required to be in the response of the tracker.
  • compact: Indicates that the client accepts a compact response. The peers list is replaced by a peers string with 6 bytes per peer. The first four bytes are the host (in network byte order), the last two bytes are the port (again in network byte order).
  • event: If specified, must be one of started, completed, stopped, (or empty which is the same as not being specified). If not specified, then this request is performed at regular intervals.
    1. started: The first request to the tracker must include the event key with the started value.
    2. stopped: Must be sent to the tracker, if the client is shutting down properly.
    3. completed: Must be sent to the tracker, when the download completes. It must not, however, be sent if the download was already 100% complete when the client started. Presumably, this is to allow the tracker to increment the "completed downloads" metric based solely on this event.
  • numwant: Optional. Number of peers that the client would like to receive from the tracker. This value is permitted to be zero. If omitted, typically defaults to 50 peers.
  • key: Optional. An additional identification that is not shared with any users. It is intended to allow a client to prove their identity, should their IP address change.
  • Tracker->Client The tracker responds with "text/plain" document, consisting of a bencoded dictionary with the following keys:
    • failure reason: Optional. If present, then no other keys may be present. The value is a human-readable error message as to why the request failed (string).
    • interval: Interval (in seconds) that the client should wait, between sending regular requests to the tracker (mandatory)

    min interval: Optional. Minimum announce interval. If present, clients must not reannounce more frequently than this.

    • peers: The value is a string of compacted peers <ip:port> 6 bytes in compact mode, or a list of dictionaries, each with the following keys:

      peer id: peer's self-selected ID, as described above for the tracker request (string)

      ip: peer's IP address (either IPv6 or IPv4) or DNS name (string)

      port: peer's port number (integer)

  • tracker_alias_url: Optional. a string list of of alias trackers. If the other tracker is in the same server, server domain may be omitted (e.g. udp://:8080/ signifies that the client can connect to a UDP tracker, instead of the http one).
  • private: Optional. If specified, can be 0 or 1 to indicate if the tracker is a private tracker. When a torrent is made under Disable Public DHT Network - Use Tracker Server - (that is to say that there is no nodes key in the torrent, and any tracker in the list announces that it is private tracker explicitly, by sending "private" = 1), BitComet will not add DHT Network as a Backup Tracker, even if all the trackers cannot be connected later, and will also disable Peer Exchange between peers.

    2. Tracker HTTP Protocol before v0.59 -- old spec before v0.59

      Please note: The Tracker HTTP Protocol was changed in v0.59 and subsequent versions! 
     
    

    [Request] Local IP Support "&localip=" Add internal IP announce so that peers in one subnet could connect to each other.

    [Request] Hide myself "&hide=1" Tell tracker "Do not tell my IP to others" (e.g. "Listening port is closed, no one can connected to me.")

    [Response] alias tracker reporting "tracker_alias_url" Contents a list of alias trackers. If in the same server, server domain can be omitted. e.g. udp://:8080/

    Multitracker Policy

    As the multitracker spec said, we define that the tracker group is comprised of several servers sharing peer information. BitComet will only connect to one of the group's members, but will try connecting to different tracker groups simultaneously. Detailed description can be found in Torrent Maker.


    BC Link Format

    A BC Link contains information for creating a BT task, HTTP task or FTP task.
    The basic plain format is as following:

      bc://bt2/${TaskName}/${FileSize}/${InfoHash}/
      bc://http2/${TaskName}/?url=${url}
      bc://ftp2/${TaskName}/?url=${url}
    

    In additional, more parameters can be added:

      bc://bt2/${TaskName}/${FileSize}/${InfoHash}/?torrent=${torrent_url}
      bc://http2/${TaskName}/?url=${url}&usr=${username}&psd=${password}&refer=${url_referer}&cookie=${url_cookie}
      bc://ftp2/${TaskName}/?url=${url}&usr=${username}&psd=${password}
    

    Finally, with the advantage of hiding details from end-user, the BASE64 encoded format is recommended to distribute for public:

      bc://bt/${BASE64_ENCODED}
      bc://http/${BASE64_ENCODED}
      bc://ftp/${BASE64_ENCODED}
      in which ${BASE64_ENCODED} = BASE64_ENCODE( "AA" + "/${TaskName}/..." + "ZZ" )
    

    Note: To support various languages besides English, all parameters should be UTF8 + URL_Escape encoded before Base64 encoding.

    SecureTorrent -- torrent site admin may would like to know

    Secure Torrent

    If one wants to ensure that BitComet obtains peers directly and exclusively from trackers, never finding peers through the DHT Network or by Peer Exchange, one may consider SecureTorrent.

    • For a torrent maker:

    Make the torrent file under "Only accept peers from tracker" (No DHT or Peer Exchange) and read Torrent Maker for more information. This option will add a private tag in the torrent file, and has the same meaning as SecureTorrent in Azureus.

    • For tracker admin:

    Add private keyword for the tracker HTTP response. Read Tracker HTTP Protocol for more information.

    • Notice

    The SecureTorrent does not work properly in v0.60 because it will sometimes connect to DHT Network. This bug was fixed in v0.61.

    See also:

    Private trackers

    A private tracker is a tracker that restricts who can use it, often by requiring registration of a user account. A common method for controlling registration among private trackers is an invitation system, in which trusted users are given the ability to grant a new user permission to register at the site. Typically, invitations (or the codes that are given to prospective users) are granted to users who have uploaded a pre-determined amount, meet specific ratio requirements, and have been registered. Private trackers usually register how much the users upload and download and may enforce a minimum upload-to-download ratio. As a result of the restricted access, their torrents usually offer better availability and speed, compared to public trackers where leeching is more common.

    Many private trackers are now implementing Passkeys in the torrent file, which gives each user a specific address to contact, in order to get a list of peers. This helps to prevent unauthorized distribution of torrent files from private trackers.


    When SecureTorrent is enabled, BitComet does not accept peers from DHT, PEX or LT Seeds.


    Back to Top