Inside BitComet
==== PeerID Format ====
-
PeerID Format - as applied in v0.59 or later\ The format of peer_id in v0.59 or later follows the Azureus-style which is stated here. It uses the following encoding: '-', two characters for client id, four ASCII digits for version number, '-', followed by random numbers.\ For example, the peer id of BitComet v0.59 will be '-BC0059-'...
-
PeerID Old Format - applied before v0.59\
Torrent File Format
-
1. Torrent File Format* -- as applied in v0.59 or later, contains important notice with regards to encoding.
The torrent file format below is used in BitComet v0.59 and later.
The content of a metainfo file (the file ending in ".torrent") is a bencoded dictionary, containing the keys listed below. Keys not marked 'optional' are always generated in BitComet:
-
*1)"info"**: dictionary that describes the file(s) of the torrent. There are two possible forms: one for the case of a single-file torrent with no directory structure, and one for the case of a multi-file torrent, which may contain subdirectory trees.
-
Info keys particular to the case of single-file mode:*
- "name": the filename of the file (string);
- "length": length of the file in bytes (integer);
-
Info keys particular to the case of multi-file mode:*
- "name": the name of the top-most directory in the structure -- the directory which contains all of the files listed in the above files list (character string) ;
- "files": a list of dictionaries, one for each file. Each dictionary in this list contains the following keys:
- "length": length of the file in bytes (integer);
- "path": a list containing one or more string elements that, together, represent the path and filename. Each element in the list corresponds to either a directory name or (in the case of the final element) the filename. For example, a the file "dir1/dir2/file.ext" would consist of three string elements: "dir1", "dir2", and "file.ext". This is encoded as a bencoded list of strings such as l4:dir14:dir28:file.exte Important Notice: BitComet also includes empty directory when creating torrent, that is the file name here may be zero-length string.
-
Info* <span id="privateflag", style=font-style:italic>keys common to both single-file and multi-file modes:
- "ed2k": (optional) 128 bit eMule hash for each file of the torrent;
- "filehash": (optional) 160 bit LT-Seeding hash for each file of the torrent;
- "piece length": number of bytes in each piece (integer);
- "pieces": string consisting of the chaining together of all 20-byte SHA1 hash values, one per piece (byte string);
- "private": (optional) if the torrent is a Private Torrent (SecureTorrent), BitComet must only announce itself to the private tracker or to a single tracker at any time, if there are several trackers in the announce-list (integer, 0 or 1)((Private Torrents));
- "publisher": (optional) the name of the torrent creator, if do not allow to be changed (string);
- "publisher-url": (optional) the URL of the torrent creator if do not allow to be changed (string);
-
*2) "encoding"**: the string encoding (string), in v0.59 or above is always "UTF-8";\
-
*3) "creation date"**: the creation time of the torrent, in standard Unix epoch format (integer seconds since 1-Jan-1970 00:00:00 UTC);\
-
*4) "created by"**: name and version of the program used to create the .torrent (string), like this: "BitComet/0.59";\
-
*5) "announce"**: (optional) The announce URL of the tracker (string);\
-
*6) "announce-list"**: (optional) multi-tracker list (a list of strings);\
-
*7) "nodes"**: (optional) A list of DHT Network nodes. Each node is encoded as a 2-element list of host (string) and port (integer);\
-
*8) "publisher"**: (optional) the name of the torrent creator (string);\
-
*9) "publisher-url"**: (optional) the url of the torrent creator (string);\
-
*10) "comment"**: (optional) free-form textual comments by the author (string);\
-
*11) "url-list"**: (optional) a list containing one or more URLs of the mirror server(s) used as webseeding download sources.((GetRight webseeding specifications))((HTTP/FTP webseeding - GetRight style)).
-
In BitComet v0.58 or prior, the string was encoded using MSCB (user's code page), and a ".utf-8" key was added for UTF-8 encoded string.
-
From v0.59, the default encoding was changed to UTF-8, if the user's code page was neither Chinese nor Japanese. BitComet will still keep adding a ".utf-8" key for all string for backward-compatibility (e.g. add a "name.utf-8" key to store utf-8 file name). If most of the users upgrade their client in the near future, BitComet may stop adding a ".utf-8" key. The reason why Chinese or Japanese torrents maintain the old format is because most of the torrent sites are not prepared for UTF-8 encoding. This situation may last for a rather long time. If you think it is better to move to all UTF-8 string torrent, please email us.
-
From v1.20, the default encoding was changed to UTF-8 for all languages, to meet BitTorrent official specification. BitComet will still keep adding a ".utf-8" key for all string for backward-compatibility.
See also: http://wiki.theory.org/BitTorrentSpecification
- 2. Torrent File Format before v0.59* -- old format before v0.59
Unicode Support, add ".utf-8" for those utf-8 encoding strings. UTF-8 values have higher priority than MBCS strings. Example:
"name" "name.utf-8" "path" "path.uft-8" "comment" "comment.utf-8" "publisher" "publisher.utf-8" "publisher-url" "publisher-url.utf-8" "announce-list" In addition to the standard "announce" key (in the main area of the metadata file and not part of the "info" section), there will be a new key, " announce-list". This key will refer to a list(s) of URLs, and will contain a list of tiers of announces. If the client is compatible with the multitracker specification, and if the "announce-list" key is present, the client will ignore the "announce" key and only use the URLs in "announce-list".
Message Stream Encryption (aka PHE) specification - new in v0.63
Message Stream Encryption\ BitComet v0.63 removed the old protocol header encryption, and supports the new Message Stream Encryption protocol which is compatible with Azureus and uTorrent.\ The specification can be found here: http://azureus.aelitis.com/wiki/index.php/Message_Stream_Encryption
Tracker HTTP Protocol
-
1. Tracker HTTP Protocol described below* -- is applied in v0.59 or later; it contains important notice regarding SecureTorrent
The Tracker HTTP Protocol below is used in BitComet v0.59 and later.
-
1) Client -> Tracker.* The tracker is an HTTP service which responds to HTTP GET requests. The base URL consists of the "announce URL", as defined in the metadata (.torrent) file. The parameters are then added to this URL, using standard CGI methods (i.e. a '?' after the announce URL, followed by 'param=value' sequences separated by '&').
Note that all binary data in the URL (particularly info_hash and peer_id) must be properly escaped. This means that any byte not in the set 0-9, a-z, A-Z, and $-_.+!*'(), must be encoded using the "%nn" format, where nn is the hexadecimal value of the byte. (See RFC 1738 for details.)
The parameters used in the client->tracker GET request are as follows:
-
info_hash: 20-byte SHA1 hash of the value of the info key from the Metainfo file.
-
peer_id: 20-byte string used as a unique ID for the client, generated by the client at startup. See PeerID Format for the peer_id BitComet will use in v0.59 or later.
-
port: The port number that BitComet is listening on.
-
uploaded: The total number of bytes uploaded (since the client sent the 'started' event to the tracker) in base ten ASCII.
-
downloaded: The total number of bytes downloaded (since the client sent the 'started' event to the tracker) in base ten ASCII.
-
left: The number of bytes this client still has to download, encoded in base ten ASCII.
-
no_peer_id: Indicates that the client accepts a no_peer_id response. The peer_id key is then not required to be in the response of the tracker.
-
compact: Indicates that the client accepts a compact response. The peers list is replaced by a peers string with 6 bytes per peer. The first four bytes are the host (in network byte order), the last two bytes are the port (again in network byte order).
-
event: If specified, must be one of started, completed, stopped, (or empty which is the same as not being specified). If not specified, then this request is performed at regular intervals.
- started: The first request to the tracker must include the event key with the started value.
- stopped: Must be sent to the tracker, if the client is shutting down properly.
- completed: Must be sent to the tracker, when the download completes. It must not, however, be sent if the download was already 100% complete when the client started. Presumably, this is to allow the tracker to increment the "completed downloads" metric based solely on this event.
-
numwant: Optional. Number of peers that the client would like to receive from the tracker. This value is permitted to be zero. If omitted, typically defaults to 50 peers.
-
key: Optional. An additional identification that is not shared with any users. It is intended to allow a client to prove their identity, should their IP address change.
-
2) Tracker -> Client*. The tracker responds with a "text/plain" document, consisting of a bencoded dictionary with the following keys:
- failure reason: Optional. If present, then no other keys may be present. The value is a human-readable error message as to why the request failed (string).
- interval: Interval (in seconds) that the client should wait, between sending regular requests to the tracker (mandatory) min interval: Optional. Minimum announce interval. If present, clients must not reannounce more frequently than this.
- peers: The value is a string of compacted peers ip:port 6 bytes in compact mode, or a list of dictionaries, each with the following keys:
- peer id: peer's self-selected ID, as described above for the tracker request (string)
- ip: peer's IP address (either IPv6 or IPv4) or DNS name (string)
- port: peer's port number (integer)
- tracker_alias_url: Optional. a string list of alias trackers. If the other tracker is in the same server, server domain may be omitted
- private: Optional. If specified, can be 0 or 1 to indicate if the tracker is a private tracker. When a torrent is made under Disable Public DHT Network - Use Tracker Server - (that is to say that there is no nodes key in the torrent, and every tracker in the list announces that it is private tracker explicitly, by sending "private" = 1), BitComet will not add DHT Network as a Backup Tracker, even if all the trackers cannot be connected later, and will also disable Peer Exchange between peers.\\
-
II. Tracker HTTP Protocol before v0.59* -- old spec before v0.59
Please note: The Tracker HTTP Protocol was changed in v0.59 and subsequent versions!
[Request] Local IP Support "&localip=" Add internal IP announce so that peers in one subnet could connect to each other.
[Request] Hide myself "&hide=1" Tell tracker "Do not tell my IP to others" (e.g. "Listening port is closed, no one can connected to me.")
[Response] alias tracker reporting "tracker_alias_url" Contents a list of alias trackers. If in the same server, server domain can be omitted.
Multitracker Policy
As the multi-tracker spec((Multitracker Metadata Extension)) says, we define that the tracker group is comprised of several servers sharing peer information. BitComet will only connect to one of the group's members, but will try connecting to different tracker groups simultaneously. Detailed description on their syntax can be found in Torrent Maker.
BC Link Format (obsoleted as of v.1.17)
A BC Link contains information for creating a BT task, HTTP task or FTP task. The basic plain format is as following:
bc://bt2/${TaskName}/${FileSize}/${InfoHash}/ bc://http2/${TaskName}/?url=${url} bc://ftp2/${TaskName}/?url=${url}
In additional, more parameters can be added:
bc://bt2/${TaskName}/${FileSize}/${InfoHash}/?torrent=${torrent_url}&infohash_v2=${InfoHashv2} bc://http2/${TaskName}/?url=${url}&usr=${username}&psd=${password}&refer=${url_referer}&cookie=${url_cookie}&user_agent=${user_agent}&mirror=${mirror_url}&custom_headers_for_mirrors=true bc://ftp2/${TaskName}/?url=${url}&usr=${username}&psd=${password}
Finally, with the advantage of hiding details from end-user, the BASE64 encoded format is recommended to distribute for public:
bc://bt/${BASE64_ENCODED} bc://http/${BASE64_ENCODED} bc://ftp/${BASE64_ENCODED} in which ${BASE64_ENCODED} = BASE64_ENCODE( "AA" + "/${TaskName}/..." + "ZZ" )
Note: To support various languages besides English, all parameters should be UTF8 + URL_Escape encoded before Base64 encoding. Note: If multiple HTTP mirrors available for a HTTP task, multiple "mirror" parameters can be added. Note: If "custom_headers_for_mirrors" parameter is true, the "referer", "cookie" and "user_agent" will also be sent to mirror addresses. Note: Starting with BitComet v.1.17 *BC Links* have been replaced by *Magnet Links* (see below).Magnet URI Format
A Magnet URI contains information for locating a P2P resource. BitComet can use it to start a BT download. The basic v1 format is as following:
magnet:?xt=urn:btih:
The basic v2 format with a 32 bytes info hash of SHA256:
magnet:?xt=urn:btmh:1220
The hybrid format with info hash v1 and v2:
magnet:?xt=urn:btih:
In additional, more parameters can be added:
magnet:?xt=urn:btih:
Mandatory parameter
-
xt*=urn:btih:
is a mandatory parameter for BitTorrent v1 magnetic links. -
* is the torrent info-hash-v1 hex encoded, for a total of 40 characters. The 32 character base32 encoded info-hash-v1 is also accepted. -
xt*=urn:btmh:1220
is a mandatory parameter for BitTorrent v2 magnetic links. -
* is the torrent info-hash-v2 hex encoded, for a total of 64 characters.
Optional parameters
- dn* is the user-friendly display name (which may be displayed while waiting for metadata); it should be UTF8 + URL_Escape encoded for non-English characters.\
- xl* is the task size.\
- tr* is a tracker URL, if there is one; for multiple trackers multiple "tr" parameters may be added.\
- ws* is the escaped web seed URL.\
- xs* is the escaped HTTP URL of torrent file.\
For a technical introduction to Magnet URIs please refer to wp>Magnet URI scheme.
Files used by BitComet
During BitComet's runtime, quite a lot of files are used or created on first start. Most of them are created in BitComet's "Application Data" folder (starting with BitComet v.1.18), others reside in BitComet's installation directory.
SecureTorrent -- info a torrent site admin might like to know
Private Torrents (Secure Torrent)
If one wants to ensure that BitComet obtains peers directly and exclusively from trackers, never finding peers through the DHT Network or by Peer Exchange, one should consider Private Torrents (Secure Torrent)((Private Torrents)).
- For a torrent maker:
Make the torrent file with the "Private torrent" (or "Only accept peers from tracker" in older versions) option enabled (no DHT or Peer Exchange allowed for that torrent and also no Torrent Exchange/Share, LT-Seeding or eDonkey connectivity enabled in BitComet). Read Making Torrents with BitComet for more information on how to enable this option upon making a torrent. This option will add a "private" tag in the torrent file, and has the same meaning as SecureTorrent in Azureus.
- For a tracker admin:
Add a private keyword in the tracker HTTP response. Read Tracker HTTP Protocol in this section for more information on all parameters used in client -> tracker communication and the keys contained in the tracker -> client response.
Note: The Private Torrent (SecureTorrent) flag does not work properly in v.0.60 because it will sometimes connect to the DHT Network. This bug was fixed in v0.61.
See also: SecureTorrent in Azureus.
Private trackers
A private tracker is a tracker that restricts who can use it, often by requiring registration of a user account. A common method for controlling registration among private trackers is an invitation system, in which trusted users are given the ability to grant a new user permission to register at the site. Typically, invitations (or the codes that are given to prospective users) are granted to users who have uploaded a pre-determined amount, meet specific ratio requirements, and have been registered. Private trackers usually register how much the users upload and download and may enforce a minimum upload-to-download ratio. As a result of the restricted access, their torrents usually offer better availability and speed, compared to public trackers where leeching is more common.
Many private trackers are now implementing Passkeys in the torrent file, which gives each user a specific address to contact, in order to get a list of peers. This helps to prevent unauthorized distribution of torrent files from private trackers.
- When SecureTorrent (private flag) is enabled, BitComet does not seek/share peers through DHT or PEX and disables Torrent Exchange/Share, LT-Seeding and eDonkey/Kad connectivity for that respective torrent.*
Installation Package and UAC
User Account Control (UAC) is a technology and security infrastructure introduced along with Microsoft's Windows Vista and Windows Server 2008 operating system and also present in the subsequent versions such as Windows 7. It aims to improve the security of Microsoft Windows by limiting the access rights of the software applications running in Windows to standard user privileges, by default, until an administrator authorizes an elevation.
Because running the BitComet installation package needs administrator privileges, in certain cases, unsuitable combinations between certain UAC settings and the user account type, may lead to installation failure.
As a basic guideline you should keep in mind that if you disabled UAC then you should run the BitComet installer under administrator account to make sure that the process won't fail.
For detailed information about the behavior of the installation package related to the user account type and UAC refer to this topic.
Hash Checking Conditions
BitComet will perform a Hash Checking for a BitTorrent task in one of the following conditions:
- When a user clicks "Manual Hash Check" in the right-click context menu of a stopped task, a Hash Checking will be performed immediately.
- When a new task is started, if BitComet finds that any same-name file exists in the save folder, a Hash Checking will be performed immediately.
- When an unfinished task is started, if BitComet finds any partially downloaded file missing, a Hash Checking will be performed immediately.
- When an unfinished task is started, if BitComet finds that the modification time attribute of a partially downloaded file has changed since last download, a Hash Checking will be performed when the download finishes.
- When a user changes the save folder of a stopped unfinished task, then starts this task, a Hash Checking will be performed immediately.
- When a task finished downloading, a Hash Checking will be performed to prepare it for Long-Time Seeding, if the torrent is not private and Long-Time Seeding is not disabled, but it was not supported by the torrent maker.
- When a task finished downloading, a Hash Checking will be performed to prepare for eMule plugin uploading, if the torrent is not private and the eMule plugin is not disabled, but it was not supported by the torrent maker.
- When a task finished downloading, a Hash Checking will be performed if the "bittorrent.hash_check_on_finished" option on the Advanced Options page is set to TRUE.
AJAX interface for Remote Download
see details in this page