The next release of DebTorrent is now available. This release includes new functionality for communicating with APT using a new transport method specifically designed for debtorrent, and many bug fixes.

The major changes in this release are all in the communication between APT and DebTorrent. The HTTP server in DebTorrent that listens for APT requests has been upgraded to support HTTP/1.1 persistent connections and pipelining, which allows APT to have multiple outstanding requests for files. This is useful as DebTorrent suffers from the typical bittorrent slow start, so requesting multiple files at a time helps to speed up the download considerably.

Though better, HTTP/1.1 is not ideal for DebTorrent however, as a maximum of 10 outstanding requests is maintained by APT's http method, and files must still be returned in the order they were requested (which is not ideal for bittorrent-type downloading since downloads occur randomly).

To further improve the APT communication I have modified APT's http method to create a debtorrent method. This new debtorrent transport for APT is packaged separately as apt-transport-debtorrent, and once installed APT can be told to use it by replacing "http://" with "debtorrent://" in your sources.list file. This method sends all requests it receives immediately to DebTorrent, and will receive responses from DebTorrent in any order. You can find this new method on the Alioth project, or in my personal repository (only amd64 and i386 versions are currently available).

Unfortunately, the story doesn't end here. The APT code responsible for sending requests to the method also limits the maximum number of outstanding requests that it will send to the method to 10, which is not really necessary since all existing methods limit the requests they send out themselves. I have therefore patched the current APT code to increase this limit to 1000 (a one line change), and released this patched version as 0.7.6-0.1. You can find this patched version in my personal repository (again, only for i386 and amd64). I have tested it with the other methods available and it causes no problems, and I hope to get the change included in the regular APT code soon.

To sum up:

  • new DebTorrent over HTTP = fast
  • new DebTorrent with new apt-transport-debtorrent = faster
  • new DebTorrent with new apt-transport-debtorrent and a patched APT = fastest

The last DebTorrent version (0.1.3.1) is currently in the NEW queue, and judging by the length of it, will be there for about another week. After DebTorrent is added to the archive, I will be upgrading it to this new version. I also hope to get the new apt-transport-debtorrent package into the NEW queue soon.

This brings to an end the Google Summer of Code (which this project was created as a part of), but development of DebTorrent will of course continue (probably a little slower). The next major change will be the addition of unique piece numbers, which is almost complete but needs to be extensively tested. I'd like to thank Anthony Towns, Steve McIntyre, and Michael Vogt for their help over the last 4 months, and also the many others who sent me encouraging emails or engaged in interesting discussions about this project. It's the people who make a project like this a fun and memorable thing to do.

Here's the changelog for the new DebTorrent release:

  • APT communication supports HTTP/1.1 connections, including persistent connections and pipelining
  • Add support for the new debtorrent APT transport method (see the new apt-transport-debtorrent package)
  • Make the Packages decompression and torrent creation threaded
  • Improve the startup initialization of files
  • Add init and configuration files for the tracker
  • bug fixes:
    • restarts would fail when downloaded files have been modified
    • deleting old cached data would fail
    • small tracker bug causing exceptions
    • prevent enabling files before the initialization is complete
    • only connect to unique peers from the tracker that are not already connected
    • tracker would return all torrents' peers for every request