|
|
| (54 intermediate revisions by 4 users not shown) |
| Line 1: |
Line 1: |
| The Lobbys currently use two download systems: Torrent and Rapid
| | There are curently three systems for spring content files: |
| | *[[rapid]] is currently for games only (rsync like) |
| | *[[upq]] allows to search for games maps and engine |
| | *[[plasma]] allows to search for games and maps |
|
| |
|
| | These systems allow to automaticly download files. |
|
| |
|
| ==Rapid== | | ==clients/implementations== |
| Rapid is used to list all mods in the (online-)repositry
| | *[[SpringLobby]] (c++, cross-platform) |
| | | *[[chobby]] (lua, cross-platform) |
| take a mod archive
| | *[[pr-downloader]] (c++, cross-platform) |
| atm the archive is a single blob
| | [[Category: Download_Systems]] |
| rapid takes all the files in an archive
| |
| it zips them separately
| |
| and gives to each zip, as filename, the md5 of the file itself
| |
| those files belong to the pool folder
| |
| mods can share files as long as the md5 is the same
| |
| a mod, is a binary file
| |
| that is what can be seen in ~/.spring/pool/ ?
| |
| which lists all the md5 files that belong to the mod itself
| |
| | |
| filesize is useless, since it assumes same md5 == same file, ie assumes no collision
| |
| | |
| then, there's a repository of mods
| |
| ie, a list of available mods, and the pool files they need
| |
| since now on, i'll refer to a mod as the file which is list of md5 that it needs
| |
| unless specified otherwise
| |
| spring can read those mods and pool folder directly
| |
| unpacking the files on the fly and reading them like it'd be a monolithic archive
| |
| then, there's yet another repository, which contains the list of repositories
| |
| if you picture in a tree graph
| |
| there's list of repositories on top
| |
| then splits to multiple repos
| |
| then each repos has multiple mods
| |
| | |
| http://repos.caspring.org/repos.gz is the master
| |
| | |
| what a rapid downloader needs to do:
| |
| download the list of repositories
| |
| for each repository
| |
| download the list of available mods
| |
| the each repo has also info about latest version of a mod
| |
| then, give a list of available mods ( packages in linux package manager terms ), and eventually some meta package that represents the latest version of a mod
| |
| on download request, download the mod descriptor
| |
| parse it to get the list of md5
| |
| check the available md5 local in the pool folder, and download the missing
| |
| upon download of each file, the rapid tool should deflate each archive and verify that the md5 is the same as the file
| |
| if not, reject it
| |
| | |
| if the test is successful, move the md5 files to the pool folder
| |
| the mod descriptor
| |
| belongs to the packages folder instead
| |
| with filename == md5 hash
| |
| and sdp extension
| |
| extra functionality, which is optional:
| |
| add possibility to delete mods ( since the filenames use md5 hash, it's quite unclear to the user which files he should remove )
| |
| and a garbage collector which removes the files from the pool which aren't linked by mod
| |
| | |
| simple reference counting should suffice for garbace collecting
| |
| | |
| | |
| ===in short===
| |
| rapid downloads the master file with all repos: http://repos.caspring.org/repos.gz
| |
| then it downloads from all repos /versions.gz ?
| |
| for example here http://packages.caspring.org/
| |
| | |
| ca-1faction:revision:6355,054cf15829e79c357a3be50236e00c4f,Complete Annihilation test-6355,Complete Annihilation 1 faction test-6355
| |
| the first section is the name: "ca-1faction:revision:6355"
| |
| | |
| the package will be downloaded from hash +.sdp
| |
| so 054cf15829e79c357a3be50236e00c4f.sdp
| |
| that file belongs to the local packages folder
| |
| that file contains the list of md5
| |
| that the mod requires
| |
| so you should open it
| |
| and then check the local pool folder
| |
| for all the missing files
| |
| | |
| http://packages.caspring.org/packages/054cf15829e79c357a3be50236e00c4f.sdp
| |
| this file contains binary md5 + filename of the original file
| |
| | |
| remember: you must keep the file names untouched
| |
| as in, everything is indexed by md5
| |
| | |
| to download the md5 files there's a streamer which will speed up the download
| |
| ==Torrent==
| |
| | |
| The torrent system can be used to download a specific file by filename
| |
| | |
| | |
| first you request the list of available files to plasma server:
| |
| | |
| (http://planet-wars.eu/PlasmaServer/Service.asmx)
| |
| | |
| iirc it uses XML + SOAP
| |
| it will give a list of archive names + path to the torrent
| |
| for a given file
| |
| you download the torrent
| |
| and query the server for the http paths
| |
| as in, will return a list of http mirrors of the file
| |
| | |
| the global list
| |
| gives a link to the torrent
| |
| the single file query
| |
| gives a list of http mirrors for the file itself
| |
| ie the map or te mod
| |
| | |
| so basically, it works like this:
| |
| get the list of files from the server
| |
| dowload the torrent file
| |
| at the same time request the list of http seeds
| |
| for the same file
| |
| open the torrent file as soon as it's downloaded
| |
| and start it
| |
| as soon as the http seeds arrive
| |
| add them to the torrent itself
| |
| so it will use them as additional source
| |
| | |
| this will speed up greatly the download
| |
There are curently three systems for spring content files:
- rapid is currently for games only (rsync like)
- upq allows to search for games maps and engine
- plasma allows to search for games and maps
These systems allow to automaticly download files.
clients/implementations
