|
|
| Line 49: |
Line 49: |
|
| |
|
|
| |
|
|
| |
| ===general description===
| |
| Rapid is used to list all games in the (online-) repositry
| |
|
| |
| as spring supports also reading such a structure, there are some docs in the [http://github.com/spring/spring/blob/master/rts/System/FileSystem/Archives/PoolArchive.h sourcecode]
| |
|
| |
| here is a short description how it works:
| |
|
| |
| take a game archive
| |
|
| |
| atm the archive is a single blob
| |
| rapid takes all the files in an archive it zips them separately and gives to each zip, as filename, the md5 of the file itself those files belong to the pool folder.
| |
| games can share files as long as the md5 is the same.
| |
| A game, is a binary file that is what can be seen in ~/.spring/pool/
| |
|
| |
| ~/.spring/packages containts all games with the associated md5s
| |
|
| |
| The filesize is useless, since it assumes same md5 == same file, ie assumes no collision.
| |
|
| |
| Then, there's a repository of games:
| |
| ie, a list of available games, and the pool files they need
| |
| since now on, i'll refer to a game as the file which is list of md5 that it needs
| |
| unless specified otherwise
| |
| spring can read those games and pool folder directly
| |
| unpacking the files on the fly and reading them like it'd be a monolithic archive
| |
| then, there's yet another repository, which contains the list of repositories
| |
| if you picture in a tree graph
| |
| there's list of repositories on top
| |
| then splits to multiple repos
| |
| then each repos has multiple games
| |
|
| |
| http://repos.springrts.com/repos.gz is the master
| |
|
| |
| what a rapid downloader needs to do:
| |
| download the list of repositories
| |
| for each repository
| |
| download the list of available games
| |
| the each repo has also info about latest version of a game
| |
| then, give a list of available games ( packages in linux package manager terms ), and eventually some meta package that represents the latest version of a game
| |
| on download request, download the game descriptor
| |
| parse it to get the list of md5
| |
| check the available md5 local in the pool folder, and download the missing
| |
| upon download of each file, the rapid tool should deflate each archive and verify that the md5 is the same as the file
| |
| if not, reject it
| |
|
| |
| if the test is successful, move the md5 files to the pool folder
| |
| the game descriptor
| |
| belongs to the packages folder instead
| |
| with filename == md5 hash
| |
| and sdp extension
| |
| extra functionality, which is optional:
| |
| add possibility to delete games ( since the filenames use md5 hash, it's quite unclear to the user which files he should remove )
| |
| and a garbage collector which removes the files from the pool which aren't linked by game
| |
|
| |
| simple reference counting should suffice for garbace collecting
| |
|
| |
| ===in short===
| |
| rapid downloads the master file with all repos:
| |
| http://repos.springrts.com/repos.gz
| |
| then it downloads from all repos /versions.gz
| |
| for example here http://packages.caspring.org/versions.gz
| |
|
| |
| ca-1faction:revision:6355,054cf15829e79c357a3be50236e00c4f,Complete Annihilation test-6355,Complete Annihilation 1 faction test-6355
| |
| the first section is the name: "ca-1faction:revision:6355"
| |
|
| |
| the package will be downloaded from hash +.sdp
| |
| so 054cf15829e79c357a3be50236e00c4f.sdp
| |
| that file belongs to the local packages folder
| |
| that file contains the list of md5
| |
| that the mod requires
| |
| so you should open it
| |
| and then check the local pool folder
| |
| for all the missing files
| |
|
| |
| http://packages.caspring.org/packages/054cf15829e79c357a3be50236e00c4f.sdp
| |
| this file contains binary md5 + filename of the original file
| |
|
| |
| remember: you must keep the file names untouched
| |
| as in, everything is indexed by md5
| |
|
| |
| to download the md5 files there's a streamer which will speed up the download
| |
|
| |
|
| =Using rapid as Gamedev= | | =Using rapid as Gamedev= |
"Rapid" is a concept for storing "archives" for spring. It consists of multiple indexes:
spring can directly read .sdp and the pool directory which contains the real files of an archive. multiple .sdp can share the same files in a pool which makes "rapid" so efficient in storage.
Serverside RapidTools are used to manage the repository of all games.
Files structures
repos.gz
gziped text file which has a list of all rapid repositories.
main,http://packages.springrts.com,,
versions.gz
gziped text file which has a list of all sdp archives in this repository.
ba-tronic:version:7.12-tronic1,baafb4fb215c60873e7d10e7ec600dc5,Balanced Annihilation V7.12,Balanced Annihilation V7.12-tronic1
the md5 hash is the filename of the sdp of the archive. so for example:
http://packages.springrts.com/packages/baafb4fb215c60873e7d10e7ec600dc5.sdp
.sdp
gziped index file with a list of all files in this archive. this is for example a game.
filename of the .sdp is:
md5(md5(filename1) + md5(content of file1) + md5(filename2) + md5(content of file2) ...)
see void PoolArchiveT::load(DigestT const & Digest):
https://github.com/spring/RapidTools/blob/master/Rapid/PoolArchive.cpp#L35
or
https://github.com/spring/spring/blob/master/rts/System/FileSystem/Archives/PoolArchive.h
pool
this folder contains all files of the .sdp's in gzipped form. the filename is the md5 of the file. this way files which are used by multiple archives can be shared.
Using rapid as Gamedev
If a Gamedev wants to use rapid, he needs to use a svn repository for his game where the game archive is in the .sdd format stored:
The svn repository needs to contains a folder which is in the structre of a .sdd. in modinfo.lua version has to be
version = '$VERSION',
$VERSION later on will be replaced by the svn commit number or, when a release is made, the version number of the release.
creating a release
When the SVN repository is connected to rapid, a release can be made by making a svn-commit with a commit message in the format:
VERSION{x.xx}
Where everything in the brackets will replace $VERSION in modinfo.lua.
This will create a tag :stable for this commit and create an archive at http://packages.springrts.com/builds/ which can be distributed/uploaded to different sites if needed.
It is also automatically published to upq at http://api.springfiles.com/ which all current download systems have access to; you can validate what happened there, too.
adding your game to the rapid download system
This is deprecated, please use a svn repository!
forum thread
To download you can use one of the Rapid_Tags.
