Patch and scripts for offline rubygems usage

I would like to be able to use rubygems to install new gems when not connected to the internet. I don't want to just automatically install every gem, though. Specifically, I'd like to be able to locally mirror any gem server. In general, I'd also like to make it easier for people to run mirrors of gem servers. To accomplish this, I've written three things: a patch to rubygems/remote_installer.rb to allow the use of file:// URLs as sources; a script called gem_mirror to mirror from an arbitrary gem server to local disk; and a script called gem_server_dumb to run a brainless gem server serving a gem_mirrored directory.

As a reminder, installed gems live in a gem directory with subdirectories such as cache and specifications; I will refer to this as installed-layout. The gem remote installer, on the other hand, looks for a file called yaml (optionally first compressed as yaml.Z) and a subdirectory called gems; I will refer to this as server-layout. The standard gem_server script creates a web server which serves sites in server-layout (plus some other files which are not used by the remote installer) based on the contents of an installed-layout gem directory.

rubygems/remote_installer.rb file:// support patch

rubygems_remote_installer_file_uri.patch adds support for rubygems sources to be file:// URIs, either by an explicit --sources argument to gem or by inclusion in .gemrc. While it would seem more clean to make this a patch to open-uri, the fact that RemoteSourceFetcher#read_size doesn't go through the open-uri abstraction would have meant that that method would have still needed to be special-cased; so instead I simply put the file:// handling inside RemoteSourceFetcher. Note that the directory specified by the file:// URL should be in server-layout form.

I hope that the rubygems maintainers integrate this patch into the distribution.


The above patch raises the question of where a local server-layout directory would come from! Using gem_mirror, you can easily mirror gem servers (which can be specified as any URL that open-uri accepts or as paths, but ironically not as file:// URLs) to local directories. gem_mirror fetches the yaml file and checks to see which gems specified in it it does not have; it saves them to the specified directory, along with the yaml file itself. You can specify as many mirroring operations as you would like in a YAML file called .gemmirrorrc; mine looks like:

  to: /Users/glasser/MyGEMS/


Just to complete the loop, gem_server_dumb is a simple WEBrick web server which serves server-layout directories to the world. So if you run gem_mirror periodically and also run gem_server_dumb, you will be an http mirror of whatever server your are mirroring from, at least as far as the remote installer is concerned. (That is, you won't be mirroring RDoc or the top page of the gem server, and so on.)

This really doesn't do anything that you couldn't to by just pointing any other web server at your server-layout directory.