Detailed Documentation

Table of Contents

Setup

Requirements

Runtime Requirements:

  • None. There are no dependencies required to run GemInstaller, other than an installation of RubyGems itself

Build/Test requirements:

  • GemInstaller has a geminstaller.yml file to specify it’s own dependencies. After installing the geminstaller gem (via ‘gem install geminstaller’), change to the root of the geminstaller source tree, and run ‘geminstaller’ to auto-install the dependencies.

Installation

Installing Latest Release: Run 'gem install geminstaller' to install GemInstaller from RubyForge. Run it via 'sudo' if necessary, but first read the section on Dealing with sudo and root-owned RubyGem installations.

Installing Source Distribution: Run 'rake install_gem' from the root of the GemInstaller source tree. This assumes that Hoe and other required build/test dependencies are installed. Note this currently won’t work on windows because Hoe always tries to run sudo. It will generate a pkg/geminstaller-version.gem file, though, which you can install with the 'gem install <pkgfile>' command.

Command Line Options

Command Line Options Summary


$ geminstaller --help
Usage: geminstaller [options]

    -c, --config=CONFIGPATHS         Comma-delimited path(s) to GemInstaller config file(s).
    -e, --exceptions                 Raise any exceptions, rather than just printing them and exiting
                                     with a non-zero return code.
    -d, --redirect-stderr-to-stdout  Redirect all STDERR output to STDOUT.  Useful to get all output when
                                     invoking GemInstaller via system().
    -g, --geminstaller-output=TYPES  Comma-delimited list of output types to show from GemInstaller.
                                       Examples:
                                         --gall
                                         --geminstaller-output=error,install,commandecho
                                       Default: error,install,info
                                       Valid types:
                                         - none:        print only fatal errors
                                         - error:       print error messages
                                         - install:     print install messages
                                         - info:        print informational messages
                                         - commandecho: print rubygems commands as they are invoked
                                         - debug:       print debug messages
                                         - all:         print all messages
    -h, --help                       Show this message.
    -p, --print-rogue-gems           Print a report of all locally installed gems which are not specified
                                     in the geminstaller config file.
    -r, --rubygems-output=TYPES      Comma-delimited list of output types to show from internal:
                                       RubyGems command invocation.
                                       Examples:
                                         --rall
                                         --rubygems-output=stderr
                                       Default: stderr
                                       Valid types:
                                         - none:        print no output
                                         - stdout:      print standard output stream
                                         - stderr:      print standard error stream
                                         - all:         print all output
    -s, --sudo                       Perform all gem operations under sudo (as root).  Will only work on
                                     correctly configured, supported systems.  See docs for more info.
    -t, --silent                     Suppress all output except fatal exceptions, and output from
                                     rogue-gems option.
    -v, --version                    Show GemInstaller version and exit.

--config option

By default, GemInstaller will look for a config file named geminstaller.yml in the working directory. The '--config' option allows you to override this with a different file name or path. You can also have multiple comma-delimited paths to custom-named config files. This is useful if you have multiple projects, and want them to share a common config file, but still have project-specific overrides. You could also have a standard set of gems across multiple systems, with additional custom gems specified by individual projects. The last entries in the list will override the first ones. For example:


geminstaller --config=../common-config/geminstaller-common-across-projects.yml,geminstaller-custom-for-myproject.yml

--exceptions option

By default, GemInstaller will not throw exceptions for errors when invoked via GemInstaller.install or GemInstaller.autogem. This is so that any GemInstaller errors will not abort startup of Rails or any other app other apps. However, if you do want errors to abort startup of Rails or your app, you can set this option.

--redirect-stderr-to-stdout option

Currently, the GemInstaller --sudo option works by recursively re-invoking the geminstaller executable via system(). This option is automatically added in this case to ensure that errors that would normally go to stderr are printed to stdout. There are other solutions to this problem, but as of Ruby 1.8.5, there is no easy way to make a system call which will give a return code AND show both stderr and stdout synchronously. GemInstaller does not have and will not have any third-party runtime dependencies, so I needed to roll my own solution, and this was The Simplest Thing That Could Possibly Work.

--geminstaller-output option

The --geminstaller-output option controls what GemInstaller prints out about it’s internal activity. It is a comma-delimited list (no spaces), which can specify any of these output types:

  • none: Prints only fatal errors/exceptions which cause geminstaller to fail. The same as specifying none of the other types.
  • error: Prints error messages.
  • install: Prints messages indicating when gems are going to be installed. This may be gems which are directly specified in a GemInstaller config file, or dependency gems which are automatically installed to meet the requirments of a dependent gem.
  • info: Prints informational messages.
  • commandecho: Print rubygems commands as they are automatically invoked by GemInstaller.
  • debug: Prints debug messages
  • all: Prints all messages. This is the same as specifying all of the other types (except ‘none’)

This option works in conjuntion the --rubygems-output, which is used similarly, but controls output which is generated by RubyGems itself.

--rubygems-output option

GemInstaller works by automatically invoking RubyGems commands. The --rubygems-output option controls what types of output from RubyGems is printed out. It is a comma-delimited list (no spaces), which can specify any of these output types:

  • none: Prints no output from RubyGems.
  • stdout: Prints standard output stream from RubyGems.
  • stderr: Prints standard error stream from RubyGems.
  • all: Prints all messages. This is the same as specifying all of the other types (except ‘none’)

--bundler-export option

The --bundler-export writes a Bundler (http://github.com/carlhuda/bundler) Gemfile based on your Geminstaller config(s). However, be aware that you may have unnecessary entries in your config, because Bundler resolves the entire dependency tree for you. So, make sure you ONLY specify your application’s DIRECT dependencies in your Bundler Gemfile.

The --print-rogue-gems option prints a report of all locally installed gems which are not specified in the active loaded GemInstaller config file(s). GemInstaller, being “Narcissistic Software”™, considers these “rogue gems”. The output format is in YAML, so it can be cut and pasted or piped directly into a new or existing config file. This allows you to bootstrap a new system, by creating an empty config file with no gems specified, and running geminstaller with the --print-rogue-gems option to identify all gems which are currently installed on the system. It can also be used to identify gems which were manually installed without being entered into a GemInstaller config.

Some gems are automatically installed with RubyGems itself. The ‘sources’ gem always exists, and the Ruby Windows One Click Installer installs several windows-specific gems. The --print-rogue-gems option will include these gems, but will mark them with a comment indicating that they may have been automatically installed by RubyGems.

--sudo option

The --sudo option runs GemInstaller as root by using the sudo command. This command is neither valid nor necessary on Windows platforms. See the Dealing with sudo and root-owned RubyGem installations section for more details.

NOTE: As a security feature, the sudo command does NOT pass along the current environment. That means that if you use the --sudo option, environment variables that are currently set will NOT be available to GemInstaller config files. There are options to get around this – read the docs on sudoers for your unix distro.

--silent option

The --silent option suppresses all RubyGems output except fatal errors/exceptions. It overrides the --geminstaller-output and --rubygems-output options.

Unsuppressible Output

Sometimes, errors from RubyGems activity will still be printed even if the '--geminstaller-output=none' or --silent options are specified. This usually due to some failure when building gem docs, or on a windows platform. This is probably due to a gem directly printing to stderr or stdout. GemInstaller does not currently capture this type of output, only messages which are directly generated by RubyGems itself. This may be fixed in a later release.

Config File

The GemInstaller config file is simple YAML file. It contains a section specifying all the gems you want to install, and their associated options/properties, and another section which contains defaults to apply to all gems. By default, GemInstaller will look for a config file named geminstaller.yml in the working directory, but you can override this file name (and specify multiple config files) with the --config option.

GemInstaller offers a great deal of flexibility in configuration. By creatively using multiple config files, defaults within single config files, erb in config files, and the features of YAML itself, you should be able to come up with a unified GemInstaller config approach that works for multiple applications across multiple machines, platforms, and environments, with a minimum of duplication.

NOTE: Currently, the order of gems in the config file matters. It will install them, and any dependencies, in the order listed. This means if you have a gem listed explicitly, but it is a dependency of an earlier gem, it won’t use the options you set – it will use default/inherited options from it’s dependent gem. Perhaps a future version of GemInstaller will be smart enough to handle this better.

Config File Syntax Summary

This is an example config file template showing all the valid properties (boolean defaults capitalized):

---
defaults:
  install_options: [ any valid option for the RubyGems 'gem install' command ]
  check_for_upgrade: [ true | FALSE ]
  fix_dependencies: [ true | FALSE ]
  no_autogem: [ true | FALSE ]
  prefer_binary_platform:  [ TRUE | false ] (deprecated)
gems:
- name: [ gem name ]
  version: '[ any valid version specification for this gem, e.g. >= 1.0.0 ]'
  platform: [ any valid platform for this Gem and version specification ]
  install_options: [ any valid option for the RubyGems 'gem' command ]
  check_for_upgrade: [ true | FALSE ]
  fix_dependencies: [ true | FALSE ]
  no_autogem: [ true | FALSE ]
  prefer_binary_platform: [ TRUE | false ]
- name: [ another gem name ]
- name: [ yet another gem name ]
  (etc...)
<%= include_config("#{File.expand_path(File.dirname(__FILE__))}/path/to/included_snippet.yml") %>

All properties are optional, except for the gem name. If a default is specified, it will be used unless the gem specifically overrides it.

name config property

Required, no default. This is the name of the gem to install. GemInstaller always does an exact match on the name, a substring will not match.

version config property

Defaults to ‘> 0’. This is the version specification used to select a gem. The highest version which meets the specification will be installed. GemInstaller always does an exact match on the name, a substring will not match.

You should enclose the version specification in single quotes. This is not always necessary, but sometimes YAML will have trouble parsing it if you don’t, so it’s best to always use quotes.

There are many different options for version specifications. For more info, see the section on Specifying Versions in the RubyGems User Guide.

platform config property

Defaults to ‘ruby’. This is the platform for the gem. This must match a platform which is valid and available for the gem name and version specification. See the sections on Automatic Platform Detection and the prefer_binary_platform config property (deprecated) for more info on how platforms are handled.

install_options config property

Defaults to empty string. The install_options config property can contain any option which is valid to pass to the RubyGems 'gem install' command. These options will be passed on when GemInstaller internally invokes RubyGems. Type gem install --help to list the options.

check_for_upgrade config property

Defaults to false. By default, GemInstaller will not perform any remote operations unless they are necessary to install a required gem. In other words, if there is a local gem which meets the specs, nothing will be done. This allows GemInstaller to still run normally when you are not connected to a network.

However, if you are using a non-specific version spec (for example, ’>= 1.0.0), there COULD be a higher version available on the remote gem server. If set to true, check_for_upgrade will force GemInstaller to check for a higher-versioned gem on the server which still matches the version specification, and install it if one is found.

fix_dependencies config property

Defaults to false. Ruby Gems can depend on other gems, but the other gem might still get manually uninstalled (by using the gem uninstall --ignore-dependencies GEMNAME) option. The --fix_dependencies option will detect these missing dependencies, and automatically fix them. You shouldn’t need this unless you go around uninstalling gems with the --ignore-dependencies option. WARNING: This option can GREATLY increase the time it takes to run GemInstaller, especially if you have a lot of gems in your config, and especially on Windows. You probably always want it left as the default of false unless you suspect that some dependencies have been uninstalled.

no_autogem config property

Defaults to false. This option will prevent the autogem command from automatically requiring a gem. If you only want to require a few gems, you could set the default for this property to true, and only enable it (by setting a false value) for specific gems. For more details see Automatically Requiring Gems with the autogem Method.

prefer_binary_platform config property (deprecated)

DEPRECATED: This is deprecated when using RubyGems >= 0.9.5, which handles platform detection.

Defaults to true. GemInstaller automatically guesses at the correct platform for gem which have no platform specified. Any platform which is not ‘ruby’ is considered a ‘binary’ platform.

This option determines whether the ‘ruby’ platform, or a ‘binary’ platform will be chosen first. By default, binary platforms will be chosen if there is one available which seems to match the platform of the current machine. In practice, this usually only applies to windows, since (hopefully) most gems which need to be compiled distribute a windows-specific precompiled gem version. See Automatic Platform Detection for more info.

include_config directive

Inside a GemInstaller yaml config file, you can use the include_config directive to include another config file via ERB. This also works recursively – included files can include other files.

This is an alternative to specifying multiple files in the --config command line option – it allows you to abstract out common portions of config files across multiple files, while still only having to specify a single top-level config file.

GemInstaller also ensures that the working directory is set to the directory containing the current yaml file – NOT the directory that geminstaller was invoked from. This allows you to use include_config to reference yaml files relative to each other. Here’s an example:

---
# .../geminstaller.yml
gems:
<%= include_config("#{File.expand_path(File.dirname(__FILE__))}/subdir1/included_gem_config1.yml") %>
---
# .../subdir1/included_gem_config1.yml
  - name: testgem1
<%= include_config("#{File.expand_path(File.dirname(__FILE__))}/../subdir2/included_gem_config2.yml") %>
---
# .../subdir2/included_gem_config2.yml
  - name: testgem2

Using GemInstaller with Ruby on Rails or Other Ruby Apps

GemInstaller can also be embedded in another application, such as a Rails app. This is done by invoking the GemInstaller class or the geminstaller executable directly from Ruby. Also see the tutorial on Using GemInstaller with Ruby on Rails .

IMPORTANT NOTE: You need to know whether you run on unix and need root/sudo access to install gems and/or gem executables. See Dealing with sudo and root-owned RubyGem installations section for more details.

First, you need to create geminstaller.yml in the RAILS_ROOT/config directory (See also Bootstrapping your GemInstaller Config with the --print-rogue-gems option). Here’s an example config file which will install (an old version of) Rails and Mongrel:

---
defaults:
  install_options: --no-ri --no-rdoc
gems:
- name: rails
  version: '= 1.1.6'
- name: mongrel
  version: '= 1.0.1'
  platform: <%= RUBY_PLATFORM =~ /mswin/ ? 'mswin32' : 'ruby'%>

Invoking GemInstaller from Rails

  • Install the GemInstaller gem and create your config/geminstaller.yml config file.
  • Determine whether you run on unix and need root/sudo access to install gems. If you do, edit your sudoers file to allow the current user to run the 'gem' command via sudo without a password. See the documentation on the GemInstaller --sudo option for more details.
  • Create a config/preinitializer.rb file to invoke GemInstaller on your Rails app startup:

RAILS_ROOT/config/preinitializer.rb:

  require 'rubygems'
  require 'geminstaller'
  # You can use the geminstaller_rails_preinitializer out of the GemInstaller gem:
  require 'geminstaller_rails_preinitializer'
  # If you require different geminstaller command options, copy the geminstaller_rails_preinitializer.rb
  # file into from the GemInstaller source into your Rails app, modify it, and require your customized 
  # version.  For example:
  # require "#{File.expand_path(RAILS_ROOT)}/config/custom_geminstaller_rails_preinitializer.rb"

  • Start your app: ruby script/server. You should see (depending on verbosity level) messages indicating the gems (and dependency gems) are being installed and auto-required.
  • Stop the app, and verify the gems are installed: gem list [some gem in your geminstaller.yml]
  • For an example, check out the Sample Rails App in the GemInstaller source.

The default geminstaller_rails_preinitializer.rb in the GemInstaller source contains several comments which are hopefully self-explanatory. See the Command Line Options Summary, especially the --config, --exceptions, and --sudo options for more details, as well as the Integrating GemInstaller into Ruby on Rails tutorial.

DEPRECATED – Invoking GemInstaller from Rails 1.0

This section describes how to use GemInstaller with Rails 1.0 (which doesn’t support config/preinitializer.rb). If you are using Rails 2.0 or above, use these instructions

Once you have your geminstaller.yml created, invoke GemInstaller on Rails 1.0 startup in your boot.rb. It should be placed right after the block which defines the RAILS_ROOT constant, as shown below (“…” indicates omitted lines):

RAILS_1.0_ROOT/config/boot.rb:

  ...
  unless defined?(RAILS_ROOT)
  ...
  end
 
  ############# Begin GemInstaller config - see http://geminstaller.rubyforge.org
  require "rubygems"
  require "geminstaller"
 
  # Path(s) to your GemInstaller config file(s)
  config_paths = "#{File.expand_path(RAILS_ROOT)}/config/geminstaller.yml"
 
  # Arguments which will be passed to GemInstaller (you can add any extra ones)
  args = "--config #{config_paths}" 
 
  # The 'exceptions' flag determines whether errors encountered while running GemInstaller
  # should raise exceptions (and abort Rails), or just return a nonzero return code
  args += " --exceptions"
 
  # This will use sudo by default on all non-windows platforms, but requires an entry in your
  # sudoers file to avoid having to type a password.  It can be omitted if you don't want to use sudo.
  # See http://geminstaller.rubyforge.org/documentation/documentation.html#dealing_with_sudo
  args += " --sudo" unless RUBY_PLATFORM =~ /mswin/ 
 
  # The 'install' method will auto-install gems as specified by the args and config
  GemInstaller.install(args)
 
  # The 'autogem' method will automatically add all gems in the GemInstaller config to your load path, using the 'gem'
  # or 'require_gem' command.  Note that only the *first* version of any given gem will be loaded.
  GemInstaller.autogem(args)
  ############# End GemInstaller config
 
  unless defined?(Rails::Initializer)
  ...

Bootstrapping Rails with GemInstaller

If you don’t already have Rails or Mongrel installed (you’ve just checked your app out or deployed onto a new machine), you obviously can’t use the above methods to run GemInstaller. In this case, you just need to run GemInstaller from the command line once to “bootstrap” and install the Rails/Mongrel gems. This assumes you already have your geminstaller.yml config file in the Rails config dir:


$ cd [RAILS_APP_ROOT]/config
$ geminstaller --sudo # only use the -sudo option if necessary

Using GemInstaller from Other Ruby Apps

If you want to automatically invoke GemInstaller from some other Ruby application, the approach will be very similar to the Rails approach shown above. Instead of preinitializer.rb, put the GemInstaller invocation into the class or script which is the very first one run when your application starts up. To use it in build scripts such as rake, include it in a task. GemInstaller (obviously) must run before any gems are loaded or referenced.

Using erb in config files

The GemInstaller config file(s) are also run through erb, so you can embed any custom ruby code to dynamically generate portions of your config. This can be used to have the same config file select gems differently on different platforms or environments. Below is an example of using erb to detect and choose the platform. This is necessary with some gems, even though GemInstaller attempts to guess at the platform. See Automatic Platform Detection for more info.

---
# geminstaller-detect-platform.yml
gems:
- name: x10-cm17a
  version: '> 1.0.0'
  platform: <%= RUBY_PLATFORM =~ /mswin/ ? 'i386-mswin32' : 'ruby'%>

Automatic Platform Detection

Up through version 0.9.4, RubyGems did not provide any way to specify a platform programatically or via the command line, and prompted via stdin if a specified version of a gem has any binary or platform-specific releases. RubyGems 0.9.5 and later should correctly determine the correct platform to install, and GemInstaller will automatically use this feature.

However, if you must run older versions of RubyGems <= 0.9.4, GemInstaller still includes support for automatically selecting the platform without any stdin prompts. If a multi-platform gem version is found, GemInstaller will handle the list prompt, and automatically select the correct platform.

This means that you don’t have to worry about what version of RubyGems your app will run on – GemInstaller handles automatic platform selection on all RubyGems versions.

If you do want explicit control over the platform which is used, all you need to do is specify the 'platform' property for any multiplatform gems can be specified in the geminstaller config file. If you know what platform you want for a specific gem, and you only run GemInstaller on one platform, you can hardcode this value. Even if you run GemInstaller on multiple platforms (and therefore can’t hardcode the platform string), you can use erb to programatically pick the correct platform. See Using erb in config files for an example.

See the section on the prefer_binary_platform config property (deprecated) for more info.

Dealing with sudo and root-owned RubyGem installations

If you only run geminstaller on Windows, you don’t have to worry about this section :)

UPDATE:

Probably the best way to “deal with sudo” is to never use it at all, and recent releases of RubyGems will automatically install gems to ~/.gem if you run gem install commands as a non-root user. However, the RubyGems developers (Eric Hodel) have indicated that this feature may not remain, since it causes issues under Ruby 1.9. I have a feature to track the progress of this issue.

FOR NOW, THOUGH, HERE’S HOW TO USE SUDO WITH GEMINSTALLER:

Important Note: For security, the sudo command does NOT pass most environment variables. See the faq item on Environment Variables and sudo for more info.

On many unix-like systems (Mac, Linux, etc.), the root user will own the local installation of RubyGems (often /usr/local/ruby) and/or the executable directory where gem executables are installed (often /usr/local/bin). If this is the case, then gems must be installed and uninstalled by the root user, or via the sudo command. This presents a problem for geminstaller, which must have this same permission to automatically install gems.

There are several different solutions this problem. The solutions that are available are also determined by the way you use geminstaller (whether you call it from the executable, or use the GemInstaller classes directly from Ruby).

A transparent solution to this problem is planned for a future release of GemInstaller. For now, however, you will need to pick one of the following approaches below. If you use sudo, you should also read the docs on configuring sudo.

Option 1 – Use the -s or --sudo option on the geminstaller executable

Examples:


$ geminstaller -s
$ geminstaller --sudo

Option 2 – Run sudo or log in as root yourself. If you are running from a deploy script (Capistrano), this is probably the best option.

Example of using sudo:


  $ sudo geminstaller

Examples of running geminstaller as root


  $ su -      # if you can log in as root on your system, OR
  $ sudo -s   # if you must use sudo to log in as root on your system
  # geminstaller

Option 3 – Make everything owned by the local user that runs geminstaller

(replace <local user> with your username)


  $ cd /usr/local/lib/ruby # or wherever you have ruby installed
  $ sudo chown -R <local user> .

Tips on configuring sudo

Sudo can be configured to not ask for a password for certain commands. This will be useful if you want to run geminstaller against a root-owned gem repository without being prompted (such as a Rails app being deployed via Capistrano).

You should consult the man or info pages on sudoers and visudo for more info (man sudoers or info sudoers). Make sure you understand the security implications of this approach.

Here’s an example of how sudoers might be configured to allow the local user to run the 'gem', 'ruby', and 'geminstaller' commands via sudo without being prompted for a password. Replace <local user> with your username, and replace ‘/usr/local/bin/’ with the appropriate path if it is different on your system:


  $ sudo visudo
  # ADD this line (Do NOT delete any existing lines, or you may 
  # lose your sudo access and ability to administer your system!!!):
  <local user> ALL = NOPASSWD: /usr/local/bin/geminstaller, /usr/local/bin/ruby, /usr/local/bin/gem

Automatically Requiring Gems with the autogem Method

GemInstaller provides a GemInstaller.autogem class method which will automatically invoke RubyGems’ ‘gem’ method to add all of the gems in your GemInstaller config file(s) to the load path. On RubyGems versions 0.8.x, the ‘require_gem’ method is used instead.

If there are some gems you don’t want to auto-require, you can set the ’no_autogem: true config config property in the GemInstaller config file. This can be set for individual gems, or as a default.

To use it, invoke the autogem method, and pass a string of arguments, just like the GemInstaller command line, but it ignores arguments which are not applicable (such as -r or -s):


  ...
  GemInstaller.autogem('--config=/path/to/one/geminstaller.yml,/path/to/another/geminstaller.yml')
  ...

Alternately, you are not required to pass any args if you invoke the command from a ruby file which is in the same directory as a default-named 'geminstaller.yml' file.

You can also use the “—exceptions” argument to cause autogem to throw an exception if it encounters any error. This is useful if you want to abort your application startup if any gems failed to load.

IMPORTANT NOTE: The RubyGems 'gem' and 'require_gem' commands will only load a single version of any given gem onto the load path. Because of this, the GemInstaller autogem command will only process the first version that it finds of any given gem.

For more info on the RubyGems ‘gem and ’require_gem’ commands, see Using Explicit Versions in the RubyGems docs.