Increasing Memory Allocation  Print-icon

GenePattern allocates memory to the server, to the "client" (the computer you are using to access GenePattern), and to individual modules. When a module fails with an out of memory error, you can try increasing the amount of memory allocated to the server, the client, or the module.

Increasing Memory for Modules

To increase the amount of memory allocated to a module written in Java or R, click Administration>Server Settings. The Programming Languages page (Programming Language Options) provides several options for increasing Java and R memory options.

You can customize memory preferences on a per-module basis. This is useful when some of your modules require more memory than others. If you haven't already done so, copy the 'config_default.yaml' file as 'config_custom.yaml'. Then set 'config.file=config_custom.yaml' in the genepattern.properties file. These files are in the resources directory of your installation. You need to restart your server for this to take effect. This ensures that your custom configuration will not be inadvertently modified when you install an updated version of GenePattern.

Set custom memory settings in the config_custom.yaml file. This is a text file in YAML format. The 'job.memory' property defines the memory requirements for your module, for example job.memory: 512m, job.memory: 2g. This defines the '-Xmx' flag passed to the java command line. It also defines the memory requirements passed along to the queuing system such as LSF or SGE. In the rare case when you want to use a different java memory flag from the queuing system flag, set both 'job.memory' and 'job.javaXmx'.

You can also customize error handling for completed jobs. When the 'job.error_status.stderr' flag is set to 'true' the server will interpret a non-empty stderr stream as a failed job. When the 'job.error_status.exit_value' flag is set to 'true' the server will interpret a non-zero exit code as a failed job. 

# example config_yaml entry
module.properties:
    # custom memory flags for the ConvertLineEndings module
    ConvertLineEndings:
        # single parameter sets both the java flag and the queuing system memory requirements
        job.memory: 2Gb

    #
    # advanced flags
    MyModule:
        # it is possible to pass one value to the java command line, e.g. java -Xmx1g
        job.javaXmx: 1g
        # and a different value to the queuing system
        job.memory: 2g
        #
        # ignore stderr output
        job.error_status.stderr: false
        # don't ignore the exit code
        job.error_status.exit_value: true
 
A module author can define job preferences in the manifest file for the module. The config_yaml file takes precedence over declarations in the module manifest. The parameters are the same, but don't forget to use the '=' sign instead of the ':' sign.
job.memory=2g
job.error_status.stderr=false
job.error_status.exit_value=true

Increasing Memory for Visualizers

Many GenePattern modules are run on the server. However, visualizers are applications that run on your computer, rather than on the GenePattern server. This means that you must have Java installed on your computer. For easy debugging, set your Java preferences so that the Java console displays.

The default visualizer memory limit is 512 MB. However, if you find that your visualizer repeatedly runs out of memory, you can try a few things to eliminate that error:

Increasing Memory for the Server and/or Client

To increase the amount of memory allocated to the server and/or the client, follow the instructions for your platform:

Mac OS X

  1. Right-click on the file GenePattern/Tomcat/StartGenePatternServer (server) or the GenePatternClient/GenePattern Client (client).
  2. Select Show Package Contents from the pop-up menu. The Contents directory should open in the finder.
  3. In the Contents directory, double-click the Info.plist file. This should open the Property List Editor program.
  4. Add the child VMOptions under the Java node.
  5. Change the Class of the added VMOptions node to ‘Array’.
  6. Add the child with Class 'String' with the value -Xmx512M. You can replace the value 512 with the maximum amount of memory in MB that you want the GenePattern Client to use.

Windows and Linux

  1. Edit the configuration file GenePatternServer/StartGenePatternServer.lax (server) or GenePatternClient/GenePattern Client.lax (client).
  2. In either file, look for the entries noted below and increase these values (for example, double the value) up to the maximum memory size of the machine you are using. (Note: Windows limits the total space available to a process to 2 GB. Some of that is used for overhead, so slightly less is really available to the JRE.)
    • lax.nl.java.option.java.heap.size.initial
    • lax.nl.java.option.java.heap.size.max

<< Using the R Installer Plug-in Up Using a Queuing System >>

Updated on April 17, 2014 23:29