Skip FOLIO Project Navigation

GitHub and Twitter:

ModuleDescriptor

Introduction

This guide provides an overview of the ModuleDescriptor.

The ModuleDescriptor (MD) is introduced in the Okapi Guide (e.g. here and here).

The MD adheres to the ModuleDescriptor.json schema.

Maintenance

Back-end modules

For back-end modules, a template is maintained. This is usually at ./descriptors/ModuleDescriptor-template.json and is transformed by the module’s build process into the target/ModuleDescriptor.json file. See example at mod-courses.

To validate changes, utilise a JSON Schema validator such as ‘z-schema’ or ‘ajv’ (see detailed notes futher below):

cd mod-courses
jq '.' descriptors/ModuleDescriptor-template.json
z-schema --pedanticCheck \
  ../okapi/okapi-core/src/main/raml/ModuleDescriptor.json \
  ./descriptors/ModuleDescriptor-template.json

Front-end modules

For front-end UI modules, the ModuleDescriptor.json is generated from various information that is maintained in the package.json file. See example at ui-users.

To validate changes, first use stripes-cli to generate the MD. Then utilise a JSON Schema validator such as ‘z-schema’ or ‘ajv’ (see detailed notes futher below):

cd ui-users
jq '.' package.json
stripes mod descriptor --full --output /tmp
z-schema --pedanticCheck \
  ../okapi/okapi-core/src/main/raml/ModuleDescriptor.json \
  /tmp/users.json

Registry of ModuleDescriptors

As part of the continuous integration process, each ModuleDescriptor.json is published to the FOLIO Registry at https://folio-registry.dev.folio.org/

Refer to explanation of how to conduct registry queries to determine modules which require and provide certain interfaces.

ModuleDescriptor properties

Introduction

Refer to the general introduction above.

General MD properties

Each main property is briefly described in the ModuleDescriptor.json schema.

The following sub-sections explain some properties in more detail …

TODO: Provide brief explanations of some sections.

metadata

The “metadata” section enables some additional descriptive items. The MD schema enables any JSON object.

LaunchDescriptor properties

Introduction

The LaunchDescriptor (LD) is introduced in the Okapi Guide (e.g. at sections Deployment and Discovery and Deployment and Auto-deployment).

The LD adheres to the LaunchDescriptor.json schema.

As explained in the Okapi Guide, the LaunchDescriptor can be a separate descriptor, or be part of the ModuleDescriptor. The LaunchDescriptor can utilise various methods for deployment.

Default LD Docker properties

For the suite of back-end modules that are hosted in the folio-org GitHub organization, each one has a LaunchDescriptor for Docker, and the LD is included in the module’s ModuleDescriptor file.

For a back-end module to be included in the reference environments, it must have such a LaunchDescriptor.

This enables ready default deployment. Each module’s LD settings are used directly in the FOLIO reference environments. Note that those installations have a small amount of data and low activity load. Other installations would probably adjust or replace these LDs.

The "env" section of the "launchDescriptor" properties contains an enumeration of variables (such as DB_USERNAME and JAVA_OPTIONS) and sample values. It is considered best practice to override these values in the Launch Descriptors created for production environments.

General LD properties

Each main property is briefly described in the LaunchDescriptor.json schema.

The following sub-sections explain some properties in more detail.

Refer to the example below.

dockerCMD

Optional. Over-ride the CMD of Dockerfile. An array of string items.

Example LDs that use this: mod-login

Memory

Limit each container’s memory usage.

All LDs must have the memory setting.

The setting must be expressed as bytes.

Take care to have the correct number of digits, typically nine.

As noted above, this memory level is appropriate for running a basic system such as the “folio-snapshot” reference environment and the Vagrant VMs. So this default container memory setting should be as low as possible.

NOTE: 20191118: Module developers: please determine the lowest possible container memory allocation, and adjust your setting (see FOLIO-2315).

With the new directions for LaunchDescriptors and Dockerfiles (see FOLIO-2358), and as shown in the example below, two-thirds of the memory allocation will be reserved for heap space.

HostPort binding

Okapi will map the “%p” value to the relevant port for this container.

Ensure that port numbers are matching – the same in ModuleDescriptor, and in Dockerfile, and in program code and configuration.

env

The default environment for deployment.

This is an array of items, each with properties: name (required), value, description.

All LDs for Java-based modules must have the base JAVA_OPTIONS setting.

If the module uses a database, then it must provide the standard set of DB_ database settings shown in the example.

Other environment variables can also be documented here (see examples). Their default values would need to make sense in the FOLIO reference environments where these LaunchDescriptors will be used as-is. The defaults also need to make sense in a cluster.

Clearly describe the env items using those available properties (name, value, description). This is necessary information for DevOps to be able to utilise the module.

env JAVA_OPTIONS

This environment variable must at least have the setting as shown in the example, which enables Java to configure the specified memory for the container.

The module’s Dockerfile needs to use a base image that has the feature “UseContainerSupport”. Use that feature in conjunction with “MaxRAMPercentage”.

Note that all Java-based modules should now be using Java 21 or Java 17.

Other necessary options can be appended (e.g. -XX:+PrintFlagsFinal).

See one explanation of Java memory footprint, monitoring, and profiling.

Also note that “setting -Xmx and -Xms disables the automatic heap sizing”.

env DB environment

If the module uses a database, then it must provide the standard set of DB_ settings shown in the example.

Some need explanation:

  • DB_HOST keyword “postgres” is automatically mapped by the relevant system.
  • DB_DATABASE is “okapi_modules” used by all modules.

Docker Hub README

As part of the CI stage “Docker Build”, various information will be gleaned from the LaunchDescriptor to generate a README and deploy to Docker Hub.

As explained above, clearly describe the “env” items.

Example LaunchDescriptors

The following example is for mod-courses which does use a database.

As explained above, this form needs the new folio docker image.

  "launchDescriptor": {
    "dockerImage": "${artifactId}:${version}",
    "dockerPull": false,
    "dockerArgs": {
      "HostConfig": {
        "Memory": 357913941,
        "PortBindings": { "8081/tcp": [ { "HostPort": "%p" } ] }
      }
    },
    "env": [
      { "name": "JAVA_OPTIONS",
        "value": "-XX:MaxRAMPercentage=66.0"
      },
      { "name": "DB_HOST", "value": "postgres" },
      { "name": "DB_PORT", "value": "5432" },
      { "name": "DB_USERNAME", "value": "folio_admin" },
      { "name": "DB_PASSWORD", "value": "folio_admin" },
      { "name": "DB_DATABASE", "value": "okapi_modules" },
      { "name": "DB_QUERYTIMEOUT", "value": "60000" },
      { "name": "DB_CHARSET", "value": "UTF-8" },
      { "name": "DB_MAXPOOLSIZE", "value": "5" }
    ]
  }

Other LaunchDescriptor examples:

Testing the modifications

Use jq:

jq '.' descriptors/ModuleDescriptor-template.json

Use z-schema:

yarn global add z-schema
cd mod-courses
z-schema --pedanticCheck \
  ../okapi/okapi-core/src/main/raml/ModuleDescriptor.json \
  descriptors/ModuleDescriptor-template.json

Or use ajv:

yarn global add ajv-cli
cd mod-courses
ajv validate \
  -s ../okapi/okapi-core/src/main/raml/ModuleDescriptor.json \
  -r "../okapi/okapi-core/src/main/raml/*.json" \
  -d descriptors/ModuleDescriptor-template.json

Deploy the module as a Docker container with a local FOLIO Vagrant VM (e.g. folio/testing). See instructions.

Inspect the container, and gather some crude statistics:

vagrant ssh
sudo docker ps  # and find your container ID
sudo docker stats [CONTAINER...]
sudo docker exec -it CONTAINER bash
pmap -x 1

See other monitoring notes above.