Welcome to the documentation of pbrx¶

Contents:

pbrx¶

Utilities for projects using pbr.

pbr is very opinionated about how things should be done. As a result, there are a set of actions that become easy to deal with generically for any pbr based project. pbrx is a collection of utilities that contain support for such actions.

Note

Each of the utilities has a primary focus of working for projects using pbr. However, some of them will also work just fine for non-pbr-based projects. When that is the case, the utility will be marked appropriately.

Free software: Apache license
Documentation: https://docs.openstack.org/pbrx/latest
Source: https://git.openstack.org/cgit/openstack/pbrx

Features¶

Each utility is implemented as a subcommand on the pbrx command.

install-siblings: Updates an installation with local from-source versions of dependencies. For any dependency that the normal installation installed from pip/PyPI, install-siblings will look for an adjacent git repository that provides the same package. If one exists, the source version will be installed to replace the released version. This is done in such a way that any given constraints will be honored and not get messed up by transitive depends.
build-images: Builds container images from a project’s source tree. The python:alpine base image is used, and dependencies are taken from bindep.txt for distro requirements and requirements.txt for python requirements. A base image is made for the project itself, and then an additional image based on the base image for every entry in entry_points.console_scripts in the setup.cfg file.

pbrx service installation guide¶

pbrx service overview¶

The pbrx service provides…

The pbrx service consists of the following components:

pbrx-api service: Accepts and responds to end user compute API calls…

Install and configure¶

This section describes how to install and configure the pbrx service, code-named pbrx, on the controller node.

This section assumes that you already have a working OpenStack environment with at least the following components installed: .. (add the appropriate services here and further notes)

Note that installation and configuration vary by distribution.

Install and configure for openSUSE and SUSE Linux Enterprise¶

This section describes how to install and configure the pbrx service for openSUSE Leap 42.1 and SUSE Linux Enterprise Server 12 SP1.

Prerequisites¶

Before you install and configure the pbrx service, you must create a database, service credentials, and API endpoints.

To create the database, complete these steps:
- Use the database access client to connect to the database server as the root user:
```
$ mysql -u root -p
```
- Create the pbrx database:
```
CREATE DATABASE pbrx;
```
- Grant proper access to the pbrx database:
```
GRANT ALL PRIVILEGES ON pbrx.* TO 'pbrx'@'localhost' \
  IDENTIFIED BY 'PBRX_DBPASS';
GRANT ALL PRIVILEGES ON pbrx.* TO 'pbrx'@'%' \
  IDENTIFIED BY 'PBRX_DBPASS';
```
  Replace PBRX_DBPASS with a suitable password.
- Exit the database access client.
```
exit;
```
Source the admin credentials to gain access to admin-only CLI commands:
```
$ . admin-openrc
```

To create the service credentials, complete these steps:

Create the pbrx user:

$ openstack user create --domain default --password-prompt pbrx

Add the admin role to the pbrx user:

$ openstack role add --project service --user pbrx admin

Create the pbrx service entities:

$ openstack service create --name pbrx --description "pbrx" pbrx

Create the pbrx service API endpoints:

$ openstack endpoint create --region RegionOne \
  pbrx public http://controller:XXXX/vY/%\(tenant_id\)s
$ openstack endpoint create --region RegionOne \
  pbrx internal http://controller:XXXX/vY/%\(tenant_id\)s
$ openstack endpoint create --region RegionOne \
  pbrx admin http://controller:XXXX/vY/%\(tenant_id\)s

Install and configure components¶

Install the packages:

# zypper --quiet --non-interactive install

Edit the /etc/pbrx/pbrx.conf file and complete the following actions:
- In the [database] section, configure database access:
```
[database]
...
connection = mysql+pymysql://pbrx:PBRX_DBPASS@controller/pbrx
```

Finalize installation¶

Start the pbrx services and configure them to start when the system boots:

# systemctl enable openstack-pbrx-api.service

# systemctl start openstack-pbrx-api.service

Install and configure for Red Hat Enterprise Linux and CentOS¶

This section describes how to install and configure the pbrx service for Red Hat Enterprise Linux 7 and CentOS 7.

Prerequisites¶

Before you install and configure the pbrx service, you must create a database, service credentials, and API endpoints.

To create the database, complete these steps:
- Use the database access client to connect to the database server as the root user:
```
$ mysql -u root -p
```
- Create the pbrx database:
```
CREATE DATABASE pbrx;
```
- Grant proper access to the pbrx database:
```
GRANT ALL PRIVILEGES ON pbrx.* TO 'pbrx'@'localhost' \
  IDENTIFIED BY 'PBRX_DBPASS';
GRANT ALL PRIVILEGES ON pbrx.* TO 'pbrx'@'%' \
  IDENTIFIED BY 'PBRX_DBPASS';
```
  Replace PBRX_DBPASS with a suitable password.
- Exit the database access client.
```
exit;
```
Source the admin credentials to gain access to admin-only CLI commands:
```
$ . admin-openrc
```

To create the service credentials, complete these steps:

Create the pbrx user:

$ openstack user create --domain default --password-prompt pbrx

Add the admin role to the pbrx user:

$ openstack role add --project service --user pbrx admin

Create the pbrx service entities:

$ openstack service create --name pbrx --description "pbrx" pbrx

Create the pbrx service API endpoints:

$ openstack endpoint create --region RegionOne \
  pbrx public http://controller:XXXX/vY/%\(tenant_id\)s
$ openstack endpoint create --region RegionOne \
  pbrx internal http://controller:XXXX/vY/%\(tenant_id\)s
$ openstack endpoint create --region RegionOne \
  pbrx admin http://controller:XXXX/vY/%\(tenant_id\)s

Install and configure components¶

Install the packages:
```
# yum install
```

Edit the /etc/pbrx/pbrx.conf file and complete the following actions:
- In the [database] section, configure database access:
```
[database]
...
connection = mysql+pymysql://pbrx:PBRX_DBPASS@controller/pbrx
```

Finalize installation¶

Start the pbrx services and configure them to start when the system boots:

# systemctl enable openstack-pbrx-api.service

# systemctl start openstack-pbrx-api.service

Install and configure for Ubuntu¶

This section describes how to install and configure the pbrx service for Ubuntu 14.04 (LTS).

Prerequisites¶

Before you install and configure the pbrx service, you must create a database, service credentials, and API endpoints.

To create the database, complete these steps:
- Use the database access client to connect to the database server as the root user:
```
$ mysql -u root -p
```
- Create the pbrx database:
```
CREATE DATABASE pbrx;
```
- Grant proper access to the pbrx database:
```
GRANT ALL PRIVILEGES ON pbrx.* TO 'pbrx'@'localhost' \
  IDENTIFIED BY 'PBRX_DBPASS';
GRANT ALL PRIVILEGES ON pbrx.* TO 'pbrx'@'%' \
  IDENTIFIED BY 'PBRX_DBPASS';
```
  Replace PBRX_DBPASS with a suitable password.
- Exit the database access client.
```
exit;
```
Source the admin credentials to gain access to admin-only CLI commands:
```
$ . admin-openrc
```

To create the service credentials, complete these steps:

Create the pbrx user:

$ openstack user create --domain default --password-prompt pbrx

Add the admin role to the pbrx user:

$ openstack role add --project service --user pbrx admin

Create the pbrx service entities:

$ openstack service create --name pbrx --description "pbrx" pbrx

Create the pbrx service API endpoints:

$ openstack endpoint create --region RegionOne \
  pbrx public http://controller:XXXX/vY/%\(tenant_id\)s
$ openstack endpoint create --region RegionOne \
  pbrx internal http://controller:XXXX/vY/%\(tenant_id\)s
$ openstack endpoint create --region RegionOne \
  pbrx admin http://controller:XXXX/vY/%\(tenant_id\)s

Install and configure components¶

Install the packages:
```
# apt-get update

# apt-get install
```

Edit the /etc/pbrx/pbrx.conf file and complete the following actions:
- In the [database] section, configure database access:
```
[database]
...
connection = mysql+pymysql://pbrx:PBRX_DBPASS@controller/pbrx
```

Finalize installation¶

Restart the pbrx services:

# service openstack-pbrx-api restart

Verify operation¶

Verify operation of the pbrx service.

Note

Perform these commands on the controller node.

Source the admin project credentials to gain access to admin-only CLI commands:
```
$ . admin-openrc
```
List service components to verify successful launch and registration of each process:
```
$ openstack pbrx service list
```

Next steps¶

Your OpenStack environment now includes the pbrx service.

To add additional services, see https://docs.openstack.org/project-install-guide/ocata/.

The pbrx service (pbrx) provides…

This chapter assumes a working setup of OpenStack following the OpenStack Installation Tutorial.

Contributor Documentation¶

Contributing¶

If you would like to contribute to the development of OpenStack, you must follow the steps in this page:

http://docs.openstack.org/infra/manual/developers.html

If you already have a good understanding of how the system works and your OpenStack accounts are set up, you can skip to the development workflow section of this documentation to learn how changes to OpenStack should be submitted for review via the Gerrit tool:

http://docs.openstack.org/infra/manual/developers.html#development-workflow

Pull requests submitted through GitHub will be ignored.

Bugs should be filed on Storyboard:

https://storyboard.openstack.org

Configuration¶

Configuration of pbrx.

Command line interface reference¶

CLI reference of pbrx.

Users guide¶

Users guide of pbrx.

Installation of Sibling Packages¶

There are times, both in automated testing, and in local development, where one wants to install versions of a project from git that are referenced in a requirements file, or that have somehow already been installed into a given environment.

This can become quite complicated if a constraints file is involved, as the git versions don’t match the versions in the constraints file. But if a constraints file is in play, it should also be used for the installation of the git versions of the additional projects so that their transitive depends may be properly constrained.

To help with this, pbrx provides the install-siblings command. It takes a list of paths to git repos to attempt to install, as well as an optional constraints file.

It will only install a git repositoriy if there is already a corresponding version of the package installed. This way it is safe to have other repos wind up in the package list, such as if a Zuul job had a Depends-On including one or more additional packages that were being put in place for other purposes.

pbrx siblings expects to be run in root source dir of the primary project. Sibling projects may be given as relative or absolute paths.

For example, assume the following directory structure:

$ tree -ld -L 3
├── git.openstack.org
│   ├── openstack
│   │   ├── keystoneauth
│   │   ├── python-openstackclient
│   │   ├── python-openstacksdk
│   │   ├── requirements

The user is in the git.openstack.org/openstack/python-openstackclient and has installed the code into a virtualenv called venv. python-openstackclient has the following requirements:

keystoneauth1>=3.3.0 # Apache-2.0
openstacksdk>=0.9.19 # Apache-2.0

And in the git.openstack.org/openstack/requirements directory is a file called upper-constraints.txt which contains:

keystoneauth1===3.4.0
openstacksdk===0.11.3
requests===2.18.4

The command:

$ venv/bin/pbrx install-siblings ../keystoneauth

would result in an installation of the contents of ../keystoneauth, since keystoneauth1 is already installed and the package name in the git.openstack.org/openstack/keystoneauth directory is keystoneauth1. No constraints are given, so any transitive dependencies that are in git.openstack.org/openstack/keystoneauth will be potentially installed unconstrained.

$ venv/bin/pbrx install-siblings -c ../requirements/upper-constraints.txt ../keystoneauth

Will also update keystoneauth1, but will apply constraints properly to any transitive depends.

$ venv/bin/pbrx install-siblings -c ../requirements/upper-constraints.txt ../keystoneauth ../python-openstacksdk

will install both keystoneauth1 and openstacksdk.

$ venv/bin/pbrx install-siblings -c ../requirements/upper-constraints.txt ../keystoneauth ../python-openstacksdk ../requirements

will also install both keystoneauth1 and openstacksdk. Even though git.openstack.org/openstack/requirements is itself a python package, since it is not one of the python-openstackclient dependencies, it will be skipped.

Building Container Images¶

Python projects that declare their distro dependencies using bindep can be built into container images without any additional duplicate configuration. The pbrx command build-images does this as minimally and efficiently as possible. The aim is to produce single-process application images that container only those things needed at runtime.

When pbrx build-images is run in a project source directory, the result will be a base image, named ‘{project}-base’, and then an image for each entry in entry_points.console_scripts with CMD set to that console script. For instance, in a python project “foo” that provides console scripts called “foo-manage” and “foo-scheduler”, pbrx build-images will result in container images called “foo-base”, “foo-manage” and “foo-scheduler”.

pbrx build-images uses volume mounts during the image build process instead of copying to prevent wasted energy in getting source code into the image and in getting artifacts out of the image. This makes it well suited for use on laptops or in automation that has access to something that behaves like a full computer but at the moment less well suited for use in unprivileged container systems. Work will be undertaken to remove this limitation.

Distro Depends¶

build-images relies on bindep and bindep.txt to get the list of packages to install.

build-images uses the Builder Image pattern so that one image is used to make wheels of the project and its dependencies, and another to install the package. Distro packages needed to build wheels of a project or its python depends from source should be marked with a compile profile in bindep.txt. Distro packages needed at runtime should not be marked with a profile.

build-images uses python:alpine as a base image. There are no plans or intent to make that configurable since these are application images and the guest distro only serves to provide Python and c-library depends. To mark dependencies in bindep.txt for images, the platform:apline profile can be used.

The following is an example bindep file:

gcc [compile test platform:rpm platform:apk]
libffi-devel [compile test platform:rpm]
libffi-dev [compile test platform:dpkg platform:apk]
libffi [platform:apk]
libressl-dev [compile test platform:apk]
linux-headers [compile test platform:apk]
make [compile test platform:apk]
musl-dev [compile test platform:apk]

The only library needed at runtime is libffi. The other dependencies are all marked compile so will be installed into the build container but not the final runtime container. bindep is useful not just for building containers, so entries for libffi-dev on debian as well as libffi-devel on Red Hat are there. Also, this example marks some packages as needed for test. pbrx and bindep appropriately ignore this information.

Note

Because of the use of the python:alpine image, it is not necessary to list python3-dev in platform:alpine.

Python Dependencies¶

build-images uses normal python mechanisms to get python dependencies. Namely, it runs pip install . in the mounted source directory.

In most cases this is sufficient, but there are times when a single set of dependencies for a set of console-scripts might not be appropriate. In this case, it is possible to add a Python extra entry for a console script to add additional python dependencies. For instance, this section in setup.cfg:

[extras]
zuul_base =
    PyMySQL
    psycopg2-binary
zuul_executor =
    ara

Will cause PyMySQL and psycopg2-binary to be installed into the base image (even though they are optional dependencies for a normal install) and for ara to be installed in the zuul-executor image.

Note

It is important to note that underscores must be used in the extras definition in place of dashes.

References¶

References of pbrx.

pbrx documentation

Welcome to the documentation of pbrx¶

pbrx¶

Features¶

pbrx service installation guide¶

pbrx service overview¶

Install and configure¶

Install and configure for openSUSE and SUSE Linux Enterprise¶

Prerequisites¶

Install and configure components¶

Finalize installation¶

Install and configure for Red Hat Enterprise Linux and CentOS¶

Prerequisites¶

Install and configure components¶

Finalize installation¶

Install and configure for Ubuntu¶

Prerequisites¶

Install and configure components¶

Finalize installation¶

Verify operation¶

Next steps¶

Contributor Documentation¶

Contributing¶

Configuration¶

Command line interface reference¶

Users guide¶

Installation of Sibling Packages¶

Building Container Images¶

Distro Depends¶

Python Dependencies¶

References¶

Indices and tables¶

pbrx

Page Contents