Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
95 changes: 95 additions & 0 deletions calico/operations/upgrading/openstack-upgrade.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,11 @@ varies by Linux distribution.

- [Upgrading an OpenStack cluster based on Ubuntu](#upgrading-an-openstack-cluster-based-on-ubuntu)

If you are upgrading from a version earlier than v3.32, you must also run a
one-time policy data migration after the package upgrade; see
[Migrating policy data](#migrating-policy-data-when-upgrading-from-earlier-than-v332)
below.

:::note

Do not use older versions of `calicoctl` after the upgrade.
Expand Down Expand Up @@ -99,6 +104,9 @@ This may result in unexpected behavior and data.

1. Remove any existing `calicoctl` instances and [install the new `calicoctl`](../calicoctl/install.mdx).

1. If you are upgrading from a version earlier than v3.32, run the
[policy data migration](#migrating-policy-data-when-upgrading-from-earlier-than-v332).

1. Congratulations! You have upgraded to $[prodname] $[version].

## Upgrading an OpenStack cluster based on Ubuntu
Expand Down Expand Up @@ -163,4 +171,91 @@ This may result in unexpected behavior and data.

1. Remove any existing `calicoctl` instances and [install the new `calicoctl`](../calicoctl/install.mdx).

1. If you are upgrading from a version earlier than v3.32, run the
[policy data migration](#migrating-policy-data-when-upgrading-from-earlier-than-v332).

1. Congratulations! You have upgraded to $[prodname] $[version].

## Migrating policy data when upgrading from earlier than v3.32

In $[prodname] v3.32, the naming convention for policies stored in the etcd
datastore changed: policies in the default tier are now stored under their
plain name, whereas $[prodname] v3.29.4 through v3.31.x stored them under a
tier-prefixed name such as `default.my-policy`. On Kubernetes clusters this
data is migrated automatically by kube-controllers. OpenStack clusters do not
run kube-controllers as a service, so the same migration must be run once, by
hand, after upgrading.

If you skip this step, policies created before the upgrade misbehave in the
following ways, until each one is migrated or recreated:

- `calicoctl get`, `calicoctl delete`, and similar commands only find such a
policy under its stored name, e.g.
`calicoctl get globalnetworkpolicy default.my-policy`; using the plain name
reports that the resource does not exist.

- Re-applying such a policy under its plain name creates a second, independent
policy object instead of updating the existing one. Both objects are then
live policy until one of them is deleted.

- The `%n` and `%p` specifiers in Felix's `LogPrefix` configuration expand to
the stored name, so log entries for pre-upgrade policies include the
`default.` prefix while entries for newly written policies do not.

### Before you begin

- **Upgrade every node first.** All Felix instances (compute nodes) and all
neutron-server instances (control nodes) must be running $[prodname] v3.32
or later before you run the migration. On a Kubernetes cluster this
ordering is verified automatically; on OpenStack it is your responsibility.

- **Install and configure `calicoctl`.** You need to be able to run
[`calicoctl`](../../operations/calicoctl/index.mdx) successfully on at least
one cluster node.

- **Resolve any duplicate policies.** If policies have already been
re-applied since the upgrade, you may have duplicate pairs - the same policy
name listed twice by `calicoctl get globalnetworkpolicy`, one showing the
tier `default` and one with an empty tier column. For each pair, compare
the two objects:

```
calicoctl get globalnetworkpolicy default.my-policy -o yaml
calicoctl get globalnetworkpolicy my-policy -o yaml
```

and, once you have confirmed that the plain-named object has the spec you
want, delete the legacy one:

```
calicoctl delete globalnetworkpolicy default.my-policy
```

If you don't resolve a duplicate pair yourself, the migration keeps the
plain-named object and deletes the legacy one.

### Run the migration

Run the following on any machine where calicoctl is installed and configured to
reach your etcd datastore:

```bash
calicoctl datastore migrate-policy-names
```

### Verify

Pick a policy that was created before the upgrade and confirm that it is now
accessible by its plain name:

```
calicoctl get globalnetworkpolicy my-policy
```

and that the legacy name is gone:

```
calicoctl get globalnetworkpolicy default.my-policy
```

The second command should report that the resource does not exist.
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,11 @@ varies by Linux distribution.

- [Upgrading an OpenStack cluster based on Ubuntu](#upgrading-an-openstack-cluster-based-on-ubuntu)

If you are upgrading from a version earlier than v3.32, you must also run a
one-time policy data migration after the package upgrade; see
[Migrating policy data](#migrating-policy-data-when-upgrading-from-earlier-than-v332)
below.

:::note

Do not use older versions of `calicoctl` after the upgrade.
Expand Down Expand Up @@ -99,6 +104,9 @@ This may result in unexpected behavior and data.

1. Remove any existing `calicoctl` instances and [install the new `calicoctl`](../calicoctl/install.mdx).

1. If you are upgrading from a version earlier than v3.32, run the
[policy data migration](#migrating-policy-data-when-upgrading-from-earlier-than-v332).

1. Congratulations! You have upgraded to $[prodname] $[version].

## Upgrading an OpenStack cluster based on Ubuntu
Expand Down Expand Up @@ -163,4 +171,99 @@ This may result in unexpected behavior and data.

1. Remove any existing `calicoctl` instances and [install the new `calicoctl`](../calicoctl/install.mdx).

1. If you are upgrading from a version earlier than v3.32, run the
[policy data migration](#migrating-policy-data-when-upgrading-from-earlier-than-v332).

1. Congratulations! You have upgraded to $[prodname] $[version].

## Migrating policy data when upgrading from earlier than v3.32

In $[prodname] v3.32, the naming convention for policies stored in the etcd
datastore changed: policies in the default tier are now stored under their
plain name, whereas $[prodname] v3.29.4 through v3.31.x stored them under a
tier-prefixed name such as `default.my-policy`. On Kubernetes clusters this
data is migrated automatically by kube-controllers. OpenStack clusters do not
run kube-controllers as a service, so the same migration must be run once, by
hand, after upgrading.

If you skip this step, policies created before the upgrade misbehave in the
following ways, until each one is migrated or recreated:

- `calicoctl get`, `calicoctl delete`, and similar commands only find such a
policy under its stored name, e.g.
`calicoctl get globalnetworkpolicy default.my-policy`; using the plain name
reports that the resource does not exist.

- Re-applying such a policy under its plain name creates a second, independent
policy object instead of updating the existing one. Both objects are then
live policy until one of them is deleted.

- The `%n` and `%p` specifiers in Felix's `LogPrefix` configuration expand to
the stored name, so log entries for pre-upgrade policies include the
`default.` prefix while entries for newly written policies do not.

### Before you begin

- **Upgrade every node first.** All Felix instances (compute nodes) and all
neutron-server instances (control nodes) must be running $[prodname] v3.32
or later before you run the migration. On a Kubernetes cluster this
ordering is verified automatically; on OpenStack it is your responsibility.

- **Install and configure `calicoctl`.** You need to be able to run
[`calicoctl`](../../operations/calicoctl/index.mdx) successfully on at least
one cluster node.


- **Resolve any duplicate policies.** If policies have already been
re-applied since the upgrade, you may have duplicate pairs - the same policy
name listed twice by `calicoctl get globalnetworkpolicy`, one showing the
tier `default` and one with an empty tier column. For each pair, compare
the two objects:

```
calicoctl get globalnetworkpolicy default.my-policy -o yaml
calicoctl get globalnetworkpolicy my-policy -o yaml
```

and, once you have confirmed that the plain-named object has the spec you
want, delete the legacy one:

```
calicoctl delete globalnetworkpolicy default.my-policy
```

If you don't resolve a duplicate pair yourself, the migration keeps the
plain-named object and deletes the legacy one.

### Run the migration

:::note

This requires the v3.32.2 (or later) image; it is not present in v3.32.0 or
v3.32.1.

:::

Run the following on any machine where calicoctl is installed and configured to
reach your etcd datastore:

```bash
calicoctl datastore migrate-policy-names
```

### Verify

Pick a policy that was created before the upgrade and confirm that it is now
accessible by its plain name:

```
calicoctl get globalnetworkpolicy my-policy
```

and that the legacy name is gone:

```
calicoctl get globalnetworkpolicy default.my-policy
```

The second command should report that the resource does not exist.