The sip:provider PRO version mr5.4.1 has several important changes comparing to the previous release:
Please find the complete changelog in our release notes on our WEB site.
| warning | |
Make sure that all the SIP domains and peering servers have appropriate rtp_interface option (e.g. ext) selected in the NAT and Media Flow Control section. If you leave default there, then incorrect network interface may be used for sending and receiving RTP media after the software upgrade. |
Make sure you’re prepared to spend two hours or so upgrading the system. There can be service interruptions, so also notify the customer and get their approval.
Check the system overal status:
ngcp-status --all
Check the system for locally modified files (move them to appropriate customtt.tt2 files if necessary):
ngcp-status --integrity
Try to find local changes to the template files by issuing:
find /etc/ngcp-config -name \*customtt.tt2
You will also need to find the dpkg-dist files under the templates files because people sometimes forget about creating customtt files and edit tt2 files directly. That makes upgrades not to replace the tt2 files. If so, you need to treat the tt2 files as if they were customtt files and make sure you merge the new templates with the changes of the old ones.
find /etc/ngcp-config -name \*.tt2.dpkg-dist
Also, please check/clean old dpkg backup files (just in case if previous person did the previous step not carefully enough). Normally the list should be empty:
find /etc/ngcp-config -name \*.tt2.dpkg\*
You will have to understand why the changes are there and if they are still needed after the upgrade. You should create a ticket in the bugtracker if there isn’t one yet.
Log into the two servers. Use their real IPs so you can switch the cluster forth and back later on. Make sure the cluster status is ok - on both nodes issue:
Slave_IO_Running: Yes Slave_SQL_Running: Yes Seconds_Behind_Master: 0
A cluster fail-over could be a good idea to see if everything works on the second node too. On the standby node issue:
ngcp-make-active
Afterwards again check monit, ngcp-check-active and ngcp-collective-check.
Create two test subscribers, or retrieve the credentials for two of them. Register a client to the platform and perform a test call between the two to ensure call routing works.
Run "apt-get update", ensure you have no warnings/errors here.
| warning | |
Installation may use locally specified mirrors. Discuss with a customer possibility to switch on Sipwise APT repositories (at least for the time of upgrades), the public Debian mirrors may not provide packages for old Releases anymore or be at least outdated! |
The sip:provider PRO system upgrade to mr5.4.1 will perform a couple of fundamental tasks:
So assuming you have a running sip:provider PRO system and want to upgrade it, start on the inactive node by upgrading software, then take over from the other node and then upgrade the other (now inactive) node, as detailed in the steps below.
Sipwise NGCP introduces Maintenance Mode with its mr5.4.1 release. The maintenance mode of NGCP will disable some background services (for instance: mediator) during the software upgrade. It thus prevents the system from getting into an inconsistent state while the upgrade is being performed. You can activate maintenance mode by applying a simple configuration change as described later.
| important | |
For sip:provider PRO systems running NGCP release earlier than mr5.4.1: As upgrading from an earlier release to mr5.4.1 or later will result in a system being aware and making use of the maintenance mode, it is necessary to prepare the NGCP for it before the software upgrade. |
Enable maintenance mode:
ngcpcfg pull
Edit /etc/ngcp-config/config.yml file:
maintenance: yes" in general section of the configuration file
general.maintenance
parameter to yes
general: maintenance: yes
Disabling background services. These tasks are for systems currently running NGCP release earlier than mr5.4.1. The maintenance mode of NGCP is not available in your system yet, before the software upgrade. Therefore it is necessary to manually disable the following background services to avoid potential inconsistency in system state:
To do:
Stop HB watchdog on all nodes:
monit stop hb_watchdog
Stop mediator and rate-o-mat services on both nodes :
monit stop mediator monit stop rate-o-mat
Prevent background services from (re)starting. You need to change configuration
in /etc/ngcp-config/config.yml file: set the enable parameter to "no" as shown below:
heartbeat:
hb_watchdog:
enable: no
...
mediator:
enabled: no
...
rateomat:
enable: noIn order to disable CDR exporter process you need to add a new custom template file because there is no such file by default:
config="/etc/cron.d/ngcp-cdr-exporter"
customtt="/etc/ngcp-config/templates/etc/cron.d/ngcp-cdr-exporter.customtt.tt2"
[ -f "${customtt}" ] || cp "${config}" "${customtt}"
vim "${customtt}"Comment out the single line in the file:
# 25,55 * * * * root . /etc/default/ngcp-roles; if ...
Please don’t forget to reverse the manual changes after the upgrade, as described in Enabling background services section of the handbook!
For all NGCP systems, regardless of their version:
Apply configuration changes, execute commands:
ngcpcfg apply 'Enable maintenance mode before the upgrade to mr5.4.1' ngcpcfg push
For upgrading the sip:provider PRO to the latest mr5.4.1 release, execute the following commands on both nodes:
NGCP_CURRENT_VERSION=$(cat /etc/ngcp_version) sed -i "s/$NGCP_CURRENT_VERSION/mr5.4.1/" /etc/apt/sources.list.d/sipwise.list ngcp-approx-cache-helper --auto --node localhost apt-get update apt-get install ngcp-upgrade-pro
Execute ngcp-upgrade in inactive node as root:
ngcp-upgrade
| info | |
sip:provider PRO can be upgraded to mr5.4.1 from previous release or previous build only. The script ngcp-upgrade will find all the possible destination releases for the upgrade and allow to choose the proper one. |
| info | |
If there is an error during upgrade, the ngcp-upgrade script will request
you to solve it. Once you’ve fixed the problem just re-execute |
The upgrade script will ask you to confirm that you want to start. Read the given information carefully, and if you agree, proceed with y.
The upgrade process will take several minutes, depending on your network connection and server performance. After everything has been updated successfully, it will finally ask you to reboot your system. Confirm to let the system reboot (it will boot with an updated kernel).
Merge/add the custom configuration templates if needed. Apply the changes to configuration templates if any and send them to the shared storage and the other node:
ngcpcfg apply 'upgrade node' ngcpcfg push --nobuild --noapply
Go to the new inactive node. Run ngcp-upgrade, as root:
ngcp-upgrade
The upgrade script will ask you to confirm that you want to start. Read the given information carefully, and if you agree, proceed with y.
The upgrade process will take several minutes, depending on your network connection and server performance. After everything has been updated successfully, it will finally ask you to reboot your system. Confirm to let the system reboot (it will boot with an updated kernel).
In order to disable maintenance mode do the following:
ngcpcfg pull
general: maintenance: no
Enabling background services. If you upgraded from NGCP release earlier than mr5.4.1 you have to manually reverse the changes you applied before the upgrade. (Disabling background services)
Enable background services to start. You need to change configuration
in /etc/ngcp-config/config.yml file: set the enable parameter to "yes" as shown below:
heartbeat:
hb_watchdog:
enable: yes
...
mediator:
enabled: yes
...
rateomat:
enable: yesIn order to enable CDR exporter process you need to remove customtt file and edit cron config file:
rm /etc/ngcp-config/templates/etc/cron.d/ngcp-cdr-exporter.customtt.tt2 vim /etc/cron.d/ngcp-cdr-exporter
Remove comment at the beginning of the single line of file on ALL nodes:
25,55 * * * * root . /etc/default/ngcp-roles; if ...
For all NGCP systems, regardless of their previous version:
ngcpcfg apply 'Disable maintenance mode after the upgrade to mr5.4.1' ngcpcfg push
When all finishes successfully check that replication is running.
Check ngcp-status --all.
Finally, do a basic functionality test.
Check web interface, register two test subscribers and perform a test
call between them to ensure call routing works.
| info | |
You can find a backup of some important configuration files of your existing installation under /var/backup/ngcp-mr5.4.1-* (where * is a place holder for a timestamp) in case you need to roll back something at any time. A log file of the upgrade procedure is available at /var/backup/ngcp-mr5.4.1-*/upgrade.log. |