Brief Description

In this tutorial, we are going to build ONOS controller cluster using tutorial VM image. (Download here)

At least two VM instances are needed to build a cluster, obviously. We use two in this tutorial.

Load ONOS Developer Environment

. ~/onos/tools/dev/bash_profile

Fix Environment Variables

ONOS_USER, KARAF_ROOT and PATH are incorrect in the tutorial VM.

# Fix ONOS_USER such that we can use ONOS deployment tools
export ONOS_USER=tutorial1
# Fix KARAF_ROOT and PATH such that we can use ONOS remote client
export KARAF_ROOT=/opt/onos/apache-karaf-$KARAF_VERSION
export PATH="$PATH:$KARAF_ROOT/bin"

Configure Node Information

If you are not using VirtualBox, the ONOS_NIC could probably be wrong. I use VMWare Fusion on OS X, in which DHCP leases 192.168.136.0/24 by default rather than 192.168.56.0/24.

# IP of this controller (optional, just set up for convenience)
export OCI=192.168.136.131
# IP of controller 1 (optional, just set up for convenience)
export OC1=192.168.136.131
# IP of controller 2 (optional, just set up for convenience)
export OC2=192.168.136.132
# Interface for inter-ONOS node communication (mandetory)
export ONOS_NIC=192.168.136.*

ONOS_FEATURES is also an important environment variable which should be set correctly when we load developer enviroment. Just make sure onos-core is included.

Create and Deploy the Package

# Create package
onos-package
# Deploy on OC1
onos-patch-vm $OC1
onos-install -f $OC1
# Deploy on OC2
onos-patch-vm $OC2
onos-install -f $OC2

Check Cluster Status

  • Login to ONOS console
onos $OC1
  • Check cluster status
onos> summary
node=192.168.136.131, version=1.0.0.tutorial1~2014/12/06@07:30
nodes=2, devices=0, links=0, hosts=0, SCC(s)=0, paths=0, flows=0, intents=0

You should see nodes=2 if nothing goes wrong.

Start Mininet

Mininet switches need to be reconfigure to communicates with all controllers in the cluster. Take ~/onos/tools/test/topos/tower.py for example, we need to add both controllers in the script.

diff --git a/tools/test/topos/tower.py b/tools/test/topos/tower.py
index b75bfe4..6ef28ce 100755
--- a/tools/test/topos/tower.py
+++ b/tools/test/topos/tower.py
@@ -46,6 +46,8 @@ topos = { 'tower': TowerTopo }
def run():
    topo = TowerTopo()
    net = Mininet( topo=topo, controller=RemoteController, autoSetMacs=True )
+    net.addController( 'c1', ip='192.168.136.131', port=6633 )
+    net.addController( 'c2', ip='192.168.136.132', port=6633 )
    net.start()
    CLI( net )
    net.stop()

After modification, start Mininet by

sudo python ~/onos/tools/test/topos/tower.py

Check Cluster Status, Again

Now you should see something like this

onos> summary
node=192.168.136.131, version=1.0.0.tutorial1~2014/12/06@07:30
nodes=2, devices=6, links=18, hosts=0, SCC(s)=1, paths=42, flows=0, intents=1

onos> nodes
id=192.168.136.131, address=192.168.136.131:9876, state=ACTIVE *
id=192.168.136.132, address=192.168.136.132:9876, state=ACTIVE 

onos> masters
192.168.136.131: 3 devices
  of:0000000000000001
  of:0000000000000002
  of:000000000000000d
192.168.136.132: 3 devices
  of:000000000000000b
  of:000000000000000c
  of:000000000000000e

What’s Next?

This environment is now ready for you to continue the official tutorial about Distributed ONOS. You can now change mastership, or, kill one of the two controllers by rebooting it. Have fun.