1.) KVM agent networking schema changes 2.) Known upgrade issues & the common upgrade method 3.) Troubleshooting and work-arounds

1. ACS 4.1.0+ | KVM Agent Networking
Schema And Upgrading
Kelcey Damage | 11/24/2013

2. Foreword
We are going to take a quick look at the networking schema
changes introduced in 4.2 for KVM agents, as well as the
upgrade process existing users.
ACS 4.1.0+ | KVM Agent Networking Schema And Upgrading

3. Agenda
1.) KVM agent networking schema changes
2.) Known upgrade issues & the common upgrade method
3.) Troubleshooting and work-arounds
ACS 4.1.0+ | KVM Agent Networking Schema And Upgrading

4. KVM networking
schema changes
The KVM Agent
There are two key areas of
interest in regards to the
changes made to networking
with KVM and Cloudstack
Schema Changes
Impact On Production
ACS 4.1.0+ | KVM Agent Networking Schema And Upgrading

5. bridge name
bridge id
STP enabled
interfaces
brbondg-1205
8000.0023aef93c7c
no
brbondg-1213
8000.0023aef93c7c
no
cloudVirBr1205
cloudVirBr1213
8000.0023aef93c7c
8000.0023aef93c7c
no
no
bondg.1205
vnet7
bondg.1213
vnet12
vnet5
vnet9
cloud0
8000.fe00a9fe014f
no
cloudbrguest
cloudbrman
8000.0023aef93c7c
8000.0026b9fdb774
yes
yes
cloudbrpub
8000.0023aef93c80
yes
virbr0
8000.525400368385
yes
vnet0
vnet4
vnet8
bondg
bondm
vnet1
vnet3
bondp
vnet2
vnet6
virbr0-nic
Bridge Schema
ACS 4.1.0+ | KVM Agent Networking Schema And Upgrading

6. Interfaces
What’s Changed:
Interfaces now require names
shorter them 8 characters.
KVM Agent
Schema as of
4.1.0+
Why:
Because the new distributed portgroup names are concatenations
of the VLAN ID and the interface
name.
“br{interface_name}-{VLAN#}”
What this replaces:
“cloudVirBr{VLAN#}”
ACS 4.1.0+ | KVM Agent Networking Schema And Upgrading

7. Important note on interfaces
If using any name other then the default:
Effect on production systems
•
“eth/em/p1p”…
… such as bonds or aliases then the
bond or alias name ..//
//..
must be shorter then 8 characters.
ACS 4.1.0+ | KVM Agent Networking Schema And Upgrading
Longer names will get truncated.
•
Routing will break and traffic may be
shared within VLANs.
Example:
•
Naming a bond as ‘bondguest’ with
a VLAN of 1204 will create the portgroup ‘brbondguest-120’. VLANs
1200 – 1209 will all get assigned to
this port-group

8. Upgrading to 4.2+
and beyond
The Upgrade Process
There are two key areas of
interest in the upgrade process
for anyone moving to the latest
version.
Documentation
The New System VM
ACS 4.1.0+ | KVM Agent Networking Schema And Upgrading

9. 1
Documentation
In the official release notes, the
steps for upgrading from 4.0
forward to 4.2+ were missing the
section on how and where to
download the latest templates.
2
The New System VM
We now use a virtual-IO socket
for communication with the
system VM, instead of the prior
‘patchdisk’ method of passing
cmdline and authorized_keys.
ACS 4.1.0+ | KVM Agent Networking Schema And Upgrading

10. Major changes in the templates require updating them to the
latest version
prior to upgrading the CloudStack.
ACS 4.1.0+ | KVM Agent Networking Schema And Upgrading

11. KVM
Name: systemvm-kvm-4.2
Description: systemvm-kvm-4.2
URL:
http://download.cloud.com/templates/4.2/systemvmtempl
ate-2013-06-12-master-kvm.qcow2.bz2
Zone: Choose the zone where this hypervisor is used
Hypervisor: KVM
Format: QCOW2
OS Type: Debian GNU/Linux 7.0 (32-bit) (or the highest
Debian release number available in the dropdown)
Extractable: no
Password Enabled: no
Public: no
Featured: no
ACS 4.1.0+ | KVM Agent Networking Schema And Upgrading

12. Oh No! I Broke My
Cloud
The Fix
If you’ve gone ahead and
upgraded your CloudStack version
before upgrading your templates,
there are ways to fix your cloud.
The Database Method
The Java Method
ACS 4.1.0+ | KVM Agent Networking Schema And Upgrading

13. The Database Method
o
Step 1): Mount your secondary storage to your management server
mount -t nfs {ip_of_storage_server}:[path_to_secondary_storage] /mnt
o
Step 2): Download the latest version of the templates:
/usr/share/cloudstack-common/scripts/storage/secondary/cloudinstall-systmplt -m /mnt -u http://download.cloud.com/templates/4.2/systemvmtemplate2013-06-12-master-kvm.qcow2.bz2 -h kvm -F
o
Step 3): Find the name of the old template in the database:
USE cloud;
SELECT install_path FROM template_host_ref WHERE template_id=3;
o
Step 4): write down the name of the .qcow2 file that your given in the previous step.
o
Step 5): from the management server locate the new template on the mounted
secondary storage:
cd /mnt/template/tmpl/1/3/
ACS 4.1.0+ | KVM Agent Networking Schema And Upgrading

14. The Database Method
o
Step 6): rename the .qcow2 file in that folder to the name we copied from the
database.
o
Step 7): edit the template.properties file in the same folder and change both instances
of the old name to the new one.
o
Step 8): we need to reset the cached template in the database:
UPDATE template_spool_ref SET
download_pct='0',download_state='NOT_DOWNLOADED',state='NULL',local_path='NULL‘
,install_path='NULL',template_size='0' WHERE template_id='3‘;
o
Step 9): Unmount your secondary storage from the management server:
umount /mnt
o
Step 10): disable the zone from the management UI.
ACS 4.1.0+ | KVM Agent Networking Schema And Upgrading

15. The Database Method
o
Step 11): update the database records for your system VMs to be ‘Stopped’. You will
need to do this for both the Secondary Storage VM and the Console Proxy. The ID of the
system VM is the number in it’s name, for example; s-34-VM,, would have an ID of ’34′.
UPDATE vm_instance SET state='Stopped' where id='{id_of_system_vm}';
o
Step 12): From the management UI, destroy both the system VMs.
o
Step 13): Once both system VMs have been destroyed, re-enable the zone.
o
Step 14): Tail the management log and watch for the VMs to start.
tail -f /var/log/cloudstack/management/management-server.log
o
Step 15): Enjoy your 4.2+ Cloud!
ACS 4.1.0+ | KVM Agent Networking Schema And Upgrading

16. IYSE Method
*(If Your Smart Enough)
For those in the USR tract that
understand gobble-de-gook below
is the JAVA method that some
prefer.
(1) register a new template (such as
systemvmtemplate-4.2) on UI or in
Upgrade*.java file
(2) Change template type from User to
System in Upgrade*.java file.
(3) cloud-install-sys-tmplt download the
system-vm template
(4) cloudstack-sysvmadm re-deploy
system-vms
ACS 4.1.0+ | KVM Agent Networking Schema And Upgrading

17. Reference
https://issues.apache.org/jira/browse/CLOUDSTACK-4826
http://cloud.kelceydamage.com/cloudfire/blog/2013/10/08/conq
uering-the-cloudstack-4-2-dragon-kvm/
ACS 4.1.0+ | KVM Agent Networking Schema And Upgrading

18. Get Involved
Web: http://cloudstack.apache.org/
Mailing Lists: cloudstack.apache.org/mailing-lists.html
IRC: irc.freenode.net: 6667 #cloudstack
Twitter: @cloudstack
LinkedIn: www.linkedin.com/groups/CloudStack-Users-Group-3144859
If it didn’t happen on the mailing list, it didn’t happen.
ACS 4.1.0+ | KVM Agent Networking Schema And Upgrading