Logicalwebhost Cheatsheet

Linux & Open Source Cheatsheets & Howto's

Skip to: Content | Sidebar | Footer

kvm virtualization howto

KVM (not keyboard/video/mouse, used for viewing multiple screens and also called KVM) is a virtualization technology where you can run lots of different operating systems if you have the right cpu for it, like a Xeon or AMD one that supports virtualization. You have to have a remote client on your LAPTOP called virt-manager (I could only find a client for Linux/BSD, there may be some for Windows/Mac though?) to load the operating systems on your SERVER, which really sucks compared to xen, but xen is limited by the number/types of guest operating system

KVM install howto

This is on Debian, though others will apply will modification. Do this on your SERVER:

egrep '(vmx|svm)' --color=always /proc/cpuinfo

you should get a few lines of code output, this means your CPU will work with KVM, if you don’t stop, you can’t install it, you’ll have to use another Virtual Machine technology, like xen/virtualbox/VMWare, etc.

aptitude install qemu-kvm libvirt-bin virtinst
adduser whateveryournormalusernameis libvirt-qemu
vi /etc/libvirt/libvirtd.conf
  # unix_sock_rw_perms = "0770" <-- comment this line out and add one below:
  unix_sock_rw_perms = "0777"
sysctl vm.swappiness=0
echo "vm.swappiness=0" >> /etc/sysctl.conf
/etc/init.d/libvirt-bin restart (/etc/init.d/libvirtd in jessie)
vi /etc/network/interfaces (change it to look like this, but with your IP info)
  # The loopback network interface
  auto lo
  iface lo inet loopback
  auto eth0
  iface eth0 inet manual
  auto br0
  iface br0 inet static
        bridge_ports eth0
        bridge_fd 9
        bridge_hello 2
        bridge_maxage 12
        bridge_stp off
/etc/init.d/networking restart
see if you can still ping google.com
mkdir /root/.ssh
touch /root/.ssh/authorized_keys
vi /etc/ssh/sshd_config
  PermitRootLogin yes
  AllowUsers root@your.laptop.i.p anotheruser someotheruser (allow here everyone you want to be able to ssh in, you can not login with virt-manager on your laptop unless you login from this IP)
/etc/init.d/ssh reload

on your LAPTOP do:

sudo apt-get install virt-manager

now you have to set up an automatic connection to your KVM SERVER, so on your LAPTOP set it up with ssh/keys (as your REGULAR USER, not root) like:

ssh-keygen -t dsa -f ~/.ssh/id_dsa -C "yourusername@"
  > hit enter 2x on passphrase
cat ~/.ssh/id_dsa.pub | ssh root@server.ip.or.hostname 'cat - >> ~/.ssh/authorized_keys'
ssh roote@server.ip.or.hostname (you should nott get a password prompt, it should just log you in)

now you have to set up a machine on your SERVER from your LAPTOP, but first you have to have an install .iso on the SERVER that you can do the install from, so download your .iso on the SERVER, and put it in /var/lib/libvirt/images, in this example we get a Debian .iso from a mirror by logging into your SERVER and doing:

cd /var/lib/libvirt/images
wget http://ftp.osuosl.org/pub/debian-cdimage/6.0.6/amd64/iso-cd/debian-6.0.6-amd64-CD-1.iso

Now, back on your LAPTOP, run virt-manager -> File -> Add connection:

Connect to remote host <--check checkbox
Method: ssh (should be by default)
Username: yourusernameonSERVER
Hostname: ip.or.hostname.ofSERVER

Now right-click on your new connection in the window, and do -> Connect. It shouldn’t prompt you for a password. Now click icon above connection to “create new virtual machine”, which it will do on the SERVER over your ssh connection now. Here’s an example machine I’m creating with the Debian .iso we just downloaded on the SERVER:

Name: whateveryouwanttocallyournewVM
Local install media checked <-- here it means local on your SERVER
Next button--->
Use ISO image <- click browse, select the image we just downloaded
OS type: whatever you are installing, same with Version
Forware --->
RAM/CPU for whatever you want
Forward --->
Change disk size to what you want
Forward --->
Advanced Options -> changet to "Specify shared device name"
Bridge name: br0
Architecture: select to match .iso
Finish --->
Allocates disk...

it should be done now, and you can click on the newly created machine and start it up and go through the install on the window on your LAPTOP.

KVM gotcha’s

if there is a not a /dev/kvm, it means your processor may have virtualization disabled in the bios. Look under advanced options for your processor, and enable virtualization. You will probably have to completely power off the server and then power it back on to enable it. after that, once it boots, run:

/etc/init.d/qemu-kvm restart (you should see something like:)
  Succesfully unloaded kvm module kvm_intel.
  Loading kvm module kvm_intel.

then double-check you have something listed in /dev/kvm

move a kvm guest to another server

You can shut down the vm, scp it to another server, then tell kvm on the new server about it, and you should be able to use it on your new server. This worked moving a guest from a Debian Squeeze server running qemu-kvm to another server running Debian Wheezy qemu-kvm on the same subnet so I didn’t have to change the guest IP configs. Here, the source server is server1 and destination server is server2. Make sure you shut down the guest on server1 before moving it, I’ll assume you’ve already done that.

server1#: su
cd /var/lib/libvirt/images/
list --all
shutdown whateverdomainyouremoving
undefine whateverdomainyouremoving
  Domain whateverdomainyouremoving has been undefined
scp guestmachinename.img ser.ver.2.ip:/var/lib/libvirt/images/
cd /etc/libvirt/qemu/
scp guestmachinename.xml ser.ver.2.ip:/etc/libvirt/qemu/
ssh user@ser.ver.2.ip
cd /etc/libvirt/qemu/
virsh define guestmachinename.xml
list --all (should now show guestmachinename)
start guestmachinename

you should now see your new machine in virt-manager on your laptop, or in virsh if you use that from the command line, and be able to login and use it on the new box.

machine won’t start after migrating

if you get an error like:

error: Failed to start domain videobox
error: internal error Process exited while reading console log output: Supported machines are:
pc         Standard PC (alias of pc-0.12)
pc-0.12    Standard PC (default)
pc-0.11    Standard PC, qemu 0.11
pc-0.10    Standard PC, qemu 0.10
isapc      ISA-only PC

it means you have to change the machine type in your migrated domain xml, so change it to something on the list in the error message like:

vi /etc/libvirt/qemu/yourdomain.xml
  <type arch='x86_64' machine='pc-0.12'>hvm</type>
virsh define yourdomain.xml
virsh start yourdomain

Resize KVM LVM partition

You can resize a normal partition using a recent version of the KVM/qemu tools, but resizing the LVM/VG/PV/partition inside that is a bit more tricky. You should back everything up before you start, or even better, just copy the image to a new one in case everything gets screwed up.

cd /var/lib/libvirt/images
> list --all

That should show all your machines, running or not, so shutdown the one you want by either logging in and shutting it down, or doing:

> shutdown whateverdomain
  domain is shutting down 
(wait a bit and then see if it really is by doing:)
> list --all
  whateverdomain     not running

now copy your image first to a backup like:

cp whateverdomain.img whateverdomain.img.bak

Now you can resize your image. In our example we resize a 20G (or whatever) to increase it to 30G (or whatever your current size + 10 GB like:

qemu-img resize vmdisk.img +10G
ls -lah *

you should see that your image “grew”, this takes a surprisingly short amount of time, like a few seconds on a powerful Xeon machine.

Now here comes the tricky part where you have to boot to a Gparted liveCD FROM INSIDE YOUR VM YOU JUST RESIZED. To do this, you have to use a virt-manager client on a Linux laptop somewhere and connect to your server running KVM for the next steps.

1. Login to virt-manager, connect to your KVM server host and then double-click on the icon for the virtual machine you’re resizing, BUT DON’T START IT ACCIDENTALLY, it has to be shut down still

2. Go to Show Details > Boot Options, check the box that says “IDE cdrom”, then use the arrow keys to move that entry to the top of the list there. Now go to the “IDE Cdrom” button and then browse and select the Gparted live .iso, which also has to be located in /var/lib/libvirt/images on your host KVM server to be visible to the browse function, so scp it to your server or use wget to download it directly to the server. Also chown libvirt-qemu.libvirt-qemu gparted-whatever.iso, so your virt-manager will see it. Sometimes you have to exit virt-manager and go back and redo the browse if it doesn’t pick it up.

3. Now when you start that machine up, you should see the boot screen for Gparted, not whatever OS is running normally on that machine. If you don’t stop and troubleshoot, you don’t want to screw up your original OS, so be sure of this.

4. On the boot screen just hit enter in most cases, that will take you to a GUI that has icons for running stuff. You actually need a terminal, but you just double-click on the terminal icon on the Gparted desktop.

5. Now you have to figure out which disk is which, otherwise if you select the wrong disk/volume you stand a fair chance of losing your data on that disk, which is never good. So take your time here and double-check you’re working on the right thing. You either have a MBR or a UEFI, and basically if you don’t know what the difference is, assume you have a normal MBR, which means you’d use fdisk (rather than gdisk/parted), and run:


build VM from command line

you can use virt-install to build a new VM from the command line like:

virt-install \
        --name whatevername \
        --accelerate \
        --virt-type kvm \
        --vnc \
        --description "Some description you want" \
        --memory 1024 \
        --disk path=/var/lib/libvirt/images/whatever.img,bus=virtio,size=100 \
        --vcpus 2 \
        --os-type Linux \
        --os-variant debianwheezy \
        --cdrom /var/lib/libvirt/images/debian-8.5.0-amd64-DVD-1.iso \
        --network bridge=br0 \
        --console pty,target_type=serial
Starting install...
Allocating 'whatever.img'                                                                              | 100 GB     00:00     
Creating domain...                                                                                     |    0 B     00:00     
Unable to init server: Could not connect: Connection refused
Cannot open display: 
Run 'virt-viewer --help' to see a full list of available command line options
Domain installation still in progress. You can reconnect to 
the console to complete the installation process.

Now you login with your virt-manager on a remote laptop and finish the install.

KVM second interface eth1

You can add another network interface, typically eth1 to your KVM host, and then let guests use it too. First, set up the second bridge, “br1”, just add it below your other bridge at the end of the file like:

auto eth1
iface eth1 inet manual
auto br1
iface br1 inet static
        address sec.on.dary.ip
        gateway sec.on.dary.gateway
        bridge_ports eth1
        bridge_fd 9
        bridge_hello 2
        bridge_maxage 12
        bridge_stp off

Now you can either manually reboot the server, restart networking (and hope it works, sometimes doesn’t on Debian), or just bring up your second interface like:

ifconfig br1 sec.on.dary.ip netmask up

Now boot your guest VM, then once you do, make your GUEST VM /etc/network/interfaces (change to match your specific environment) look something like:

# The primary network interface
allow-hotplug eth1
iface eth1 inet static
        address pub.lic.static.ip
        gateway pub.lic.static.gateway
        post-up route del default gw whate.ver.private.ip eth0
        post-up route add default gw pub.lic.ip.gateway eth1
# The primary network interface
allow-hotplug eth0
iface eth0 inet static
        address whatever.private.i.p
        gateway private.ip.gate.way

If you don’t do the post-up delete/add, it will try to route stuff out your private IP/subnet, instead of your public one.

References: http://wiki.debian.org/KVM

bridging for IDS/promiscuous monitoring

First, set up the second interface so it will bridge to eth1:

auto eth1
iface eth1 inet manual
#Alienvault OSIM Interface
auto br1
    iface br1 inet manual
    bridge_ports eth1
    bridge_stp off


ifup eth1

then see if it’s capturing packets like:

tcpdump -i br1

you should get a ton of stuff like:

14:46:53.507328 IP > ipv4_1.cxl0.c154.sea001.ix.nflxvideo.net.https: Flags [.], ack 2897, win 5611, options [nop,nop,TS val 3160018074 ecr 2918482309,nop,nop,sack 1 {28961:46337}], length 0
 14:46:53.507402 IP ipv4_1.cxl0.c154.sea001.ix.nflxvideo.net.https > Flags [.], seq 56473:57921, ack 0, win 2050, options [nop,nop,TS val 2918482612 ecr 3160018052], length 1448

like pages and pages flowing past very quickly, this is good. Now add your bridge like:

brctl setageing br1 0
brctl setfd br1 0

Now fire up your VM and add a second NIC of > Specify shared device name > br1. You’ll probably have to reboot the VM, but then login to the VM and you should see a second NIC of eth1, which is bridged to your mirror interface and can start looking at packets. I still haven’t figured out how to get vswitch to copy mirror packets to multiple interfaces, in case you want to have multiple sensors look at the same data, so that’s my next issue. To make your brctl config persist a reboot, do:

cd /etc/network/if-up.d
touch br1-mirror
chmod +x br1-mirror
vi br1-mirror
  if [ "$IFACE" = br1 ]; then
  brctl setageing br1 0
  brctl setfd br3 0

Mad props to the help here and also also this guy, very helpful all http://www.ryanhallman.com/kvm-configure-mirrored-ports-traffic-to-be-visible-in-guest-snort/ hope this all helps someone else, tough to figure out how to bolt all the pieces together.

Also, bridge tutorial:


Write a comment

You need to login to post comments!