/|\

Welcome to the Slackware Documentation Project

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
howtos:emulators:helper_script_for_managing_qemu_virtual_machines [2016/08/31 13:38 (UTC)]
louigi600 [VM Creation Example]
howtos:emulators:helper_script_for_managing_qemu_virtual_machines [2016/09/04 20:18 (UTC)]
louigi600 [Basic Configuration]
Line 1: Line 1:
 ====== Preface ====== ====== Preface ======
-Qemu is a popular and powerful open-source emulator often used for running KVM virtual machines. In fact qemu supports emulating so many thins that it can be quite challenging, unless you do it very often, to manually start a VM by using qemu-system-* from console. Ho would want to write the below command for starting a VM:+Qemu is a popular and powerful open-source emulator often used for running KVM Virtual Machines (VMs). In fact qemu supports emulating so many things that it can be quite challenging, unless you do it very often, to manually start a VM from a text console. Who would want to write the below command for starting a VM ?
  
   qemu-system-arm -name armedslack -M versatilepb -m 256 -k en-us -vnc :5,password -usb -kernel /VM/armedslack/zImage-versatile -initrd /VM/armedslack/initrd-versatile -append 'root=/dev/sda1 rootfs=ext2' -monitor telnet:127.0.0.1:1035,server,nowait  -drive file=/VM/armedslack/qemu_hdu.raw,index=0,media=disk -drive file=,index=1,media=cdrom  -net nic,macaddr=52:54:57:c0:2c:bb -net tap,ifname=tap5   qemu-system-arm -name armedslack -M versatilepb -m 256 -k en-us -vnc :5,password -usb -kernel /VM/armedslack/zImage-versatile -initrd /VM/armedslack/initrd-versatile -append 'root=/dev/sda1 rootfs=ext2' -monitor telnet:127.0.0.1:1035,server,nowait  -drive file=/VM/armedslack/qemu_hdu.raw,index=0,media=disk -drive file=,index=1,media=cdrom  -net nic,macaddr=52:54:57:c0:2c:bb -net tap,ifname=tap5
  
-Ok everyone might want console redirect on vnc and monitor redirect via telnet but non the less that's still a relatively small subset of the options supported by qemu-system-arm and only has one disk one cdrom and one Network Interface Controller (NIC) so things can be much worse then this.+Not everyone might want console redirect on vnc and monitor redirect via telnet but non the less that's still a relatively small subset of the options supported by qemu-system-arm and only has one disk one cdrom and one Network Interface Controller (NIC) so things can be much worse then this.
  
-Is is common practice that people running qemu VMs use some sort of software to help them createrun and maintain them. If you want an easy way out you might consider virtmanager or something like that. Personally I chose to manage my VMs from text console because for me it's an added value to be able to do such jobs even without GUI, leading me to write my onw scripts for managing my VMs.+Is is common, for people running qemu VMs, to use some sort of software for creatingrunning and maintain the VMs. If you want an easy way out you might consider virtmanager or something like that. Personally I chose to manage my VMs from text console because for me it's an added value to be able to do such jobs even without GUI, so possibly like many others I wrote my onw scripts for managing my qemu VMs.
  
-Over the years I've radically changed the helper script form having text configuration files for each VM to a centarl VM configuration database. I'd like to share my experience in doing so without presumptuously declaring that I do this any better then anyone else, letting you decide what's good or bad for your needs. It's likely that someone else has done this and a lot better then me but nevertheless I'still like to hare with you the route I took.+Over the years I've radically changed the helper script form having text configuration files for each VM to a centarl VM configuration database. I'd like to share my experience in doing so without presumptuously declaring that I do this any better then anyone else, letting you decide what's good or bad for your needs. It's likely that someone else has done this and a lot better then me but nevertheless I'still like to hare with you the route I took.
  
 ====== Problems ====== ====== Problems ======
Line 14: Line 14:
   * ability to run both x86 and ARM virtual machines   * ability to run both x86 and ARM virtual machines
   * ability to run several VMs simultaneously   * ability to run several VMs simultaneously
 +  * have the VMs appear as a real server in the LAN
   * flexibility on the number of disks assigned to a VM   * flexibility on the number of disks assigned to a VM
   * flexibility on the number of NICs assigned to a VM   * flexibility on the number of NICs assigned to a VM
-  * avoid conflicts on tap and MAC address 
   * avoid conflicts on VM console vnc port   * avoid conflicts on VM console vnc port
   * avoid conflicts on VM monitor port   * avoid conflicts on VM monitor port
-Dealing with such issues on text based configuration file per each VM started making the code unnecessarily complicated. + 
-Manually creating a configuration file for a new VM required looking for information across all previously configured VMs configuration files.+Wanting the VMs to appear like real servers on the LAN, along with my other requirements, made me opt for bridged tap NICs, which in turn created another potential conflict on the tap device ans MAC address. 
 + 
 +Using the initial, per VM text based configurationstarted to make it difficult to deal with such issues automatically (making the code unnecessarily long and complex) while manually creating a configuration file for a new VM required looking for information across all previously configured VMs to avoid potential conflicts 
  
 ====== Proposed Solution ====== ====== Proposed Solution ======
-It quickly became apparent to me that the VM configuration would need to be generated rather then manually created and that a central configuration repository would much aid the process. Again a text based central configuration file would make the code in inherently complicated (having to deal with an arbitrary number of VMs each with arbitrary number of disks and NICs).  +It quickly became apparent to me that the VM configuration would need to be generated rather then manually created and that a central configuration repository would much aid the process. Again a text based central configuration file would make, either the code or the config file, inherently complicated (having to deal with an arbitrary number of VMs each with arbitrary number of disks and NICs).  
-Having some experience on database administration made it a little unappealing to use LDAP for central repository and even if I had no DB experience at all I doubt I'd actually want the overhead of running LDAP just for this. Running MariaDB or Postgres for the same reason was rather unappealing toofor my small requirements, so I chose to use sqlite3. In my case it would be extremely rare that, 2 or more simultaneous executions of the management script, make a mess on the DB but if you have several people managing creating or deleting VMs you might want to opt for MariaDB or Postgres.+Having some experience on database administration made it a little unappealing to use LDAP for central repository and even if I had no DB experience at all I doubt I'd actually want the overhead of running LDAP just for this. Running MariaDB or Postgres was equally unappealing too for my small requirements, so I chose to use sqlite3. In my case it would be extremely rare that, 2 or more simultaneous executions of the management script, make a mess on the DB but if you have several people managing creating or deleting VMs you might want to opt for MariaDB or Postgres. 
 + 
 + 
 +Another thing that quickly became apparent was the almost repetitive code required to prompt for all the options so I decided to address that in 2 ways: 
 +  - have as much of the promoting automatically generated with a clever workaround 
 +  - use dialog to further simplify the UI for prompting
  
-Another thing quickly became apparent was that a lot of code was required to prompt for all the options on the text console. I find that dialog can be really handy for this along with making a better appealing interface. 
  
 ===== Basic Configuration ===== ===== Basic Configuration =====
-To get better flexibility to where things are stored it's a good idea to have a basic configuration file that tells the management script there the important things are:+To get better flexibility for configuring where things are stored it's a good idea to have a basic configuration file that tells the management script where the important things are:
   * Path to folder that will contain all the VMs   * Path to folder that will contain all the VMs
   * Path to where the centralized VM configuration DB is   * Path to where the centralized VM configuration DB is
Line 251: Line 257:
 As mentioned above, getting bugged with all the details required for configuring a new VM can be a a killer unless you do it really often so here's how I go about it in a similar fashion to creating the basic configuration. The idea is to reduce the amount of code written specifically to each VM option allowing for a relatively slender script and ease to add new VM options if required. As mentioned above, getting bugged with all the details required for configuring a new VM can be a a killer unless you do it really often so here's how I go about it in a similar fashion to creating the basic configuration. The idea is to reduce the amount of code written specifically to each VM option allowing for a relatively slender script and ease to add new VM options if required.
  
-Unfortunately here things are a little more complicated because different qemu-system-* produce slightly different output when prompted with help and because most of the VM options require a separate dialog asking for some specific action nonetheless a lot of the code and data required is self generated the **PARAMS** variable. The only specific code is to work around the differences between the various qemu-system-*.+Unfortunately here things are a little more complicated because different qemu-system-* produce slightly different output when prompted with help and because most of the VM options require a separate dialog asking for some specific action nonetheless a lot of the code required is self generated from the **PARAMS** variable. The only specific code is to work around the differences between the various qemu-system-* emulators.
  
   PARAMS="EMULATOR MACHINE CPU MEM KEYBOARD DISK DISKSIZE CDROM BIOS BOOT_ORDER KVM DISPLAY VNCPW USB KERNEL INITRD APPEND MAC"   PARAMS="EMULATOR MACHINE CPU MEM KEYBOARD DISK DISKSIZE CDROM BIOS BOOT_ORDER KVM DISPLAY VNCPW USB KERNEL INITRD APPEND MAC"
Line 285: Line 291:
     EMULATOR=$(< $DIALOG_OUTPUT)     EMULATOR=$(< $DIALOG_OUTPUT)
     case $EMULATOR in     case $EMULATOR in
-    qemu-system-i*86)  +    qemu-system-i*86|qemu-system-x86_64) 
-      MACHINE=pc +
-      grep -qwE "vmx|svm" /proc/cpuinfo && KVM=1 || KVM=0 +
-      [ $KVM -eq 1 ] && CPU=host || CPU=pentium3 +
-      BIOS="file=/usr/share/qemu/bios.bin" +
-      BOOT_ORDER="c" +
-    ;; +
-    qemu-system-x86_64)+
       MACHINE=pc       MACHINE=pc
       grep -qwE "vmx|svm" /proc/cpuinfo && KVM=1 || KVM=0       grep -qwE "vmx|svm" /proc/cpuinfo && KVM=1 || KVM=0
Line 362: Line 361:
   }   }
  
 +===== Networking =====
 +As mentioned above I want my VMs to look like real machines on the LAN the host server is connected on, this will require bridging the tap devices. Newer versions of qemu can automatically create and use tap device but I it will not do bridging and besides that it you need to tell it which tap device anyway. If you intend to run several VMs at once that all look like real servers on the LAN you will need to write qemu-ifup and  qemu-ifdown scripts in /etc to deal with that.
 +
 +I like to write a single script qemu-nethelper and have qemu-ifup and qemu-ifdown linked to it. The qemu-system-* emulators all execute /etc/qemu-ifup when bringing up a VM with a tap device, with the tap device parameter, and similarly execute /etc/qemu-ifdown when taking down a VM with a tap device. 
 +
 +As mentioned above newer versions of qemu (I think 1.1+) automatically create the tap device so the qemu-nethelper only needs to do the bridging. Now to make things a lot easier I like to have the host on which I run VMs with br0 configured at boot and then the qemu-nethelper only needs to add the tap device to the bridge, making it extremely simple. If br0 is already configured at boot then you need not restart any iptables so long as the chains use the bridge devices and the kernel has support for ebtables.
 +
 +  #!/bin/bash
 +  NAME=$(basename $0)
 +  tun_up ()
 +  { /sbin/ifconfig $1 0.0.0.0 promisc up
 +    /usr/bin/sleep 0.5
 +    /sbin/brctl addif br0 $1
 +  }
 +  
 +  tun_down ()
 +  { /sbin/ifconfig $1 down
 +    /usr/bin/sleep 0.5
 +    /sbin/brctl delif br0 $1 
 +  }
 +  
 +  case $NAME in
 +    qemu-ifup) tun_up $1;;
 +    qemu-ifdown) tun_down $1;;
 +    *) echo fail; exit 1;;
 +  esac
 +
 +===== Using Qemu from Unprivileged Users =====
 +Using root for doing your everyday tasks is commonly discouraged so let's see how we can work around using qemu from unprivileged users.
 +Some say that it's sufficient to give sudo execution on /etc/qemi-if* but that not really enough because qemu-system-* needs to run as root or it will not be able to create the taps and access other resources. Although it is technically possible to give an unprivileged user sufficient privileges to execute correctly qemu-system-* emulators it is much easier to give users the sudo right to run qemu-system-* as privileged user.
 +
 +  User_Alias QEMUERS = al, john, jack
 +  
 +  Cmnd_Alias QEMUCMD = /usr/bin/qemu-system-*
 +  
 +  QEMUERS ALL=(ALL) NOPASSWD: QEMU
 +
 +This would be sufficient to run the VMs as any of the unprivileged users in QEMUERS user alias (al, john, jack) but the management script would need to run sudo qemu-system-* .... this is easy to obtain:
 +
 +  [ $(/usr/bin/id -u) -ne 0 ] && CMD="sudo qemu-system-.... " || CMD="qemu-system-...."
 +  eval $(echo "$CMD &")
 +
 +Alternatively you could give privileges to execute the management script as root.
 +===== Examples =====
 +Here are examples of the dialogs that user would see wile creating, starting, stopping and deleting a VM.
 +q
 ==== VM Creation Example ==== ==== VM Creation Example ====
  
Line 520: Line 565:
   |               <  OK  >        <Cancel>                  |   |               <  OK  >        <Cancel>                  |
   +---------------------------------------------------------+   +---------------------------------------------------------+
 +
 +==== Starting and Stopping VM Examples ====
 +  +----------------------------+
 +  | Choose Virtual Machine     |
 +  | +------------------------+ |
 +  | |   ( ) ORACLE      1    | |
 +  | |   ( ) ORACLE2        | |
 +  | |   ( ) armedslack  3    | |
 +  | |   ( ) freenas        | |
 +  | |   ( ) test        5    | |
 +  | |   (*) test2          | |
 +  | +------------------------+ |
 +  +----------------------------+
 +  |   <  OK  >  <Cancel>       |
 +  +----------------------------+
 +
 +After the command for starting the VM is executed 2 more things take place:
 +  - a quickstart.sh script (containing the required commands to start the VM) is created in the VM folder
 +  - the qemu-system-* command to start the VM is echoed back on the console
 +
 +Stopping a virtual machine has the same dialog.
  
 ==== VM Delete Example ==== ==== VM Delete Example ====
Line 556: Line 622:
  
 ====== Sources ====== ====== Sources ======
 +I've a blog entry on LQ where I talk a little more extensively on minimizing the code in bash scripts.
 +[[http://www.linuxquestions.org/questions/blog/louigi600-808242/minimize-the-amount-of-code-in-your-bash-scripts-37137/|Minimize the amount of code in your bash scripts ]]
  
 <!-- If you are copying information from another source, then specify that source --> <!-- If you are copying information from another source, then specify that source -->

In Other Languages
QR Code
QR Code howtos:emulators:helper_script_for_managing_qemu_virtual_machines (generated for current page)