Updates
- Updated 07.03.2019 Removed all outdated information, almost everything
- Full changelog script
- Updates
- Virtualization With KVM On Ubuntu 18.04 LTS
- Modifying KVM (qemu-kvm) settings for malware analysis
- QEMU/KVM and Memory Forensic (Volatility):
- Assign static guest IP addresses using DHCP on the virtual machine
- virt-sparsify - Make a virtual machine disk sparse
- Virt-manager for mac
- I still can’t connect to a remote URI, why?
- Run virt-manager with either the –debug or –no-fork option to get password prompt via the CLI.
- Why can’t I connect to a local URI (e.g., qemu:///system)?
- Everything was working yesterday, but it’s not working today, can you help?
- Xen/Kvm manual from OpenSuse
- Some compilation problem
Virtualization With KVM On Ubuntu 18.04 LTS
- Qemu Documentation - will be installed at the end, in hardening part
Instalation for best performance is done using this script
The GUI - Virt-Manager
- GUI will be installed by script
KVM usage
First check if your CPU supports hardware virtualization - if this is the case, the command
-
To check if KVM has successfully been installed, run
virsh -c qemu:///system listORvirsh list
-
It should display something like this:
root@server1:~# virsh -c qemu:///system list
Id Name State
----------------------------------
-
Before we start our first virtual machine, I recommend to reboot the system:
reboot
-
If you don’t do this, you might get an error like open /dev/kvm: Permission denied in the virtual machine logs in the /var/log/libvirt/qemu/ directory.
-
shows all VMs,
virsh # list --all Id Name State ---------------------------------- - vm1 shut off - vm2 shut off
-
Before you start a new VM for the first time, you must define it from its xml file (located in the /etc/libvirt/qemu/ directory):
define /etc/libvirt/qemu/vm1.xml
-
Please note that whenever you modify the VM’s xml file in /etc/libvirt/qemu/, you must run the define command again!
-
Now you can start the VM:
start vm1
-
After a few moments, you should be able to connect to the VM with an SSH client such as PuTTY; log in with the default username and password. After the first login you will be prompted to change the password.
-
should now show the VM as running:
virsh # list Id Name State ---------------------------------- 1 vm1 running
-
To stop a VM, run
shutdown vm1
-
To immediately stop it (i.e., pull the power plug), run
destroy vm1
-
Suspend a VM:
suspend vm1
-
Resume a VM:
resume vm1
-
To leave the virtual shell..
quit
KVM-QEMU, QCOW2, QEMU-IMG and Snapshots: Source1 | Source2
There are several different types of snapshots possible. Some idea on that:
-
Internal snapshot: A type of snapshot, where a single QCOW2 file will hold both the ‘saved state’ and the ‘delta’ since that saved point. ‘Internal snapshots’ are very handy because it’s only a single file where all the snapshot info. is captured, and easy to copy/move around the machines.
-
External snapshot: Here, the ‘original qcow2 file’ will be in a ‘read-only’ saved state, and the new qcow2 file(which will be generated once snapshot is created) will be the delta for the changes. So, all the changes will now be written to this delta file. ‘External Snapshots’ are useful for performing backups. Also, external snapshot creates a qcow2 file with the original file as its backing image, and the backing file can be /read/ in parallel with the running qemu.
-
VM State: This will save the guest/domain state to a file. So, if you take a snapshot including VM state, we can then shut off that guest and use the freed up memory for other purposes on the host or for other guests. Internally this calls qemu monitor’s ‘savevm’ command. Note that this only takes care of VM state(and not disk snapshot).
-
Before creating a snapshot, we need to create a snapshot xml file with 2 simple elements (name and description) if you need sensible name for the snapshot. Note that only these two fields are user settable. Rest of the info. will be filled by Libvirt.
$ cat /var/tmp/snap1-f15guest.xml
<domainsnapshot>
<name>snaphot_name </name>
<description>SomeDescription</description>
</domainsnapshot>
Commands
-
List of all available snapshots
virsh snapshot-list NAME_HERE
-
To revert to a particular snapshot
virsh snapshot-revert name_vm snapshotname
-
Taking a snapshot while the ‘guest’ is running live
virsh snapshot-create name_vm /var/tmp/snap1-f15guest.xml
-
To take snapshot of disk image too
qemu-img create -f qcow2 -b centos-cleaninstall.img snapshot.img
-
Dump some basic information about the image file. Includes the size on disk of the file, the name of any backing file referenced and a list of the snapshots available.
qemu-img info <imagename>
-
Create a simple QCOW2 image file. The disk space used for this file will be relatively small but the maximim storage capacity will be max-storage.
qemu-img create -f qcow2 <imagename> <max-storage> -
Create a QCOW2 image file named imagename2 which is based on a backing image, imagename1. The new image file will reference the backing file. Any clusters written by the VM will be written to imagename2 so that the backing file remains unchanged. The backing file can be referenced by many future images but must not be changed by any of them. Warning: If any process modifies the backing file the image file(s) will be corrupted.
qemu-img create -b <imagename1> -f qcow2 -l <imagename2> -
List all the snapshots in the specified imagename file.
qemu-img snapshot -l <imagename>
-
Create a snapshot and name it snapshot-name. This snapshot is a simple picture of the VM image state at the time the snapshot is created.
qemu-img snapshot -c <snapshot-name> <imagename>
-
Apply a snapshot named snapshot-name. This function simply restores the clusters that were saved when the snapshot, snapshot-name, was created. It has the effect of returning the VM image to the state it was in at that time.
- qemu-img snapshot -a
- qemu-img snapshot -a
-
Delete the snapshot named snapshot-name from the specified image file, imagename. Snapshots can gobble-up a significant amount of disk space. The delete command does not actually release any disk space allocated to the image file but it does release the associated clusters - effectively making them available to the VM for future storage.
qemu-img snapshot -d <snapshot-name> <imagename>
-
The convert command, when converting to and from the same QCOW2 format, acts as a copy command that only copies the current state of the VM image to the output file. The -p option displays progress information during the copy operation - which is often quite time consuming. The output file, imagename2, will contain all the clusters of a backing file that were referenced in the original image, imagename1. It will not copy any snapshot information. This has the effect of creating a standalone image with no references to any backing image.
qemu-img convert -p -f qcow2 <imagename1> -O qcow2 <imagename2>
The QEMU image snapshot create command is, as you might expect, also simple:
-
qemu-img snapshot -c <snapshot-name> <imagename>- The snapshot option tells qemu-img that we want to work with snapshots
- The -c option tells qemu-img that we want to create a snapshot of the current VM image state.
- The snapshot-name is the ID that we want to assign to this state of the VM image.
- The imagename is the disk file name for the VM image with which we are working.
-
Putting it all together: Open a shell and change to the directory in which you have your image files. In the following example I am working on the same VM image that I created above and now I’m saving the image state using the name base-win7pro-winupdates-ie9.
-
As you can see there is no output from the snapshot create operation so I follow that up with an info request. Take a look (though it’s slightly edited.)
-
qemu-img snapshot -c base-win7pro-winupdates-ie9 win7demo-kvm.qcow2 qemu-img info win7demo-kvm.qcow2
Applying different Snapshots:
-
qemu-img snapshot -a <snapshot-name> <imagename>- The snapshot option tells qemu-img that we want to work with snapshots
- The -a option tells qemu-img that we want to apply a previously created snapshot of the VM image state - ie: we want to apply that snapshot so that it becomes the current image state.
- The snapshot-name is the ID of the previously created snapshot.
- The imagename is the disk file name for the VM image with which we are working.
Converting image formats
-
This example will convert a raw image file named centos7.img to a qcow2 image file.
qemu-img convert -f raw -O qcow2 centos7.img centos7.qcow2
-
Run the following command to convert a VMDK image file to a raw image file.
qemu-img convert -f vmdk -O raw centos7.vmdk centos7.img
-
Run the following command to convert a VMDK image file to a qcow2 image file.
qemu-img convert -f vmdk -O qcow2 centos7.vmdk centos7.qcow2
-
VBoxManage: VDI (VirtualBox) to raw
VBoxManage clonehd ~/VirtualBox\ VMs/fedora21.vdi fedora21.img --format raw
-
.OVA to QCOW2
tar xvf MyAppliance.ovaqemu-img convert -O qcow2 MyAppliance-disk1.vmdk MyAppliance.qcow2
Change default image storage
virsh pool-dumpxml default > pool.xml edit pool.xml # with new name and path virsh pool-create pool.xml virsh pool-refresh name
Enabling clipboard transfer (Copy + Paste)
- To enable copy and paste between virtual machines and host in RedHat KVM do the followings:
- Shutdown you machine if it is turned on.
- Click on “Show virtual hardware details“.
- Choose the “Display VNC” hardware and set the type to “Spice“
- Click Apply and choose YES to the “You are switching graphics type to spice, would you like to add Spice agent channels?” question.
- Now, select Video and set the Model to “qxl” and click apply.
- Turn on your machine and login.
- Run the followings as super user:
sudo apt-get install spice-vdagent python-spice-client-gtk spice-client spice-client-gtk
- Then for all clients you need install guest software
- Reboot the guest, and voila it works.
File draging/sharing
- ToDo
Modifying KVM (qemu-kvm) settings for malware analysis
Some parts of this block is from: blog.prowling.nu
-
I have found that using libvirt and virsh edit is a simple way to change the settings for the guest OS.
-
General guidelines DON’T RUN VM BEFORE ALL PATCHING: 0. sh QEMU and seobios correct installer 1. Change mac address 2. Add CpuID bit vm detect patching and ACPI tables 3. Add SeoBios info 4. Copy host cpu info(Depends on the host)
To edit VM conf -> virsh edit vm_name
0. Use this kvm-qemu.sh to install all software and REPLACE <WOOT>
Script: sudo ./kvm-qemu.sh --help
1. Change mac address
<interface type='network'>
<mac address='xx:xx:xx:xx:xx:xx'/>
<source network='default'/>
<model type='rtl8139'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
</interface>
2 Add CpuID bit vm detect patching and ACPI tables
- KVM by default will pass through a feature flag, viewable in ECX as the 31st bit after executing the CPUID instruction with EAX set to 1. Some malware will use this unprivileged instruction to detect its execution in a VM. One way to avoid this is to modify your VM definition as follows: find the following line:
<domain type='kvm'>
Change it to:
<domain type='kvm' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'>
Then within the domain element, add the following:
<qemu:commandline>
<qemu:arg value='-cpu'/>
<qemu:arg value='<REPLACEME>,-hypervisor,kvm=off'/>
<qemu:arg value='-L'/>
<qemu:arg value='/usr/share/qemu/bios.bin'/>
</qemu:commandline>
-
Instead of using “host”, you can also choose a number of other CPU models from the list displayed with the “qemu-system-i386 -cpu help” command (SandyBridge, Haswell, etc). So replace REPLACEME with your cpu which you selected.
-
Now test your virtual machine, if everything works prepare it for snapshotting while running Cuckoo’s agent. This means the virtual machine needs to be running while you are taking the snapshot. Then you can shut it down. You can finally take a snapshot with the following command:
3 Change BIOS information
- Start by retrieving the dmidecode information for your host.
<os>
<smbios mode='sysinfo'/>
...
</os>
<sysinfo type='smbios'>
<bios>
<entry name='vendor'>XXXX</entry>
<entry name='version'>XXXXXX</entry>
<entry name='date'>XXXXX</entry>
<entry name='release'>XXXXX</entry>
</bios>
<system>
<entry name='manufacturer'>XXXXX</entry>
<entry name='product'>XXXXX</entry>
<entry name='version'>XXXXX</entry>
<entry name='serial'>XXXXX</entry>
<entry name='uuid'>XXXXXXXX</entry> <-- This values has to be the same as the other UUID variable found in the xml file
<entry name='sku'>XXXXXX</entry>
<entry name='family'>XXXXXX</entry>
</system>
</sysinfo>
Example
<os>
<smbios mode='sysinfo'/>
...
</os>
<sysinfo type='smbios'>
<bios>
<entry name='vendor'>American Megatrends Inc.</entry>
<entry name='version'>1101</entry>
<entry name='date'>02/04/2013</entry>
<entry name='release'>1.71</entry>
</bios>
<system>
<entry name='manufacturer'>System manufacturer</entry>
<entry name='product'>System manufacturer</entry>
<entry name='version'>System Version</entry>
<entry name='serial'>System Serial Number</entry>
<entry name='uuid'>21804FA0-D7DA-11DD-B21C-08606E6814BB</entry>
<entry name='sku'>SKU</entry>
<entry name='family'>To be filled by O.E.M</entry>
</system>
</sysinfo>
Create snapshot
$ virsh snapshot-create "<Name of VM>"
-
Having multiple snapshots can cause errors.
ERROR: No snapshot found for virtual machine VM-Name -
VM snapshots can be managed using the following commands.
$ virsh snapshot-list "VM-Name" $ virsh snapshot-delete "VM-Name" 1234567890
QEMU/KVM and Memory Forensic (Volatility):
How to get ram memory dump:
Executing a virsh dump command sends a request to dump the core of a guest virtual machine to a file so errors in the virtual machine can be diagnosed. Running this command may require you to manually ensure proper permissions on file and path specified by the argument corefilepath. The virsh dump command is similar to a coredump (or the crash utility). To create the virsh dump file, run:
#virsh dump <domain> <corefilepath> [--bypass-cache] { [--live] | [--crash] | [--reset] } [--verbose] [--memory-only]
-
While the domain (guest virtual machine domain name) and corefilepath (location of the newly created core dump file) are mandatory, the following arguments are optional:
- –live creates a dump file on a running machine and doesn’t pause it.
- –crash stops the guest virtual machine and generates the dump file. The main difference is that the guest virtual machine will not be listed as Stopped, with the reason as Crashed. Note that in virt-manager the status will be listed as Paused.
- –reset will reset the guest virtual machine following a successful dump. Note, these three switches are mutually exclusive.
- –bypass-cache uses O_DIRECT to bypass the file system cache.
- –memory-only the dump file will be saved as an elf file, and will only include domain’s memory and cpu common register value. This option is very useful if the domain uses host devices directly.
- –verbose displays the progress of the dump
-
The entire dump process may be monitored using virsh domjobinfo command and can be canceled by running virsh domjobabort.
Assign static guest IP addresses using DHCP on the virtual machine
View the current dnsmasq DHCP configuration
-
Type the following command to list networks
# virsh net-list
-
Sample outputs:
Name State Autostart Persistent ---------------------------------------------------------- default active yes yes
- To see the default network information, enter:
# virsh net-dumpxml default
- Sample outputs:
<network connections='2'>
<name>default</name>
<uuid>e346291e-f86b-4f2f-a16e-654136441805</uuid>
<forward mode='nat'>
<nat>
<port start='1024' end='65535'/>
</nat>
</forward>
<bridge name='virbr0' stp='on' delay='0'/>
<mac address='52:54:00:12:fe:35'/>
<ip address='192.168.122.1' netmask='255.255.255.0'>
<dhcp>
<range start='192.168.122.100' end='192.168.122.254'/>
</dhcp>
</ip>
</network>
- The DHCP range is between 192.168.122.100 and 192.168.122.254.
How to configure static guest IP addresses on the VM host
- First find out your guest VM’s MAC addresses, enter:
# virsh dumpxml {VM-NAME-HERE} | grep -i '<mac'
# virsh dumpxml xenial | grep -i '<mac'
-
Sample outputs:
<mac address='52:54:00:4c:40:1c'/>
-
Please note down the MAC addresses of the xenial VM that you want to assign static IP addresses.
Edit the default network
-
Type the following command:
# virsh net-edit default
-
Find the following section:
<dhcp>
<range start='192.168.122.100' end='192.168.122.254'/>
-
Append the static IP as follows after range:
<host mac='52:54:00:4c:40:1c' name='xenial' ip='192.168.122.4'/>
-
Where,
mac='52:54:00:4c:40:1c' – VMs mac address
name='xenial' – VMs name.
ip='192.168.122.4' – VMs static IP.
- Here is my complete file with three static DHCP entries for three VMs:
<network>
<name>default</name>
<uuid>e346291e-f86b-4f2f-a16e-654136441805</uuid>
<forward mode='nat'/>
<bridge name='virbr0' stp='on' delay='0'/>
<mac address='52:54:00:12:fe:35'/>
<ip address='192.168.122.1' netmask='255.255.255.0'>
<dhcp>
<range start='192.168.122.100' end='192.168.122.254'/>
<host mac='52:54:00:a0:cc:19' name='centos7' ip='192.168.122.2'/>
<host mac='52:54:00:f7:a1:c8' name='puffy' ip='192.168.122.3'/>
<host mac='52:54:00:4c:40:1c' name='xenial' ip='192.168.122.4'/>
</dhcp>
</ip>
</network>
- Restart DHCP service:
# virsh net-destroy default # virsh net-start default
- Sample outputs:
Network default destroyed
Network default started
- If you are running the guest/VM called xenial shutdown it:
# virsh shutdown xenial # /etc/init.d/libvirt-bin restart # virsh start xenial # ping -a 192.168.122.4
- Sample outputs:
PING 192.168.122.4 (192.168.122.4) 56(84) bytes of data. 64 bytes from 192.168.122.4: icmp_seq=1 ttl=64 time=0.518 ms 64 bytes from 192.168.122.4: icmp_seq=2 ttl=64 time=0.202 ms 64 bytes from 192.168.122.4: icmp_seq=3 ttl=64 time=0.327 ms ^C --- 192.168.122.4 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 1999ms rtt min/avg/max/mdev = 0.202/0.349/0.518/0.129 ms
- Each time the guest or VM called xenial comes online (or rebooted for the kernel update) it will get 192.168.122.4 as static IP address by dnsmasq DHCP server.
virt-sparsify - Make a virtual machine disk sparse
sudo apt install libguestfs-tools
Examples
-
Typical usage is:
virt-sparsify indisk outdisk- which copies indisk to outdisk, making the output sparse. outdisk is created, or overwritten if it already exists. The format of the input disk is detected (eg. qcow2) and the same format is used for the output disk.
-
To convert between formats, use the –convert option:
virt-sparsify disk.raw --convert qcow2 disk.qcow2- Virt-sparsify tries to zero and sparsify free space on every filesystem it can find within the source disk image. You can get it to ignore (don’t zero free space on) certain filesystems by doing:
-
To sparse space of disk image and ignore filesystem
virt-sparsify --ignore /dev/sda1 indisk outdisk
Virt-manager for mac
brew tap jeffreywildman/homebrew-virt-manager brew cask install xquartz brew install virt-manager virt-viewer virt-manager -c qemu+ssh://user@libvirthost/system?socket=/var/run/libvirt/libvirt-sock virt-viewer -c qemu+ssh://user@libvirthost/system?socket=/var/run/libvirt/libvirt-sock
I still can’t connect to a remote URI, why?
- This formula for virt-manager does not include the openssh-askpass dependency and does not prompt for passwords in a popup window. Here are two workarounds:
Run virt-manager with either the –debug or –no-fork option to get password prompt via the CLI.
- Set up SSH keys between your local and remote system to avoid the prompt.
Why can’t I connect to a local URI (e.g., qemu:///system)?
- I’ve not yet tested virt-manager against any local URIs/hypervisors. If you get virt-manager working with a local hypervisor and needed to take any special steps, feel free to share the details.
Everything was working yesterday, but it’s not working today, can you help?
- If virt-manager or its dependencies have been upgraded recently (brew upgrade), it’s possible that a reinstall may fix the issue (see #39).
Xen/Kvm manual from OpenSuse
Some compilation problem
- I saw first time in few year problem with compilation, where before it worked just fine. I had installed all libz related packages and still had problem with this.
no dependency information found for /usr/local/lib/libz.so.1
- SOLUTION I dislike this solution but that was uniq way to get it working just fine.
sudo sed -i 's/my $ignore_missing_info = 0;/my $ignore_missing_info = 1;/g /usr/bin/dpkg-shlibdeps- another options from here not worked for me
+15.40.15.png)
+15.44.44.png)
+13.55.59.png)
+14.00.59.png)
