Virtualization VGA Passthrough Concepts

VGA Passthrough Concepts and Terminology

I want to run Windows CAD on my Linux box without dual-booting.  Realistically, that means running a VM with GPU passthrough.  Under the current state of things, that means a separate VGA monitor.  That in turn means passing my USB keyboard and mouse back and forth between the VM and the host.  That's complicated.

**Update 2018:** The Looking-Glass project provides low-latency screen transfer.  Think of it as no-lag VNC.  Alterate UI no longer necessary, although mouse integration is still sketchy.

**Update 2017**: Used a modern PCI-E Nvidia adapter, and it mostly just works with LibVirt and OVMF/UEFI.  There are a few hoops to jump through to fix the Code 43.

• This  tutorial worked, but doesn't run under libvirt, so it doesn't support using virsh to automate a hot USB passthrough
• Adding the qemu arguments to my libvirt definition didn't work at all, but merits further exploration
• Adding the PCI device in virt-manager resulted in the pci device being present, but inoperable with a Code 10.  However, it actually started working after I force-installed the NVidia driver.  I don't seem to be able to dual-head with the virtual display adapter though.

Here's the (editorialized) terminology list I wish I'd had before setting off on this journey.

Hypervisor - The VM software.  Think of type 2 as emulation software that runs in your OS, perhaps using hardware acceleration (EG, VirtualBox).  Type 1 is like an OS container that runs on the bare metal (eg Xen and KVM).  Qemu-KVM is essentially type 2, but uses the KVM kernel modules to get many of the advantages of a type 1.

VirtualBox - Oracle's type 2 hypervisor.  Runs on Windows, OSX, and Linux.  I could not get VGA passthrough working on it.

Qemu - an open-source hypervisor.  Most of the VGA passthrough guides for it walk you through constructing a very hairy Qemu commandline.  That's unfortunate, because it precludes using the Libvirt management gui (below).

Passthrough - providing the VM with direct access to hardware.  In our case, this is a PCI, PCIe, or USB device.

VGA Passthrough - Providing the VM with hardware access to a display device.

GPU Passthrough - Mostly (incorrectly) used synonymously with VGA passthrough.  However, it also includes the concept of passthrough of a GPU that is *not* the display device, something that isn't possible on any of the Linux Hypervisors AFAIK.

Composing - This is the best word I have seen for the concept of passing through a rendering GPU which is something other than the display device.  In theory, you might passthrough a PCIe GPU for rendering your CAD model, and compose the display to your virtual display device.  None of the Linux hypervisors I've looked at can handle this.

Libvirt - a virtualization API, providing a consistent management interface for pretty much every linux hypervisor out there except VirtualBox.

USB passthrough - super easy on VirtualBox.  Super easy on Qemu, if you define it in the commandline. 

USB passthrough hotplug - This is built into the gui in Virtualbox.  Impossible in Qemu, without libvirt.  If you want to hotplug a usb device to a running libvirt VM, you need to use virsh, basically a commandline front end for libvirt.  I never found a way to do it without libvirt  (meaning if you ran your qemu VM from the commandline, you're out of luck).

http://rolandtapken.de/blog/2011-04/how-auto-hotplug-usb-devices-libvirt-vms-update-1

HMI/SCADA software for xLogic PLCs

ProView

I haven't actually used this yet, but it looks very promising.  Totally open-source, has process editors and multi-trends, and has build packages specific for linux, android, and raspberry pi.

RapidSCADA

This is a .NET/mono-based web interface.  Partly open-source, partly some (very cheap) licensed modules, it's relatively simple to follow their cookbook to get a bare-bones setup working.

In spite of my criticisms, of the options I've tried, it's the best combination of flexibility and easiness.

Criticism:

It's built with .NET, but in theory the service (but not the builder / config tools) should run on linux (and Raspberry Pi!) with the mono libraries.  I'm having some difficulty, but it might just be that my mono is 3.10 instead of 3.28.

The user interface is not very friendly.  I found myself having to repeat the same operation dozens of times, when I really wanted to multi-select and do the operation on all of them simultaneously.

The table creator feature is awesome, but the graphic editor is really basic (and does not come with a symbol library AFAIK).  It's really not a useful HMI without a multi-trend (which you have to pay for).

It's also not really SCADA without the automation plugin (which you have to pay for).

Overview:

The package consists of several distinct modules:

• Server - basically a database with some data flow functionality.  Has an "admin" gui, which does not run on linux.
• Comm - basically a collection of drivers, it interfaces between the Server and your devices.  Divided into a service and a gui (which always starts minimized to the tray!).
• Webservice - provides a user-interface to the server.  Consists of:
• Table: provides hourly summaries of all of your input points
• Chart: provides a graph of one point
• ChartPro: provides a graph of several points (paid)
• Schema: A graphic that displays differently based on input values.
• Table / Schema editors: creates interface elements for the web service.

Gotchas:

• The communicator front end always opens minimized to the tray.
• Follow the Modbus TCP recipe precisely, without deleting any extraneous points.  Do not change anything until that works. 
• Change the "chart gap" setting on the web service configuration page to 120s, or at least play with it until there is no gap between your points.

xLogic EasySCADA

XLogic supplies this development environment for free.

I don't recommend it, there's no way to test/debug without compiling and installing.   Also, the registration key needs to be generated again for every client computer, and that gets old really fast.

Overview

The basic intent of the software is to create a Windows HMI application that connects to your PLC network.  It appears to support USB RS485 and Modbus RTU, but I only ever used it to connect directly to my EXM's IP interface.

The only real advantage it has is you don't have to look up the Modbus address of the register you're interested in, you can just use xLogic's (semi-) descriptive name.

Notes

• You have to install their (bundled) Mysql6 and connector first.    
• When compiling, the "publisher id" field must be numeric.  
• Save the publisher ID.  You'll need it again later when you generate a registration key to install your HMI application.

OpenScada

First off, don't be fooled.  The openscada.org project has been dead for a couple of years.  All active development is on oscada.org, and appears to mostly be the work of one incredibly prolific developer.

If I'd persevered, I would've ended up with a SCADA server that would run on a Linux-based Raspberry Pi, with a touchscreen interface AND a web interface.  That's pretty cool.  The software appears incredibly versatile.  It is very modular, and there are modules for pretty much any kind of input or output device I've ever heard of.

Unfortunately, that versatility comes at the price of extreme complexity.  It took me most of two days to get a very basic HMI up showing a single modbus register, and I didn't really feel like I understood what I was doing.  It looked like every single field I wanted to add to the HMI was going to require a half-hour of tedius re-entry of long address strings.

Also unfortunately, the huge reams of very wordy documentation appear to have been written by a garrulous Ukranian, google-translated to Russian, and then to English (possibly by way of German).  I really didn't understand the intent of what I was reading.

AdvancedHMI

At the opposite end of the spectrum from OpenScada, AdvancedHMI makes heavy use of Visual Studio / .net / visual basic, and is very drag-and-drop.  It is almost entirely undocumented, but there's an active user community, and my only question was answered within a couple of hours.

The end result will probably only run on a windows platform, but it's so easy to get working that it's really hard to hate.

Gotchas:

This may  actually be a problem with the xLogic PLCs: register 1024 on xLogic, I had to instruct AdvancedHMI to access 41025.  With some help from the forums, I got this by

• prepending a 4 -- Modbus addresses starting in the 40,000 range  are the ones which are accessed using modbus command 0x3.
• adding 1 -- xLogic's register address reference is 0-based, but apparently all modbus addressing is 1-based.

Energy Monitor front ends

I have an ongoing energy monitoring project -- I'm archiving meter consumption data for a couple of clients, and would like to make it more usable.  I'm currently in the process of evaluating visualization front-ends.

Energy Star Portfolio Manager

Provides a REST api, and no other way to bulk upload data.  The only way to upload CSV data is to paste it into a preformatted excel spreadsheet.

Does not provide any useful visualization with anything finer grained than monthly billing data.

It may be useful to upload monthly electricity, gas, and water data if I have it all in one place.  If I can find a simple example, I'll take a crack at it.

OpenEnergyMonitor

This project has a lot of promise.  It consists of some hardware for sale, pre-configured to talk to the project's open-source PHP server software.  You have your choice of hosting the data on your own server (which has some hiccups), or for no charge on the project's server (which didn't immediately work for me).

My server

I was able to get the software installed and my data imported on my linux workstation with a minimum of fuss.   Some of the features didn't immediately work because redis.so and dio.so aren't installed by default and have no ubuntu packages.  Pretty simple -- `sudo pecl install dio-beta` worked just fine.

Project Server

Unfortunately, the same upload tools would not work to upload to the project server (the uploads failed silently with a success message!), and none of the live examples from the project website are currently working.

Conclusions

Most of the project's energy seems to be (understandably) going into making their for-profit products work with the software, so there isn't a lot of effort left over to fully document the API for edge-cases like my own. Asking on the (very active) community forum returned suggestions that the system was not designed for displaying historical data.  I belive that, so I'll be looking elsewhere.

Processing.js

The great-granddaddy of data visualization, this comes down to rolling my own data visualization website using some pretty awesome tools.  Ultimately, I'm not a front-end guy, so making this into something that can be customer-facing will take a lot more work than I want to invest.