I wanted to connect to an OpenVPN server from my Ubuntu on a stick.

There are different clients but the one I liked was the Linux Network-Manager GUI for OpenVPN. To install it you need to enable the Universe repository first:

sudo add-apt-repository universe

sudo apt-get update

and then type:

sudo apt-get install network-manager-openvpn

After the installation completes add a VPN connection by clicking on the wireless icon from the top menu and selecting VPN Connections -> Configure VPN. The network connections dialog pops up:

Click the Add button and from the drop-down either select OpenVPN or Import a saved VPN connection… and then click Create button:

In the first case you have to manually configure everything like the location of the user certificate, the private key, etc.

The second option allows you to import an existing configuration file that already have all these settings defined. What I did was to put the configuration file with all the files referenced by it in one folder and import the configuration file.

You can also configure the existing network connections (both wired and Wi-Fi) to automatically connect to your VPN. This is done in the General tab.