Archive

Author Archive

Install R from source: a simple way to compile R from source and use it in RStudio or on servers without X

Sometimes we need to use special versions of R (e.g. the latest version of R is not available in the repositories of the machine we need to use). This post describes a simple way of compiling R from sources and using it in RStudio or on servers without X.

Get latest R source

Get the source of your desired R version from https://www.r-project.org/. You want to obtain a file named R.x.y.z.tar from there and untar it using tar -xf R-x.y.z.tar.gz. Change into the untar-ed folder then.

More details on this are available at https://cran.r-project.org/doc/manuals/r-release/R-admin.html#Getting-and-unpacking-the-sources

Build R from source for RStudio

Built R using the following commands. Be aware that there are a number of prerequisites to this compilation, like having gcc and g++ installed.

./configure --enable-R-shlib
make
make check
make check-all

 

Passing the checks creates the R and Rscript frontends in ./bin/ — those are what you most likely need. You can call these directly, link to them, or use them in RStudio (see below). If you forgot to add the –enable-R-shlib parameter to configure you need to delete and re-extract the tar before redoing the process — otherwise make will fail.

More details on this are available at https://cran.r-project.org/doc/manuals/r-release/R-admin.html#Simple-compilation

Define the R to use with RStudio

RStudio uses the R defined by which R. Hence we can add the following to the ~/.profile file to define which R is used by RStudio:

# export RSTUDIO_WHICH_R=/path/to/your/R # this is what RStudio usually uses
export RSTUDIO_WHICH_R=/usr/local/bin/R  # this is what we use
. ~/.profile # reloads the profile without requiring us to logout+login again

More details on this are available at https://support.rstudio.com/hc/en-us/articles/200486138-Using-Different-Versions-of-R

Build R for environments without X/without Java

Do the same steps as above, but use this configure instead:

./configure --with-x=no --disable-java

More details on this are available at https://cran.r-project.org/doc/manuals/r-release/R-admin.html#Essential-programs-and-libraries

Hint for “package not available for R version x.z.y…” errors

If package installations fail because your repositories don’t contain the required SW, try the Berkeley mirror: from our experience, they host lots of packages in many versions. For example:

install.packages("ggplot2", repos="http://cran.cnr.berkeley.edu")

Alternatively, the URL to the source archive can be specified directly:

packageurl <- "http://cran.r-project.org/src/contrib/Archive/XXXX/XXXX_A.B.C.tar.gz"
install.packages(packageurl, contriburl=NULL, type="source")

More details on this are available at http://stackoverflow.com/questions/17998543/what-to-do-when-a-package-is-not-available-for-our-r-version

Yet another option is to use devtools to fetch code e.g. directly from GitHub:

devtools::install_github("hadley/lineprof")

More details on this example can be found at http://adv-r.had.co.nz/Profiling.html

 

Categories: Data Analysis Tags: , , , , , , ,

Rhythmbox is silent: reset pulseaudio configuration

Rhythmbox is silent?

After playing around with different Linux Shells/UIs/window managers (including i3, Gnome Shell, etc) and some frequent restarts, I noticed that for some reason my music player Rhythmbox had stopped playing any sounds. Though it was still “playing” songs, no sounds were actually made by speakers, headphones, etc. In contrast, all other sounds were OK, including OS sounds like info/error messages, or other players like VLC. Changing sound setting, volumes, as well as changing audio devices did not have any effect on this.

Resetting pulseaudio configuration

Turns out that the pulseaudio configuration was messed up. The solution to this problem is to move/delete the old configuration, then let it be recreated (the original problem description and solution to this is here):

mv ~/.config/pulse/ ~/.config/pulse_old/ # move/delete the old config
pulseaudio -k # restart pulseaudio

After doing this and restarting Rhythmbox its sound should be back to normal, – though you might need to restart the OS if you run into problems with external speakers/headphones.

i3 window manager: quick setup and configuration

February 5, 2017 Leave a comment

i3_window_manager_screenshot

i3 example [Wikipedia]

The i3 window manager is a tiling window manager: it enables power users to operate/arrange/manage application windows in a very fast way in tiles, without any requirements to use a mouse. Alex Booker provides an excellent introduction and overview to the i3 window manager, which is divided into 3 videos: general introduction, configuration, and (visual) fine tuning. In this post we summarize the i3 configuration we use atm for a quick lookup (which is partially based on Alex Booker’s videos).

Installation


sudo apt-get install i3

Logout, then login to i3. The i3 configuration wizard starts automatically if there is no i3 configuration file yet (we recommend using Super as the mod key at this point, because Alt is used by a huge amount of applications for other tasks). You can always remove the i3 config file at ~/.config/i3/config and start the wizard again using i3-config-wizard.

Basic usage

We strongly recommend to go through all the shortcuts in the official i3 reference card (it just takes a moment and is the core of how you will operate i3 afterwards). Be aware that if you are using an alternative keyboard layout, i3 automatically detects which keys are located at the positions that i3 would normally use – so you will be able to use the same physical keys, even if you are using a completely different layout.

After being aware of the basic usage of i3, skimming through the official i3 user’s guide will make sense. This should highlight the possibilities with i3 (in both how to operate i3 and how to configure it), which will make later configuration tasks easier by far.

i3 config file

The main i3 config is located in ~/.config/i3/conf. This file is created by i3-config-wizard at first and can be modified as wished afterwards.

Be aware that changing any i3 config at least requires i3 to be reloaded (“restarted in place”) with Mod+Shift+R. Some configuration also require the underlying application that is started/controlled by i3 (e.g. compton) to be restarted. If i3 restarts those applications on each reload depends on its configuration (that’s the difference between exec and exec_always in the config file). If the underlying application was started with exec, restarting the application by hand or logging out/loging back in is required.

(Re)starting i3 from command line

To start i3 from a native tty command line:

DISPLAY=:0 # or :1 etc...
export DISPLAY
# killall i3 # required in case i3 is running
i3 -c .config/i3/config

To instead restart a running i3 from a native tty command line:

i3-msg reload
i3-msg restart

Media buttons etc.

Most likely i3 does not recognize your media buttons correctly from the beginning. You at first need playerctl to enable control over (parts) of the media buttons, of which you can get different packages (including .deb) here. The i3 FAQ then provides a config snippet that can be copied (appended) to your i3 config file to map keys to the media buttons. Be aware that the number of the pulse audio device may be different (in the script it’s 0, but for others it might e.g. be 1). If e.g. the buttons cause the currently focused window to flicker, but the volume does not change, this might be an indication for using the wrong device.

Set wallpaper

You need e.g. feh to set a wallpaper in i3:


sudo apt-get install feh

By adding the following to your i3 config file you always set the wallpaper when i3 is started/reloaded:


exec_always feh --bg-scale /path/to/your/wallpaper.jpg

For different wallpaper scaling options, check out the man page of feh (it provides e.g. –bg-center, –bg-fill, –bg-max, –bg-scale, –bg-tile, …)

Launch applications on specified workspaces

At first, you need to find out the xprop window class of your target application. For this, start xprop and click on the application window. The 2nd string under WM_CLASS(STRING) is the string you need to know in the next step: add the following line to your i3 config file (and replace CLASS with the just determined string):


assign [class="CLASS"] 10

10 in this case indicates the nr. of the workspace the application window should be positioned.

Autostart applications

To automatically start applications, add one of the following lines to your i3 config file:


exec APPLICATION_COMMAND


exec_always APPLICATION_COMMAND

exec only executes the command when i3 is started, and exec_always executes the command each time i3 is reloaded. For example, starting your web browser is only useful for each fresh i3 start, but setting the wallpaper is useful to be done each time i3 is reloaded.

Monitor configuration

An easy way to configure the monitor setup using an UI is with arandr (UI for randr):


sudo apt-get install arandr

If you want to make a certain monitor configuration permanent you can a) configure monitors as wished in arandr, then b) save the configuration to a file, and c) add “exec_always CONFIG_FILE” to your i3 config file.

Wifi controls

i3 does not come with a built in UI for to control your wifi. To avoid configuring it via config files (which is still pretty cumbersome in Linux after all those years) you can use e.g. the graphical frontend wicd-gtk:


sudo apt-get install wicd-gtk
wicd-gtk

Change system fonts

One possible cosmetic improvement is to use different system fonts in i3. Alex recommends the Yosemite San Francisco Font: you can download it from its github page (under “manual install”), then change the font entry in the i3 config file to:


font pango:System San Francisco Display 10

Further, changing the font for the used gtk is recommended. You can do this is the gtk configuration file (in ~/gtkrc-2.0 and/or .config/gtk-3.0/…) by changing the font line to e.g.:


gtk-font-name="System San Franzisco 10"

Another option is to use the lxappearance config UI (but it currently seems buggy as not detecting all installed fonts correctly, as long they are not already in use with the system):


sudo apt-get install lxappearance
lxappearance

Replace dmenu with rofi


sudo apt-get install rofi

In the i3 config file, replace the dmenu-shortcut with e.g.


bindsym $mod+a exec rofi -show run -lines 20 -eh 1.3 -opacity "85" -bw 0 -font "System San Francisco Display 10"

or, to have different colors:


set $bg-color&amp;amp;amp;amp;nbsp;&amp;amp;amp;amp;nbsp; #2f343f
set $text-color #f3f4f5
bindsym $mod+a exec rofi -show run -lines 20 -eh 1.3 -opacity "90" -bw 0 -bc "$bg-color" -bg "$bg-color" -fg "$text-color" -hlbg "$text-color" -hlfg "$bg-color" -font "System San Francisco Display 10"

Transparency and fading effects

i3 does not provide for transparency or fading effects itself (it has less purpose in doing so, as e.g. windows don’t overlap in a tiling arrangement). To add those, use e.g. compton:


sudo apt-get install compton

and add “exec compton” to your i3 config file. The configuration is for compton is done via ~/.config/compton.conf, which you first have to copy from the example file location. With Ubuntu, this is:


cp /usr/share/doc/compton/examples/compton.sample.conf ~/.config/compton.conf

Afterwards, you can change settings there to your needs. We usually do the following:

  • Increase window fading speed, which is specified in milliseconds between fading steps: “”fade-delta = 5;”
  • Remove transparency from unfocused windows and menues (increases their readability). To do so, comment out “inactive-opacity” and “menu-opacity”.

Replace the i3status bar (bottom/top bar) with i3blocks


sudo apt-get install i3blocks

Copy the example i3blocks config file:


cp /etc/i3blocks.conf ~/.config/i3/i3blocks.conf

And change the value of the “status_command” from “i3status” with “i3blocks -c ~/.config/i3/i3blocks.conf” in your i3 config file.

Then you can customize i3blocks in your ~/.config/i3/i3blocks.conf file. For example:

  • Set the interval of date/time to 1 to have a 1s time update interval.
  • For volume add “command=/usr/share/i3blocks/volume 5 pulse” and set the interval 1 to use the pulse audio volume level with 1s interval.
  • Replace the text of labels with character/utf-8 icons as you like – this saves space in the bar. Alex has some nice examples for this in his own config file on github.

Workspaces on predefined screens

For users using a multi-monitor setup configuring to which screen specific workspace automatically go might be interesting:


# put workspaces on specific screens
workspace 1 output HDMI-1
workspace 2 output DVI-I-1
...

More i3 shortcuts

  • Auto back-and-forth: after switching from workspace A to workspace B, pressing the shortcut for workspace B again to switch back to workspace A. Add “workspace_auto_back_and_forth yes” to enable this feature.

Application configuration

Usability of some applications benefits from some more configuration:

  • The new mail attention add-on for Thunderbird makes i3 raise the urgent notification flag for new mails, so that the corresponding workspace is highlighted.
  • Enabling the “Message Notification” add-on in Pidgin makes i3 raise the urgent notification flag for new messages, so that the corresponding workspace is highlighted.

More hints

As i3 does not come with a built in network configuration UI: to configure networks like VPNs you can use the following (parts of the network-manager and network-manager-gnome packages):

  • nm-connection-editor allows editing network settings
  • nmtui allows for activating/deactivating networks over the command line

i3 does also not come with a built in UI for configuring computer mice. The mouse pointer speed can be adapted with xinput:

  • use “xinput –list –short” to determine the name of your mouse device, then
  • “xinput –set-prop “MOUSE DEVICE NAME” “Device Accel Constant Deceleration” 1.8″ to change the mouse pointer speed. The latter can be added to the i3 config file to automatically keep the speed after a restart of i3.

Backup your i3 configuration

Most (if not all) i3 configuration is located in those 3 files, which consequently are the only ones that need to be backed up:


~/.config/i3/config
~/.config/i3/i3blocks.conf
~/.config/compton.conf

Categories: Linux, Misc

How I learned to type the Neo2 keyboard layout: a review after the first 1.5 years

December 27, 2016 Leave a comment

neo-iconA couple of years back, I was asking myself: should I switch my keyboard layout to a more effective one? If yes, which new layout should I learn? For obvious reasons I searched for reports of people that did such a switch before. And this is why I try to give such a report myself now, after typing Neo2 for about 1.5 years – which seems to be a good point for summarizing my success story with Neo2 so far.

But first: some details about my typing characteristics before I learned typing the Neo2 layout. I…

  • …was “successfully” using the German QWERTZ keyboard layout for something like 12 years with a 10-finger-system.
  • …was typing with somewhere around 450 characters per minute for normal text.
    …did software development for a living and especially disliked the access to special characters my back-then keyboard layout.
  • …didn’t like the idea of using an ineffective keyboard layout in general. The layout I was using back then was a) not optimized for today’s hardware and b) I used it all day as it is the number 1 tool used in my profession, so investing in improvements should really pay off on the long run.

Overview of alternative keyboard layouts

There are (and were back in the day) many alternatives to the layout I used. For some examples, see e.g.:

I looked a bit into heuristically optimized keyboard layouts. Some of them use the same higher layers as Neo2 and just have layer 1 and 2 rearranged heuristically, such as the bone layout (German: http://wiki.neo-layout.org/wiki/Bone). Why didn’t I chose such a layout? Those had/still have less support, as they are not used so widely. This brings additional effort when configuring/using different PCs/hardware.

Additionally, for something so important for me and my daily business I preferred to use something already proven to work well, and would therefore bring less problems with it.

Why use Neo2 instead of other alternatives?

Neo was designed back in 2004 and was optimized for the German language. Due to German being very similar to English from key frequency point of view the “being optimized to German” did not make a big difference to me (and still does not make any, you can rest assured that it’s very comfortable to type English with it). Neo2 has a very good heatmap (optimized for key position and typing interleaving with left/right keys:

keyboard_heatmap_for_german_neo

Though there are arrangements that are a little more effective and further optimized (e.g. heuristically),  when it comes to key frequencies, having such a heatmap still is an amazing thing.

To me, even more important was that access to special characters is solved very cleverly with Neo2. There are 6 layers in total:

  • Layer 1+2 are lower and upper case characters – those are the ones you are probably used to on your keyboard. The “modifier” keys are the Shift keys.
  • Layer 3 are special characters (right below your fingers, e.g. in the home row!). The “modifier” keys are the CapsLock button on the left side and the “#” key on the right side.
  • Layer 4 are other special characters and special keys
  • Layer 5+6 are Greek letters.

The special characters on layer 3 make accessing special characters for programming very easy. And layer 4 is something that is easily underestimated: it features:

  • Keys to control cursor position like up/down/left/right, page up/down, delete left/right, home/end, escape, tab, enter, etc. – all under your left hand.
  • All numbers in the form of a keypad – all under your right hand.

From this point of view, Neo2 is more effective for programmers than e.g. Dvorak, Colemak, or similar keyboard layouts (still, those are pretty cool too). But besides the Neo2 layout having a lively community and already being proven to working well, this was the main reason for learning Neo2 instead of any other layout.

Learning and switching to Neo2

How not to do it

Maybe 3 years before I already had tried Neo2 – for very short time. Back then I set my goal similar to “I want to type in Neo!”, printed a Neo2 layout onto an A4 sheet of paper, switched my keyboard layout to Neo2 and just started typing it. In short: it did no work out. I had to search for every key using the printout, and of course got frustrated over the extremely low speed of typing (which was to be expected) very fast – and quit within hours.

How to do it

After the first try, the goal of switching keyboard layout remained. I wanted to use a more effective keyboard layout/Neo2 someday, as it would just be so much more effective for my everyday usage. The second time (maybe 2 years after the first try) I was better prepared to really perform a switch:

  1. I used KTouch with Neo2 to train single keys – one after another (the KTouch “beginner mode”). I did this is my free time, at work I continued to use my QWERTZ layout. Further, I made sure that I typed all keys with the correct fingers (I had had a slight mash-up with the QWERTZ layout before).
  2. In KTouch I used standard settings, which are minimum 180 chars per minute and 98% accuracy to pass each lesson and unlock the next one. This ensures you don’t try to learn  too much at once, which was the problem with my first try.
  3. Using KTouch I trained about 1/3 of the Neo2 lessons provided. Those were all about introducing and training new character keys. It was no problem to write with my QWERTZ layout at work at the time (switching was smooth).
  4. After finishing about 1/3 of the Neo2-lessons provided with KTouch (within 1.5 months as far as I remember), I did the “all in” thing: I switched all my layouts to Neo2 and typed all texts with it. This included my keyboard at work as well as e.g. my on-screen keyboard on my mobile phone. Of course it was very slow at the beginning again (first few days), but now I learned very fast. After about 2 to 3 weeks of typing, my speed was OK, while I was still learning at a fast rate without requiring any extra training time. This was when typing the old QWERTZ layout became more difficult for me.
  5. Since then I’ve just continued to write Neo2. Once in a while I did some more lessons in KTouch – but these were just for training and comparing typing speeds. But overall, using a typing trainer like KTouch definitely was the “enabler” of learning a new keyboard layout for me.

Progress report after the first 1.5 years

I started learning Neo2 around end of February 2015. Within about 1.5 months (after successfully completing about 1/3 of the lessons in KTouch) I made the “all in” move and switched all my layouts to Neo2. My progress since then:

  • Mid of April (~2 months): about 190 chars/min, 95.5% accuracy in the 1000 English words test from KTouch Neo2.
  • End of April: feeling more comfortable every day.
  • Mid of May (~3 months): about 250 chars/min, 95.5% accuracy in the German/English words test from KTouch Neo2.
  • Mid of June (~4 months): 286 char/min, 98.5% accuracy (09:02 in the “repeat many words test” in KTouch Neo2).
  • End of September (~7.5 months): 286 char/min, 96.3% accuracy (21:13 in the 1000 most frequently used English words test in KTouch Neo2) and 289 char/min, 97.2% accuracy (08:57 in the “repeat many words test” in KTouch Neo2).
  • Beginning of February (~1 year): 337 char/min, 96.8% accuracy (6:50 in the KTouch Neo2 train comma lesson).
  • Mid of February 2016 (~1 year): 330 char/min, 97.2% accuracy (18:20 in the 1000 most frequently used English words test in KTouch Neo2) and 311 char/min, 96.4% accuracy (8:20 in the KTouch Neo2 “repeat many words test”).
  • April 2016 ~1 year 2 months: 318 chars/min with 97.2 accuracy with the KTouch Neo2 congratulation text for finishing Neo2 training. Programming: 241char/min with 96.5% accuracy in KTouch Neo2 shell text (special characters, numbers, …). This pretty much is what I wanted in the beginning – it made me writing code on my keyboard more effectively. Additionally: have the feeling that I need to think less about how I do things, and more about what I want do to. Specially the position keys under left hand at level 4 (move cursor up/down/left/right, page up/down, delete left right, home/end) have proven extremely effective – much more than I initially thought – to quickly move the cursor around when programming.
  • July 2016 ~1 years 5 moths: 333 chars/min with 98.0% accuracy (18:09 in the 1000 most frequently used English words test in KTouch Neo2).

This is an example of how your progress for learning Neo2 in KTouch could look like:

  • Left: progress for one single lesson: usually, you will always increase in speed and accuracy when repeating the same lesson (which is why you are able to learn a new keyboard layout relatively fast if you take it seriously).
  • Right: progress for different/all lessons: usually, you increase in speed and accuracy for a specific lesson, and when you move on to the next lesson your speed and accuracy will be lower for it at the beginning – which is because you are just learning completely new things there. I can scroll far to the left in this view – and it’s a good feeling to see speed and accuracy continuously increasing over time there 😉

 

 

 

 

The final (and frequently asked) questions

Is my typing speed still increasing? Yes, definitely. Was it worth the effort? Would I do it again? Yes, definitely, to both. Though, at the moment I’m still a bit slower in typing regular texts than I was before, programming is definitely faster already. Typing feels less cumbersome: especially complex combinations of symbols/characters are easier to type, and I don’t need to move my right hand anymore for reaching e.g. arrow keys, delete/insert, or similar. Besides making a large number small tasks a little faster (which sums up to a lot in the end), this also just feels more satisfactory.

So, bottom line: in my opinion, especially when your job involves something like programming, learning Neo2 really pays off in the end.

Weka on Android: load precomputed model and predict new samples

November 27, 2016 Leave a comment

Weka on Android

Imagine you want to use machine learning on Android to predict the value for your target variable for new samples. Usually, neither the storage capabilities (for storing the training data) nor the computational power for training such models is available on mobile devices – assuming that you use a medium or large dataset and thorough evaluation of different model types and model parametrizations with a subsequent model selection. In such a scenario, the way to go therefore is:

  1. Train, evaluate, and select the optimal model for your machine learning prediction offline, e.g. using a desktop PC or server hardware.
  2. Export the ready trained model (“model file”, which usually tends to be relatively small) and ship it with your Android application.
  3. The “application case”/”production case” on the mobile device simply involves loading and using the model to predict new, yet unseen samples. Thereby, the prediction on the mobile device requires much less computational power and storage capabilities than the previous training of the model.

Based on the previous post about loading and using Weka models with JavaSE, this example demonstrates how to load and use a model Android which was previously trained with Weka “offline”, aka on a desktop PC. The core points you have to look out for when setting up the Android project:

  1. Use the exact same version of Weka you used for offline training the model also on Android.
  2. Copy the weka.jar to libs/, then select “Add as library…”.
  3. Ensure that “compile files(’libs/weka.jar’)” is present in build.gradle.
  4. Do a clean build: at the first time, this is going to require some CPU power and take some time.
  5. Don’t do anything with Weka that is UI related: Weka uses the JavaSE UI, which is not available on Android.
  6. Follow the details in the example below:
    1. Structure the new, yet unseen instance you want to predict the target variable so that Weka can work with it.
    2. Load the classifier object.
    3. Predict the target variable for the new instance.

The complete Android example project containing the code below can be downloaded here:

  • Download (7z): Android Weka Example.
  • SHA256sum: aaf8594954d3561218dc0dd621740f3184a171efa7016a2efe4b743a19845ffb
package at.fhooe.mcm.ml.androidwekaexample;

import android.content.res.AssetManager;
import android.os.Bundle;
import android.support.v7.app.AppCompatActivity;
import android.support.v7.widget.Toolbar;
import android.util.Log;
import android.view.View;
import android.view.Menu;
import android.view.MenuItem;
import android.widget.TextView;
import android.widget.Toast;

import java.io.IOException;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;
import java.util.Random;

import ml.mcm.fhooe.at.androidwekaexample.R;
import weka.classifiers.Classifier;
import weka.core.Attribute;
import weka.core.DenseInstance;
import weka.core.Instances;

public class MainActivity extends AppCompatActivity {

    private static final String WEKA_TEST = "WekaTest";

    private Random mRandom = new Random();

    private Sample[] mSamples = new Sample[]{
            new Sample(1, 0, new double[]{5, 3.5, 2, 0.4}), // should be in the setosa domain
            new Sample(2, 1, new double[]{5.6, 3, 3.5, 1.2}), // should be in the versicolor domain
            new Sample(3, 2, new double[]{7, 3, 6.8, 2.1}) // should be in the virginica domain
    };

    private Classifier mClassifier = null;

    TextView mTextViewSamples = null;

    @Override
    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.activity_main);
        Toolbar toolbar = (Toolbar) findViewById(R.id.toolbar);
        setSupportActionBar(toolbar);

        // show samples
        StringBuilder sb = new StringBuilder("Samples:\n");
        for(Sample s : mSamples) {
            sb.append(s.toString() + "\n");
        }
        mTextViewSamples = (TextView) findViewById(R.id.textViewSamples);
        mTextViewSamples.setText(sb.toString());

        Log.d(WEKA_TEST, "onCreate() finished.");
    }

    @Override
    public boolean onCreateOptionsMenu(Menu menu) {
        // Inflate the menu; this adds items to the action bar if it is present.
        getMenuInflater().inflate(R.menu.menu_main, menu);
        return true;
    }

    @Override
    public boolean onOptionsItemSelected(MenuItem item) {
        // Handle action bar item clicks here. The action bar will
        // automatically handle clicks on the Home/Up button, so long
        // as you specify a parent activity in AndroidManifest.xml.
        int id = item.getItemId();

        //noinspection SimplifiableIfStatement
        if (id == R.id.action_settings) {
            return true;
        }

        return super.onOptionsItemSelected(item);
    }

    public void onClickButtonLoadModel(View _v) {
        Log.d(WEKA_TEST, "onClickButtonLoadModel()");

        AssetManager assetManager = getAssets();
        try {
            mClassifier = (Classifier) weka.core.SerializationHelper.read(assetManager.open("iris_model_logistic_allfeatures.model"));

        } catch (IOException e) {
            e.printStackTrace();
        } catch (Exception e) {
            // Weka "catch'em all!"
            e.printStackTrace();
        }
        Toast.makeText(this, "Model loaded.", Toast.LENGTH_SHORT).show();
    }

    public void onClickButtonPredict(View _v) {
        Log.d(WEKA_TEST, "onClickButtonPredict()");

        if(mClassifier==null){
            Toast.makeText(this, "Model not loaded!", Toast.LENGTH_SHORT).show();
            return;
        }

        // we need those for creating new instances later
        // order of attributes/classes needs to be exactly equal to those used for training
        final Attribute attributeSepalLength = new Attribute("sepallength");
        final Attribute attributeSepalWidth = new Attribute("sepalwidth");
        final Attribute attributePetalLength = new Attribute("petallength");
        final Attribute attributePetalWidth = new Attribute("petalwidth");
        final List<String> classes = new ArrayList<String>() {
            {
                add("Iris-setosa"); // cls nr 1
                add("Iris-versicolor"); // cls nr 2
                add("Iris-virginica"); // cls nr 3
            }
        };

        // Instances(...) requires ArrayList<> instead of List<>...
        ArrayList<Attribute> attributeList = new ArrayList<Attribute>(2) {
            {
                add(attributeSepalLength);
                add(attributeSepalWidth);
                add(attributePetalLength);
                add(attributePetalWidth);
                Attribute attributeClass = new Attribute("@@class@@", classes);
                add(attributeClass);
            }
        };
        // unpredicted data sets (reference to sample structure for new instances)
        Instances dataUnpredicted = new Instances("TestInstances",
                attributeList, 1);
        // last feature is target variable
        dataUnpredicted.setClassIndex(dataUnpredicted.numAttributes() - 1);

        // create new instance: this one should fall into the setosa domain
        final Sample s = mSamples[mRandom.nextInt(mSamples.length)];
        DenseInstance newInstance = new DenseInstance(dataUnpredicted.numAttributes()) {
            {
                setValue(attributeSepalLength, s.features[0]);
                setValue(attributeSepalWidth, s.features[1]);
                setValue(attributePetalLength, s.features[2]);
                setValue(attributePetalWidth, s.features[3]);
            }
        };
        // reference to dataset
        newInstance.setDataset(dataUnpredicted);

        // predict new sample
        try {
            double result = mClassifier.classifyInstance(newInstance);
            String className = classes.get(new Double(result).intValue());
            String msg = "Nr: " + s.nr + ", predicted: " + className + ", actual: " + classes.get(s.label);
            Log.d(WEKA_TEST, msg);
            Toast.makeText(this, msg, Toast.LENGTH_SHORT).show();
        } catch (Exception e) {
            e.printStackTrace();
        }
    }

    public class Sample {
        public int nr;
        public int label;
        public double [] features;

        public Sample(int _nr, int _label, double[] _features) {
            this.nr = _nr;
            this.label = _label;
            this.features = _features;
        }

        @Override
        public String toString() {
            return "Nr " +
                    nr +
                    ", cls " + label +
                    ", feat: " + Arrays.toString(features);
        }
    }
}
Categories: Data Analysis

Weka in Java: Predict new samples using a precomputed and exported model object

October 2, 2016 1 comment

weka

Weka allows for exporting/saving computed models into a binary model file (usually having a “.model” file extension). On other machines (that e.g. don’t have access to training data or would lack the computational power for training the model themselves), those model files can be loaded and used to predict new samples. The following example in Java highlights two aspects of this process, which are a) loading such a model object and b) how to format a new, yet unclassified instance so that its target variable can be predicted using the loaded model:

import java.util.ArrayList;
import java.util.List;

import weka.classifiers.Classifier;
import weka.core.Attribute;
import weka.core.DenseInstance;
import weka.core.Instances;

public class Main {

	public static void main(String[] args) {
		new Main().main();
	}

	public void main() {

		// we need those for creating new instances later
		final Attribute attributeSepalLength = new Attribute("sepallength");
		final Attribute attributeSepalWidth = new Attribute("sepalwidth");
		final Attribute attributePetalLength = new Attribute("petallength");
		final Attribute attributePetalWidth = new Attribute("petalwidth");
		final List<String> classes = new ArrayList<String>() {
			{
				add("Iris-setosa");
				add("Iris-versicolor");
				add("Iris-virginica");
			}
		};

		// Instances(...) requires ArrayList<> instead of List<>...
		ArrayList<Attribute> attributeList = new ArrayList<Attribute>(2) {
			{
				add(attributeSepalLength);
				add(attributeSepalWidth);
				add(attributePetalLength);
				add(attributePetalWidth);
				Attribute attributeClass = new Attribute("@@class@@", classes);
				add(attributeClass);
			}
		};
		// unpredicted data sets (reference to sample structure for new instances)
		Instances dataUnpredicted = new Instances("TestInstances",
				attributeList, 1);
		// last feature is target variable
		dataUnpredicted.setClassIndex(dataUnpredicted.numAttributes() - 1); 

		// create new instance: this one should fall into the setosa domain
		DenseInstance newInstanceSetosa = new DenseInstance(dataUnpredicted.numAttributes()) {
			{
				setValue(attributeSepalLength, 5);
				setValue(attributeSepalWidth, 3.5);
				setValue(attributePetalLength, 2);
				setValue(attributePetalWidth, 0.4);
			}
		};
		// create new instance: this one should fall into the virginica domain
		DenseInstance newInstanceVirginica = new DenseInstance(dataUnpredicted.numAttributes()) {
			{
				setValue(attributeSepalLength, 7);
				setValue(attributeSepalWidth, 3);
				setValue(attributePetalLength, 6.8);
				setValue(attributePetalWidth, 2.1);
			}
		};
		
		// instance to use in prediction
		DenseInstance newInstance = newInstanceSetosa;
		// reference to dataset
		newInstance.setDataset(dataUnpredicted);

		// import ready trained model
		Classifier cls = null;
		try {
			cls = (Classifier) weka.core.SerializationHelper
					.read("/home/pirius/iris_model_logistic_allfeatures.model");
		} catch (Exception e) {
			e.printStackTrace();
		}
		if (cls == null)
			return;

		// predict new sample
		try {
			double result = cls.classifyInstance(newInstance);
			System.out.println("Index of predicted class label: " + result + ", which corresponds to class: " + classes.get(new Double(result).intValue()));
		} catch (Exception e) {
			e.printStackTrace();
		}
	}
}
Categories: Data Analysis

Ubuntu Gnome 16.04: Some Useful Configuration

August 12, 2016 Leave a comment

Ubuntu Logo      Ubuntu Gnome Logo

A lot has changed in Ubuntu Gnome in the past 2 years. With the release of Ubuntu Gnome 16.04 a few months ago we take the chance to do a follow-up of our 2014 post and summarize the configuration we usually apply to a fresh installation of Ubuntu Gnome 16.04.

To consider for backup before installation

Just for completeness: you might want to backup some things beside private documents, music, videos etc, e.g.:

  • ~/.bashrc contains path variables, changes to bash behaviour etc.
  • ~/.bash_aliases contains aliases, other additions to bashrc.
  • ~/.conkyrc contains configuration for conky (if you use conky).
  • ~/.bash_profile “The personal initialization file, executed for login shells ” (see bash man page).
  • ~/bin/ may contain scripts and links to executables (as ~/bin is usually added to PATH).
  • ~/.config: contains configurations for different applications (e.g ~/config/compton.conf or ~/.config/i3/ if compton or i3 they are used).
  • ~/.gnupg/: contains PGP keys (used e.g. via Thunderbird enigmail).
  • ~/.local/share/rhythmbox if you use Rhythmbox, this folder contains your local playlists.
  • ~/.mozilla/: contains Firefox configuration, history etc.
  • ~/.purple/: contains purple based messaging configuration (e.g. pidgin), including possibly used OTR keys.
  • ~/.ssh/ contains ssh private and public keys, authorized keys for ssh login and ssh fingerprints seen in the past.
  • ~/.thunderbird/ contains Thunderbird configuration and mails.
  • GTK bookmarks: contains GTK based bookmarks (used e.g. by Nautilus, Nemo, Thunar, PCManFM etc). With Ubuntu 14.04 an newer they are located at ~/.config/gtk-3.0/bookmarks.
  • /etc/fstab contains automatic mounted partitions and folders.
  • cron jobs can be found on multiple locations (root-level /etc/crontab, root-level anacron and periodics /etc/cron*, user-level /var/spool/cron/crontabs)

Gnome Shell

In case you are new to Gnome Shell we strongly recommend reading a short intro, as this will probably save you lots of time later: Gnome shell cheat sheet one and two (they are not always updated to the newest…). Some examples for of useful features of Gnome Shell that might not be intuitive if you were using other UIs before:

  • The main menu of Gnome Shell style windows is hidden in the black area right next to the “Activities” button (top left) and gets shown only on click.
  • The message tray (lists messages from apps like Skype, Pidgin, Thunderbird etc.) is hidden as well and accessible via the shortcut Super+M.
  • The suspend button is hidden in the top right user menu under the power button and becomes visible on pressing Alt.
  • You can restart Gnome Shell with executing “r” in the Alt+F2 box without closing your applications etc. Useful to developers as well as to “soft reset” stuff in case of window freezes or other bugs. This is btw. equivalent to calling gnome-shell –replace directly (in case you do that from a tty without X server you need to specify the display – which will usually be 0 as you will typically only have one X server running: gnome-shell –replace -d :0).
  • Executing “lg” in Alt+F2 brings up the Gnome Shell log, which is useful in case of problems with Gnome Shell and its extensions.

Gnome Shell search providers

  • Type “search” in the activities view to bring up the search configuration. You might want to disable and/or reorder some choices here.
  • Type “privacy” in the activities view to bring up the privacy settings, which might be worthwhile to review as well.

Gnome Shell tweak tool

The tweak tool provides access to a number of tuning options. It should come preinstalled already, but in case it is not there yet:

sudo apt-get install gnome-tweak-tool

For example:

  • Changing the system font size (we usually change it from 11 to 10).
  • Changing the cursor theme (we usually change it to the dmz cursor theme:
     sudo apt-get install dmz-cursor-theme
  • Change workspace behavior, so that workspaces don’t only affect the primary screen but all screens.
  • Change how modal dialogs are attached to their parent windows.
  • Window selection model, if you’re used to mouse-over window selection (as I’m personally). Here’s the difference between mouseover and sloppy in case you wonder. I personally don’t like windows to be raised to the front automatically after the mouse pointer standing still for a certain time in it. This can be changed by turning off “automatically raise windows” in tweak tool or “auto-raise” in org–>gnome–>desktop–>wm–>preferences, or by adapting “auto-raise-delay” next to it. And in case you want windows to immediately get focus as you hover the mouse over them deselect “focus-change-on-pointer-rest” in org–>gnome–>shell–>overrides.

Gnome Shell extensions

We typically install some Gnome Shell Extensions, including the following ones:

  • AlternateTab: don’t group windows of same application in Alt+Tab, can be configured to show only the application icon instead of the window preview.
  • Dash to Dock: easy way to set maximum dock icon size + disabling “require pressure to show dock” to achieve nearly-normal dock behaviour.
  • gTile: a window tiling utility – supports aligning windows to tiles spanning your screen (e.g. 2×2, 3×2, 4×4).
  • Native windows placement: more intuitive placing of windows in the activities window.
  • Notification Alerts: makes the user’s menu blink on new messages in the message tray.
  • Recent Items: adds list of recently used items to the top bar.
  • Search recently used files: integrated into the Gnome Shell activities view – this seems to work no longer as it’s not compatible with Gnome Shell >= 3.14. We’re still investigating how to best include recently used files in the activities search (frankly neither the Gnome Shell search providers “files” and “documents” nor other extensions seem to do this trick).
  • TopIcons Plus: one successor of TopIcons that nicely integrates into Gnome Shell and shows legacy application “tray” icons on the right side of the shell top bar.
  • User themes: enables loading of themes from ~/.themes/, which can then be selected in the gnome tweak tool to customize Gnome Shell.
  • WindowOverlay Icons: shows application symbol above windows in the activities overview. With centred and enlarged icons (~150px) helps pinpointing windows fast in our opinion.

Window button layout: put buttons left

We usually change the button layout so that the buttons are arranged on the left side (like with MAC systems) – as this saves you mouse distance. For Gnome Shell the needed tool is dconf-editor:

sudo apt-get install dconf-editor

In dconf-editor changes have to be applied to “button-layout” in  to org.gnome.desktop.wm.preferences in Ubuntu 16.04. Our usual button layout is “close,minimize,maximize”, all on the left side.

More look and feel: themes, icons, …

If you feel the need for changing the look and feel of your Gnome Shell even more, you can add different themes, icons sets, etc to Gnome Shell (all in the tweak tool). We sometimes use these:

  • Super Flat Remix Gnome Shell theme – “super lightweight”. If you use conky you might want to adapt the opacity to fit this theme (its at about 15%).
  • Elementary Icon set – the old one from 2013, “super clean”.
  • Depending on your needs you might also consider installing a dock shell extension (e.g. Dash to Dock, which features a bit more options for configuration than e.g. SimpleDock).
  • If you are instead up for a good looking and nicely working MAC oriented theme, consider the Gnome Shell/GTK+ theme OS X 10.11 El Capitan.

Some further configurations

Custom window shortcuts

Window specific shortcuts are mostly in dconf-editor in org–>gnome–>desktop–>wm–>keybindings. Additionally to adding assignments you might need to remove some to get your configuration working. An example of what we often use:

  • maximize: []
  • maximize-horizontally: [‘<Super>Page_Up’]
  • minimize: [‘<Super>Down’]
  • switch-to-workspace-up: [‘<Control><Alt>Up’]
  • toggle-maximized: [<Super>’Up’]
  • unmaximize: []

Custom command shortcuts

User specific commands can be defined in “Keyboard–>Shortcuts”. We typically create a shortcut to open a file manager at the data partition, such as Ctrl+Alt+E for nemo /media/data or nautilus /data/media, depending on your file manager.

Show all startup applications

Since Ubuntu 12.04 some startup applications are hidden, therefore not shown in gnome-session-properties. To show those applications use:

sudo sed -i "s/NoDisplay=true/NoDisplay=false/g"; /etc/xdg/autostart/*.desktop

Special Keyboard Buttons with MS Ergonomic Keyboard

If you happen to use a MS Ergonomic Keyboard (e.g. the Natural Ergonomic Keyboard 4000) some buttons might not work for you by default. It might be helpful to change the following:

  • For multimedia forward/backward: in Keyboard->Shortcuts->Sound and Multimedia: assign “Forward” and “Backward” to the previous/next track actions.
  • To use the zoom slider for scrolling add the following in etc/udev/hwdb.d/61-keyboard-local.hwdb
    keyboard:usb:v045Ep00DB*
     KEYBOARD_KEY_0c022d=pageup
     KEYBOARD_KEY_0c022e=pagedown

    Then recreate the hwdb:

    sudo udevadm hwdb --update

    Those might require to replug the keyboard or reboot.

Tooltip background color

The default tooltip background color is black – which is horrible for programs using dark fonts (like Eclipse). To change the tooltip background color install gnome-color-chooser and in Specific–>Tooltips change Foreground to black and Background to pale yellow, as Nick Andrik suggested here.

Application Configuration

File browser

  • Though Nautilus file browser has become better with the latest releases some might still not be convinced by it. One of the disadvantages it brings for power users is the missing tree view feature in the main area (the right one, not the left tree or places view). Options for replacing Nautilus include: Nemo, Thunar, PCManFM, Dolphin, and many more.
  • In case you stick with Nautilus:
    • If you dislike the recursive-search-as-you-type feature: you can disable it (and enable the old interactive search) by enabling org.gnome.nautilus.preferences.enable-interactive-search with the dconf-editor.
    • Most file browsers support the creation of files from templates in the context menu (right click -> “new document” -> select one of your templates). The location you need to put template files that should show up in this menu is ~/Templates/. If you notice that with Nautilus the “new documents” entry in the context menu is missing entirely, try creating an empty file in ~/Templates/, which should make it visible again.
    • Add custom actions with nautilus-actions-config-tool, such as running Terminator from the current location as stated in detail here.

Bash

We usually adapt Bash behavior to immediately log commands of all opened terminals.

Terminal emulation

Usually we also replace the default terminal emulation if it’s not very powerful (split screens, all hotkeys configurable etc.). Our current choice for Ubuntu is terminator, which features split screens and a detailed hotkey configuration – but of course there are many others out there with similar features.

Firefox

Besides enabling the “do not track me” flags in browsers other measures might also help reducing the traceability. In Firefox–>Properties–>Privacy (actual names might differ slightly, but the meaning is clear):

  • Enable “Firefox will record custom history”: discard third party cookies when Firefox closes
  • Enable “Clear history on Firefox close”: check “empty cache”, “discard cookies” and “discard active logins”

Firefox addons

Some Firefox addons we find useful, therefore tend to install straight away as well:

Thunderbird

We usually install some Thunderbird addons directly via Thunderbird->Tools->Addons. Addons we find useful are:

Disabling unnecessary notifications

With Gnome Shell, notifications tend to get persisted in the message tray. This is a good thing, as we won’t miss e.g. that we’ve been pinged in a chat, even if we were not at the PC when the ping happened. But we also don’t want unnecessary messages to show up there, as those tend to be distracting. Usually, we disable the following notifications (and alike in other programs):

  • Notifications from contacts coming online/offline in Pidgin, Skype, etc.
  • Notifications of music players playing the next track (e.g. Rhythmbox: disabling the “Notifications” plugin).
  • Notifications from any all-time-running terminal, like Yakuake or Guake.

Finally: root partition backup

Discussions about backups are omnipresent, so we won’t discuss the problem again here. In case there is no automatic backup having a “clean” – but properly configured system right after installation might be a good time to do a backup. That might save you lots of time in case users mess up the system. An option for quick backups is using fsarchiver from a live media (the installation media might come in handy). For a quick start with fsarchiver we’d recommend their quick start page. We assume that your installation was not split across multiple partitions in our example. The fsarchiver-command for backing up a partition on /dev/sda1 to /media/data/backups/datetime.fsa using 2 cores (-j2), splitting archives to multiple files of 4000MB (-s4000), standard compression (-z3) and verbose output (-v) from a live media would be:

sudo fsarchiver savefs -j2 -s4000 -z3 -v /media/data/backups/datetime.fsa /dev/sda1

The fsarchiver-command for restoring partition /dev/sda1 from /media/data/backups/datetime.fsa from a live media would be:

sudo fsarchiver restfs -j2 -v /media/data/backups/datetime.fsa id=0,dest=/dev/sda1

 

Categories: Linux