Playing with JsRender and JsFiddle.net

Here's an example:

Machine Learning - Supervised Learning

Just finished a online extension course on Machine Learning via Stanford Engineering school. Excellent course.

Linear Regression

We learned about simple linear regression - fitting a line/curve to a 2D set of data points. This was then expanded to include 3 or more dimensions. This is useful for prediction and extrapolation where 2 or more variables (dimensions) represent correlated features (height and weight for example.) In a more complex scenario, we focused on predicting house prices base on various features such as square footage, number of rooms, number of floors, etc. Some important learning from this was the need to normalize/scale the features so they have similar ranges: Xs = (Xi-mean(X))/range(X).

Linear regression fits a straight line, but by applying increasing powers to each feature variable, (x1, x2^2, x3^3, etc), and passing the results into the gradient descent algorithm, you can fit a polynomial curve. Or take logarithms, roots, etc., to fit other types of curves.

The algorithm which finds the best fit is called gradient descent. We form a hypothesis formula to predict our desired outcome (in this case price): h(X) = p1*x1 + p2*x2 + p3*x3...
The trick is to find the values of the vector P (p1,p2,p3,...) which gives the best prediction. We determine how good/bad the prediction is by comparing the training data X with the predicted data Y in a cost function J = mean((X-Y)^2) - so the best fit would equate to the lowest cost function results.

Gradient descent uses the derivative of the cost function which calculates the rate of change of the cost function with respect to each parameter in P. p1 = p1 - a*derivative(J,p1)
'a' here is the learning parameter which decides how big a step along the cost curve to take. The parameter is updated. As it reaches the minmum point of the cost curve, the derivative essentially becomes 0 (zero slope), so it no longer changes. At this point, the parameter has been found for that minimum.

The algorithm converges then on the parameter values, which, when fed into the hypothesis equation above - should represent the best fit to the training data.

Logistic Regression

Linear regression is good for prediction of real valued numbers, but in many cases, we want to use a set of features to classify something true or false. For example, whether a tumor is malignant/benign or whether an email is spam or not. Linear regression does not reliably work in these cases because the resulting variable is integral (0 or 1 in these cases) - not a real number.

In this case we need a hypothesis function that examines variables bounded between 0 and 1. We use a "sigmoid" function to bound the values h(x) = 1 / (1 + e^(theta*x)) where "theta" is a parameter similar to linear regression. This function has values between 0 and 1 - essentially representing the probability that, given theta and x, the result y is true or false.

With this, we can create a cost function that compares the difference between the predicted values y and the training data. We calculate the derivative of this function and again use gradient descent to find the optimal parameters (thetas).

Neural Networks

Neural networks are a possibly much more efficient way to find the optimal parameters for a classification problem - such as the ones we solve with Logistic Regression. Especially ones with large number of classes. In logistic regression, our output classes are usually yes/no (k=2), or perhaps a prediction of the letter grades students will get based on certain habits they exhibit, a multi-class definition k>2 where k is less than 10.

Neural networks would extend to something like predicting letters in an alphabet based on the pixels (each one being a variable) in an image. This has a k=26 set of classes and for an image of 20*20 gray scale pixels, 400 variables (and thus 400 parameters to calculate).

Logistic regression would take a lot of time and would examine many non-optimal parameter values as it searched. Neural networks have been found to more quickly converge to optimal parameters by calculating the parameter values in stages (or "layers") that feed outputs from an input layer to one or more inner layers, that feed an output layer.

We're still working with a hypothesis function and cost function, and we use numerical method to minimize the cost function - which essentially is the neural network. The cost function uses the neural network algorithm to determine a cost, along with an array of gradients (derivatives) for each variable. These outputs are fed into the minimization function to calculate the optimal parameter weights for the neural network layers.

Once this "training" is completed, we have a set of weights for each layer of the network. The prediction calculation then boils down to a series of matrix multiplications for each layer to calculate the output values - which is very speedy.

In all the above cases, we use a training set of data with known inputs/outputs. The hypothesis function is written in a way that pairs each variable (feature) with a parameter that weights the variable in some way. The comparison between the known predictions and the hypothesized predictions is the "cost" function. In all cases, our goal is to minimize the difference or cost by choosing the best weights (parameters) to use in the hypothesis function.

Getting Layout Dimensions in Android

It's always interesting to learn a new operating system or API, but it can often be a little frustrating. You never know when you're going to run into a couple of hours of head scratching. Often, it seems to be the simplest tasks - such as determining the height or width of an element you can plainly see rendered on the screen - that turn into hours of twiddling.

In Android, you attach a view tree to your Actitity in order to be able to interact with the UI.
Once this is done, you can get references to your widgets (views) using a call like this from your Activity:
TextView t = (TextView) findViewById(R.id.txtName);

Now, views all sport a method like this: t.getHeight()
Which, presumably returns the actual height of that view element. Problem is, this method invariably returns zero.

At first I thought it was because I was calling this during the onCreate() method in my activity class. Perhaps the layout wasn't fully calculated at this time. So I moved my code to the onResume() method. This method is called whenever the activity is entered or re-entered. But again, no luck. Strange, as the layout at this point should be fully ready to display. I think there's a bug here in Android (I'm at version 2.2 currently.)

Luckily, I accidentally hit the search button while running a test on my EVO and, lo and behold, the height values I was trying to report on a TextView in my layout suddenly had non-zero values in them. Obviously, some sort of layout process had been forced, and the state of the view tree had been updated.

How can I get notified when the layout is fully calculated? The short answer is to attached a listener for a callback when the layout is calculated. Here's what it looks like.

@Override
public void onCreate(Bundle savedInstanceState)
{
  super.onCreate(savedInstanceState);
  RelativeLayout vMain = (RelativeLayout)
     this.getLayoutInflater().inflate(R.layout.main, null);
  vMain.getViewTreeObserver().addOnGlobalLayoutListener(
  new ViewTreeObserver.OnGlobalLayoutListener() {
    public void onGlobalLayout() {
      DisplayLayoutDimensions();
    }
  });
  setContentView(vMain);
}


public void DisplayLayoutDimensions()
{
  StringWriter sw = new StringWriter(1000);
  PrintWriter out = new PrintWriter(sw);
  TextView t = (TextView) findViewById(R.id.textview);
  ImageView img = (ImageView) findViewById(R.id.hp67);
  out.printf("\nImage Drawable\n");
  out.printf("ImageView Height dp: %d\n", img.getHeight());
  out.printf("ImageView Measured Height dp: %\n",
     img.getMeasuredHeight());
  t.setText(sw.toString());
}

Android image drawing

When the emulator is in programming mode (or a program is currently entered/loaded), I wanted to display a magnetic card image in the magnetic card holder slot in the same place it would go on the real HP67.

Problem: how to have the image be invisible when no program is loaded or being entered?

There is a setVisibility() method on the ImageView class, but I'm darned if I could get it to work. So, I adjusted the alpha instead, using the setAlpha() method, and this works beautifully.

Here's how I did it. This works, but right now it uses absolute margin offsets in the RelativeLayout view group. Later, I'm going to investigate how to make this more dynamic to the actual screen dimensions on the device.

1. Place the card image file in the project's res\drawable folder. In my case, the card file is card.png.
   
2. Add a tag to my main.xml layout - which is a RelativeLayout view group. Positioning the image where it should be displayed. The android:src attribute points to the image in the drawable folder. The id is assigned as well so we can get a reference to this view later:
   
   <RelativeLayout>
   ...
   <ImageView
   android:id="@+id/card"
   android:layout_alignParentLeft="true"
   android:layout_alignParentTop="true"
   android:layout_width="260dp"
   android:layout_height="50dp"
   android:layout_marginLeft="30dp"
   android:layout_marginTop="87dp"
   android:src="@drawable/card"
   />
   </RelativeLayout>
   
3. In the activity class where this view is used, load the view. In my case, I want to have the image be initially invisible, so I set the alpha to zero immediately after setting the content view:
   
   _vMain = (RelativeLayout) this.getLayoutInflater().inflate(R.layout.main, null);
   setContentView(_vMain);
   ImageView card = (ImageView) findViewById(R.id.card);
   card.setAlpha(0);
   
4. Later, when the user clicks the program-mode button, I have logic that reveals the card image like this:
   
   ImageView card = (ImageView) findViewById(R.id.card);
   card.setAlpha(255);

Android windowBackground and Title hiding

To create a window without a window title, and including a custom window background - in this case an image:

1. Create a style in your strings.xml file under the res\values folder in your project:
<style name="hp67Background" parent="android:Theme">
<item name="android:windowBackground">@drawable/hp67</item>
<item name="android:windowNoTitle">true</item>
</style>

This style inherits from the android theme via the "parent" attribute. The background can be any drawable type, including, if desired a color code - such as #ff0000. In this case, I'm using a png image (hp67.png) saved under the res\drawable folder.

The android:windowNoTitle setting is set to true.

2. Edit your manifest XML file to include the style as a theme on the tag:

<application android:icon="@drawable/icon" android:label="@string/app_name" android:theme="@style/hp67Background">

This should do the trick!

TableLayout Example

After piecing together posts from the Android developer Google group , and referring to the Android SDK reference, I was able to create a working XML example for the TableLayout:


<?xml version="1.0" encoding="utf-8"?>
<LinearLayout xmlns:android="http://schemas.android.com/apk/res/android"
android:orientation="vertical"
android:layout_width="fill_parent"
android:layout_height="fill_parent"
>
<TextView
android:layout_width="fill_parent"
android:layout_height="wrap_content"
android:background="#300000"
android:textColor="#ff0000"
android:text="@string/display_value"
/>
<TableLayout
android:layout_width="fill_parent"
android:layout_height="fill_parent"
>
<TableRow>
<Button
style="@style/calcButton"
android:layout_column="1"
android:layout_height="wrap_content"
android:text="@string/one"
/>
<Button
android:layout_column="2"
android:layout_height="wrap_content"
android:text="2"
/>
<Button
android:layout_column="3"
android:layout_height="wrap_content"
android:text="3"
/>
<Button
android:layout_column="4"
android:layout_height="wrap_content"
android:text="+"
/>
</TableRow>
<TableRow>
<Button
android:layout_column="1"
android:layout_height="wrap_content"
android:text="4"
/>
<Button
android:layout_column="2"
android:layout_height="wrap_content"
android:text="5"
/>
<Button
android:layout_column="3"
android:layout_height="wrap_content"
android:text="6"
/>
<Button
android:layout_column="4"
android:layout_height="wrap_content"
android:text="-"
/>
</TableRow>
<TableRow>
<Button
android:layout_column="1"
android:layout_height="wrap_content"
android:text="7"
/>
<Button
android:layout_column="2"
android:layout_height="wrap_content"
android:text="8"
/>
<Button
android:layout_column="3"
android:layout_height="wrap_content"
android:text="9"
/>
<Button
android:layout_column="4"
android:layout_height="wrap_content"
android:text="/"
/>
</TableRow>
<TableRow>
<Button
android:layout_column="2"
android:layout_height="wrap_content"
android:text="0"
/>
<Button
android:layout_column="3"
android:layout_height="wrap_content"
android:text="="
/>
<Button
android:layout_column="4"
android:layout_height="wrap_content"
android:text="&#x00f7;"
/>
</TableRow>
</TableLayout>

</LinearLayout>

TopShelf - Shelving

Played around with a nifty new feature for TopShelf 2.0 called "shelving" that allows you to create a service simply by dropping a .Net assembly dll into a folder.

Topshelf is a tool for easily creating and installing a Windows service in .Net that can run as both a console program, as well as a service.

The new feature is itself a Topshelf-created service called Topshelf.Host.exe. When installed, the service appears in the services control panel as "Topshelf.Host".

The documentation for this new feature is a little sparse. OK, it's really sparse. So, I'm posting a few tips to get started with this new feature.

1. Start by getting the source code and building it.
-The current project is on GitHub Topshelf Repository.
-Run build.bat.
-This creates a folder:
{Your Project Root}\Topshelf\build_output\Topshelf

2. Install the Topshelf.Host service.
- Copy the Topshelf folder under build_output to a location of your choice, for example: c:\tools\TopShelf.
- Navigate to the folder in a command shell.
- Execute:
Topshelf.Host install

3. Setup logging.
The default log4net.config file only has a consoleappender - not very useful when the host service is running as a service. There is a clock.log4net.config in the Topshelf folder which you can copy and rename to log4net.config - then edit it to change the logfile name from "..\..\clock.log" to "topshelf.log".

4. Install "StuffOnAShelf" sample service.
The docs here are not super clear. There is a sample project in the TopShelf solution; however, the output of build.bat doesn't emit the sample folders, and, if you build the project in Studio, you'll find the full set of required DLLs are not placed in the "obj" folder.

But, the build does put the binaries and sample configs in the TopShelf directory. You just need to assemble everything yourself in a folder and drop in into the TopShelf\Services folder.

The easiest thing to do is create a folder under the Topshelf directory called "StuffOnAShelf", and copy the following items into it from the Topshelf directory:
clock.log4net.config
log4net.dll
Magnum.dll
TopShelf.dll
clock.config (rename this to StuffOnAShelf.config)
StuffOnAShelf.dll

Then drag the entire "StuffOnAShelf" directory under the Topshelf\Services folder.

5. Examine the results.
If everything went well, you should find two log files in the TopShelf directory:
clock.log
topshelf.log (assuming you used the log4net config suggested above).

You should see the topshelf host messages showing the loading of the StuffOnAShelf service.
You should see the output of the service itself in the clock.log.

Summary

This is a pretty new feature. There is no way to directly stop the "sub-services" hosted by the Topshelf.Host service - short of removing their directory out from under the Topshelf\Services folder.

Also, there's no direct way to know if they are actually running - short of looking at their log files.

At some point. it's likely that a console could be developed to watch the "sub-services" and control them. Until then, if you want full control, you'll need to look at the other sample project - called "Stuff" - which let's you create a full service that shows up in the services control panel.