Saturday, March 18, 2017

Dealing with legacy Matlab code

A friend of mine has a project that works in Matlab Simulink 2007. He recently opened that simulink model in Matlab 2010 and could run it from Simulink without problems. But when he tried to transfer that model to xpc target for real time integration, the simulation hung and became unresponsive.

He and his team tried to find the problem by disabling Simulink blocks one at a time and in the end found the block that was causing problems. When they compared the 2007 version with the 2010 version, they could not find any differences which means that there is probably a backward compatibility issue that affects real time deployment. They had to manually recreate the whole model (more than hundred blocks) manually in Matlab 2010. Doing manual editing in code always carries the risk of breaking something. You might not know it unless you do tests with high coverage.

My first advice was to create a unit test framework so that you are sure that the new version yields the same outputs for the same inputs. A good strategy is to write scripts that vary inputs using unform distribution, run simulation on xpc target for every input, record outputs (or timeout when simulation hangs) and then compare outputs with previous results and create a report that highlights differences.

My second advice was to automate the update-to-newer-Matlab process by writing a program that automatically creates a new Simulink mdl file from the old one, beause it is easy to parse the mdl file which has an xml-look. Steps:
  1. Takes as input the depth of parsing. 1 = Copy the whole model. 2=Go one step deep and create separate mdl files for blocks in that depth 3=Go 2 steps deep etc.
  2. Parses the old Simulink mdl file and extracts the blocks
  3. Creates an mdl file in the new version format.
  4. Inserts the blocks into new mdl file(s)
  5. Runs unit tests on xpc target to see if the two versions have the same outputs (i.e. they are within 1e-15) neighborhood.
With this program, my friend can first check if for input depth =1, the model passes tests. If not, he can increase the depth until he can isolate the problem. Increasing depth on error could be automated too.

Saturday, March 11, 2017

Intelligence tests

I don't have much faith in intelligence tests. My definition of intelligence is successful adaption to new circumstances. Adaption and persistence determine your material success. How can you hope to test adaption, especially the social component in an hour long test?

Recently a friend of mine told me that his child's test score indicated his child being super intelligent but my friend didn't value the result much, although it was better than having an imbecile score. Our conclusion is that such tests mainly measure how much care the child was given by the parents. If parents have done a good job of parenting, I suspect any kid who does not have a mental condition will score highly in those tests.

Unrealistic proposal requests

Besides software development, my responsibilities also include assessing and answering simulation related proposal requests from other firms. Since simulation is usually poorly understood, the requester puts in sentences like "the simulation shall be realistic" without knowing how vague and overly ambitious that is.

Without going into philosphical details about what is "realistic", I can say that it involves a lot of real world testing and modeling. The budget that is available is usually less than 1% needed to fulfill realism. In that case we have two choices:
  1. Prepare a proposal that takes every possible risk into account and demand a billion dollars (same as saying NO).
  2. Think about what can be done with the given budget, write down your assumptions and limitations and present that as a proposal. This is the way to go if you want to avoid suffering.
The major cost factor in a simulation project is integration and testing. Your first question to any proposal request should be "how will you know that our simulation works as you expect". The answer will determine not only how much testing is needed but also how detailed your models should be. It is like unit tests driving design. If you don't get a satisfactory answer, write down in your proposal how and where you will test it.

Sunday, March 05, 2017

Software manager reading list

A few days ago, a manager friend of mine asked me for advice about what to do in a project that is near crisis, i.e. it has missed a deadline and looks like will be over budget.

Crises are common in software projects because their complexity usually is beyond our time and resource prediction capabilities. I offered my friend the following books and recommended reading them first before demanding that his team starts working overtime:
These books will ground him in reality and make him realize what kind of a mess he is in. I also told him that he should only tell his superiors the reality if he is certain that they are interested in the truth. A lot of times people want to hear only good news and are content with ignorance until it hits.

If you sense that your superiors want to live in fantasy land, never tell them the bad news because you will be labeled as a pessimist. Even after your predictions prove to be true in the end, you being labeled as a nuisance won't be washed away.

These books will help you understand what is going to happen and why. You might not be able to change the outside world, but you will have less stress inside, not feeling any guilt, knowing that you did the best you could.

Monday, February 27, 2017

Probability: Difference between "exactly..." and "at least..."

In probability, I have to pay close attention to the problem statement because I easily make errors. As an example, let's say I toss a fair coin n times. What is the probability of
  1. Exacly 1 head
  2. At least 1 head
The figure below has n as the x axis and the two different probabilites as y axis. As you can see, the "exactly 1 head" probability decreases with n while the "at least 1 head" probability increases with n.

Matlab script for generating the plot:
n = 1:16;
pExactly1 = n./2.^n;
plot(n, pExactly1); grid on
hold on
pAtLeast1 = 1-(1-0.5).^n;
plot(n, pAtLeast1)
legend('exactly 1 heads', 'at least 1 heads')
xlabel('n'); ylabel('p')
Note that you can also get probability of at least 1 heads with this sum: pExactly1Head + pExactly2Heads + ... + pExactlyNHeads, where
pExactlyKHeads = nchoosek(n,k) * ph^k * (1-ph)^(n-k)
Even simpler: 1 - pExactly0Heads = 1 - nchoosek(n,0) * ph^0 * (1-ph)^(n-0).

For more information, visit Khan Academy.

Car IoT

As aircraft have become software with wings, so are cars on their way to become software on wheels. Self driving and IoT have accelerated this trend. Yesterday, when I was talking with my friends we envisioned a design that could improve driving comfort: Cars could collect vibration data measured by their sensors and transmit it together with location information to servers. This data would indicate road quality. The server could process and broadcast data to all cars. Other cars could adjust suspension settings in advance that would maximize comfort on that road.

With this technique, only the first few cars that collected data would suffer and the rest would benefit. Data would be current at all times. I think Google Maps uses a similar approach to guess traffic congestion by collecting data from smartphones.

To collect data we need:
  • Vibration sensors on car.
  • Access to internet and access to central data server
To adapt to road conditions we need:
  • Adaptable suspension on car (i.e. actuators and sensors)
  • Access to internet and data server by car.
If we could protect privacy, we could automatically tap into the intelligence of the swarm.

Note to self: Do a simpler version of this concept as a hobby project.

How to evaluate personnel

It would be easy to evaluate performance if everybody (including the evaluator) was of high quality and there was no Dunning–Kruger effect. Your only criteria would be years of experience. Unfortunately, real life is more complex than that and there are large differences between employees.

The next best thing might be to have three bins: low, average, high. The low bin might be for fresh graduates who just started to work. The high bin consists of people who are highly productive and experienced.

You divide your staff according to their general level into these bins. Each bin has a lower and upper salary bound. Then you rank the people in each bin according to their productivity. I know that realistically measuring productivity is a tough problem but there is no escaping from it. For example, you have to decide if the person with 3 months of overtime had really productive overtimes or was just producing 8 hours of work in 12 hours. In each bin, at the top of the pyramid should be people with high productive overtime and at the bottom should be persons with no or low productive overtime. That way, you assure that everybody gets their fair share and productive individuals get their extra.