pfh blogs


~ 1% inspiration, 99% perseveration ~

30 July 2015, 0:04 UTCLinear models, a practical introduction in R

These are slides to a talk I gave introducing linear models and nested model hypothesis testing in R. These are the basis of many common statistical tests.


7 May 2015, 5:47 UTCWhen I was a young lad in the '90s

When I was a young lad in the '90s, Bill Gates was a bad guy. Turns out he just hadn't gotten to the "and give to the poor" bit. Now Google and Apple are battling for OS dominance and the cloud runs Linux.

When I was a young lad in the '90s, geeks and nerds were oppressed and didn't form roving online gangs.

When I was a young lad in the '90s, free software was going to change the world. That kinda half worked.

When I was a young lad in the '90s, the internet was going to facilitate new systems for creating intellectual property without resorting to copyright. Whuffie would be the new unit of currency. Kickstarter is pretty neat. Reddit scares me a little. Anita Sarkeesian had a pretty successful kickstarter, and then so did her opponents.

When I was a young lad in the '90s, blogging was going to change everything. We streamlined it and got facebook and tumblr and memes. The past is now unimaginable.

When I was a young lad in the '90s, the internet offered wonderful new ways of making money with very little startup cost, offering the chance to spread wealth down the long tail. Now agile startups trash whole industries overnight. Etsy has some neat stuff on it though.

When I was a young lad in the '90s, GUIs were making computers more than just a niche interest. Now everyone's learning to code.

Victory, terrible victory, and I'm so lost.


3 November 2014, 11:41 UTCVirtualenv Python+R

The following creates a Python virtualenv that can also hold R libraries.

    virtualenv venv

    mkdir venv/R
    echo 'export R_LIBS=$VIRTUAL_ENV/R' >>venv/bin/activate
    for LIB in venv/lib/python*
        echo 'import os,sys; os.environ["R_LIBS"]=sys.prefix+"/R"' >$LIB/

    echo ". `pwd`/venv/bin/activate && `which R` \$@" >venv/bin/R
    echo ". `pwd`/venv/bin/activate && `which Rscript` \$@" >venv/bin/Rscript
    chmod a+x venv/bin/R venv/bin/Rscript

Python accessed from R using the R package "rPython" will correctly use Python packages installed in the virtualenv. (Note: As at 2014-12-05, rPython is pretty dodgy, floats are only sent accurate to 5 digits by default and strings not correctly quoted. May also need to use jsonlite, and call jsonlite::toJSON with digits=50.)

R accessed from Python with "Rpy2" etc will correctly use R packages installed in the virtualenv.

See also Packrat for R.


24 October 2014, 22:08 UTCSexism: spreading from computer science to biology
Working occasionally with two junior researchers, one female, one male. Wonder if I'm being fair. Both wet/dry, both learning a bit of R+, excellent combo to be developing skills in. Woman is shy, hesitant, but putting in quite a lot of work. Man is confident despite a bad honours year, and taller than me, solidly built. Shows me his code, impressive for someone new to programming, loops and functions and so on. Maybe he has some latent skills from growing up with computers, and I'm seeing his current level rather than rate of growth? Is there some problem in my own manner?

Wednesday I spent time in a meeting pulling apart the woman's statistics; p=0.14, not correcting for multiple testing, why are you pursuing this? She's trying to make the best of poor data, with considerable stubbornness, in the sort of deathmarch project that backs up one under-powered experiment with another under-powered experiment, working with people who think that if you have a time series you don't need replicates, and expect a meaningful comparison between in-vivo brain tissue with a brain-cancer cell line.



21 August 2014, 6:03 UTCFirst-past-the-post voting outcomes tend to surprise the candidates
Candidates, in order to maximise their chance of winning, will take on positions distinct from each other so as to claim as much of the landscape as they can.

I'm assuming the candidates all have fairly similar beliefs about what is actually popular, even if they differ on what platform should win.

The winner is then determined by the difference between the candidates beliefs in what will be popular and the voting public's actual preferences. The winner inhabits a part of the landscape more sparsely populated by candidates than by voters.

It's an interesting property of a not very good voting system!

Looking at the Hugo award system, as the results have just come out: Hugo winners are based on preferential voting and should be good choices (from among the five nominees in each category). However Hugo nominees (with all eligible fiction for that year as candidates) behave more like first-past-the-post and may indicate differences between what authors and publishers think readers want and what readers actually want.

This year we see the amoeba of reader preference more elongated along a liberal/conservative axis than authors expected, perhaps.


21 August 2014, 2:59 UTCDates in Google Search aren't trustworthy
xkcd 979 published 2011 mentions a post by DenverCoder9 in 2003.

Search google for "DenverCoder9" in 2003.

Surprise! Find a link. Web page source code tells us this is a spoof page.

So either this is an extremely elaborate spoof involving a time machine or Google has inferred a publication date from the page text. Very clever, Google is able to give an exact date to a page that it maybe only indexes infrequently, however it is possible for the page to lie and Google will happily believe and report the date it finds, even backdating the page several years.


27 June 2014, 2:22 UTCReading "Practical Foundations of Mathematics"
I am currently reading "Practical Foundations of Mathematics" by Paul Taylor.

It's pretty heavy going, but with considerable aid from Wikipedia I'm about a quarter of the way through without being completely lost (I think). This is pretty good going for a mathematics text for me.

Taylor's attitude is interesting:

Instead of trying to find one mathematical world (set of axioms) in which all the mathematics we are interested in is provable, he asks what minimum set of machinery is required for various ideas. So, for example, various properties of categories can be understood in the context of partially ordered sets (posets), so these ideas are first introduced for posets.

He is considering mathematics as a branch of computer science. Again using the minimum tools required, where possible he gives "intuitionistic" or "constructive" proofs of the existence of mathematical objects. Such proofs can be translated into programs that produce an instance with the properties required, linking proof and computability.

He is also considering mathematics as a human activity. It's not enough to give a foundation of mathematics, he also wants to describe how mathematicians actually proceed and show that the somewhat informal language of mathematics as actually practised can be translated into something rigorous.


18 May 2014, 10:34 UTCCellular automaton tiles revisited
Previously I wrote about implementing 1D cellular automata as tile puzzles.

My previous solution needed four linker tiles. I've now found a way to get rid of these. Each tile simply transmits the state of the tile above it to its left and right neighbours.

These tiles implement Rule 30:

Seed tile:

Starting from this seed in the middle of a line of empty squares, the puzzle has a single, infinite solution. The column of tiles below the seed passes tests of randomness used to test pseudo-random number generators. Rule 30 is used as a random number generator in Mathematica. Rule 30 is a simple demonstration of Stephen Wolfram's observation that mathematics is random, and that "creating information" is trivial.

I previously speculated about making these tiles from plastic. Technology to do this is now readily available.

Source code:

Requires Python 2, "shapely" python library. Produces files for OpenSCAD in a directory "output". Usage:

python 30

You might also like to try other elementary cellular automata, such as Rule 110 which is a computer.


7 April 2014, 8:27 UTCSelfish sweep
A selective sweep occurs when some member(s) of a species gain a massive fitness advantage over the rest of the population. This might be a novel mutation, or some particularly audacious horizontal gene transfer, or a new environmental challenge that only a few members of the population can cope with.

Soon the gene that produced the gain in fitness is present in all surviving members of the species. However, this comes with a loss in genetic diversity. The species' pan-genome is reduced in size. The other genes in the fitter individual(s) might not have been particularly better than those in other individuals, but they get a free ride in the sweep.

Many species can reproduce asexually but also have some form of sex. That is, individuals can clone themselves, but they also have some way of mingling their genes with other members of the species. This might be by sex as practiced by eukaryotes, or some form of horizontal gene transfer as practiced by bacteria. This sex could potentially prevent this loss of genetic diversity. The gene producing the fitness advantage diffuses into the general population.

However, when a selective sweep is occurring, it seems to me there is a huge incentive for a gene to arise in the fitter individuals putting a stop to sexual reproduction or horizontal gene transfer. i.e. under this condition sex, which usually increases the odds of every gene's survival, stops being favourable.

It's a disaster for the species, but the unit of selection is the gene.

Obvious parallels to cultural exchanges, software compatability, etc etc.


3 April 2014, 2:40 UTCBagpipes kickstarter
Donald Lindsay in the UK has a kickstarter to develop 3D printable bagpipe chanter.

He also has various instrument designs for sale at low prices. The designs look nice. The use of sockets probably requires laser-sintered powder printing (Shapeways) rather than filament deposition printing for these designs.

Note: I have a penny-whistle design on thingiverse which can be printed on a filament based printer.


1 March 2014, 5:36 UTCTabor pipes on thingiverse
14 February 2014, 7:19 UTCDemakein: introducing --tweak-emission
12 January 2014, 21:56 UTCAngry White Men
15 December 2013, 10:09 UTCBreathing
11 November 2013, 8:53 UTCReductionism meets Buddhism
9 November 2013, 1:36 UTCDemakein 0.12: more example scripts
25 October 2013, 1:01 UTCWe haven't won, we just got enough power to censor them
28 July 2013, 5:45 UTCMental clock games
11 June 2013, 9:46 UTCHumanity has declined
1 June 2013, 1:40 UTCProgrammer nature
16 April 2013, 9:55 UTCAcetone vapour [detonation] chamber for ABS plastic smoothing
10 April 2013, 8:06 UTCDigital devices for the punk
5 March 2013, 1:39 UTCAlternative architecture: Giant roundhouse
15 February 2013, 13:15 UTCRockstar job market
13 February 2013, 8:04 UTCClostridium perfringens story
12 February 2013, 0:23 UTCThings that Professor Richard Dawkins will never say
23 November 2012, 3:20 UTCMixture-model of politics, with application to climate change
4 November 2012, 22:56 UTCPeer learning and gender discrimination
28 October 2012, 20:30 UTCDemakein 0.2 release: shawms
24 October 2012, 6:36 UTCGains and trade-ups

All older entries


[atom feed]