A definition of "grok"

homeblogmastodonthingiverse



Modelling large systems is computationally hard, it involves estimating many model parameters -- an optmization problem in high dimensions. Finding such models is an "ah-ha" moment, what Heinlein dubbed "grokking".

Optima often nestle in steep walled valleys. The optimization must align itself with the direction of the valley, or always be bumping into its sides. In high dimensions, there are many many possible diagonals. Optimization is hard. Human ability to model systems of even moderate complexity far exceeds what can be done on a computer.

Two types of system i'm currently interested in:

Other people seem to be naturally good at grokking these types of things, but i seem to need to consciously direct my attention to them.

I find that once i have the appropriate mental posture of attention -- which details to observe, which to ignore -- non-obvious patterns emerge. It's like performing optimization in a sub-space, working on just a few dimensions of an otherwise intractibly complex system at a time. Not guaranteed to work, there'll be very diagonal valleys that i miss, but it turns up some interesting stuff.




[æ]