OneEye

A program to extract moves of a go game from a video and save them to SGF or broadcast them online.

The basic prototype is already working. However, it requires the user to manually mark the board. This is bothersome and more importantly, the same would be required again for any possible movement of the camera or the board. Which is unavoidable.

Position detection

Therefore our current research focuses on automatic position detection. It is planned as a sequence of three stages.

Sansa

First detector approximately locates the board in the picture. It uses SSD (single shot detector) with MobileNet and is working decently.

Examples (downloaded from Flickr under CC0 license):

Dochi

Then the cropped and normed image is fed to the second detector, designed to return precise grid corners' coordinates. This is currently implemented by a convolutional neural network and obtaining the required precision is a major hurdle to be passed.

Pretty good:

Also good, but not enough:

Genjo

With grid corners it is not difficult to rectify the image and cut it into tiles, one for each intersection. Then a third stage, Genjo classifier, decides whether the intersection is empty or occupied by a black or white stone. This is currently realized by a sample of pixels voting according to their HSI intensity and it suffices to produce satisfying results. Further improvements should be relatively easy.

Stringing a sequence of board states to a sequence of moves

Base case: we have two correctly recognized positions differing by a single move (single added stone). Easy.

Issues:

• illegal positions -> ignorable
• positions unreachable from the previous state
  • reachable from any past state. (Incorrect states inbetween.) How to pick the correct leaf of such a tree?
  • reachable by more than one move. (Lost states inbetween.) Issues with branching factor.
• stone shifts
  • stone stops being recognized -> fixable manually and even ignorable
  • stone is recognized at an empty intersection. It can be occupied later for real. What do?

This is handled by the statebag package and its documentation also states the problem and used solution in precise terms.