George Kinsman

Improving Test Observability with Fixie+Seq

Fri, 22 Apr 2022 00:00:00 +0000

For years now it’s been possible to forward logs from Serilog to xUnit’s ITestOutputHelper, but in doing so we lose the structure that makes structured logs great. How can we get the best of both worlds - the advantages of ITestOutputHelper and the analysis capabilities of a local structured log viewer like Seq?

Where are we now?

Using Serilog.Sinks.Xunit, you can view application logs emitted during a test right in your IDE’s test viewer:

code

This is super useful for debugging problems in tests, as we can view just the logs for the failing test without any other noise getting in the way. However, we miss out on one of the prime benefits of structured logging: the structure!

Since this is Serilog, we can plumb the test logs through to a local structured log viewer, like Seq:

code

Fantastic! We can now see and query the structure of the test logs. But we’ve lost something important here - can you spot it?

We can no longer isolate the application logs for a particular test, so if there’s one test that’s hanging for example, we won’t be able to view just the logs leading up to that hang. We’re writing the log output to xUnit’s ITestOutputHelper, but that’s only useful when viewing logs in a test runner - Seq has no idea which test these logs came from.

Additionally, xUnit unfortunately doesn’t currently have a way to run code around a test, requiring you to add code within tests to add this metadata, which I think we can agree isn’t a great state of affairs.

Enter Fixie

Fixie is an alternative test runner for .NET with an ‘emphasis on low ceremony defaults and flexibile customization’.

It inverts the way our tests are run, so that instead of xUnit calling our [Fact]s, we tell fixie how to find and/or execute our tests.

Fixie’s extensibility is in customising discovery and execution. Given that I’m already pretty happy with how Fixie discovers tests (public instance methods on classes ending with Tests), we’re just going to change how it executes them (fixie will auto-discover implementations of this interface):

public class AppTestProject : ITestProject
{
    public void Configure(TestConfiguration configuration, TestEnvironment environment)
    {
        configuration.Conventions.Add<DefaultDiscovery, SingleConstructionExecution>();
    }
}

Execution

So let’s now walk through the code of the custom execution bit by bit - the whole app can be found here, and the code for this section is here.

Implement IExecution which gives us the discovered TestSuite, and we give Fixie a Task back to execute.

class SingleConstructionExecution : IExecution
{
    public async Task Run(TestSuite testSuite)
    {

I use a shared context, but you could create this for each test if you wanted!

var context = new ApiTestContext();

We’re a little too far away from the app execution context here so we can’t use LogContext. Instead, we’ll add a simple enricher that lets us Set props on the global logger, which will enrich every log event. We also override Log.Logger here with our own so that we can customise it just for tests, and to add this enricher.

var mapEnricher = new MapEnricher();

Log.Logger = new LoggerConfiguration()
    ...
    .Enrich.With(mapEnricher)
    ...

Create a test run guid and add it to the enricher so that we can filter by TestRunId later on.
Log an event to bookend the test run

var testRunId = Guid.NewGuid().ToString();
mapEnricher.Set("TestRunId", testRunId);

Log.Information("Beginning test run {TestRunId}", testRunId);

I’m allowing test classes to accept an optional context (the ApiTestContext from before). Fixie provides a helpful abstraction over Activator.CreateInstance in Construct, so we use that.

try {
    foreach(var testClass in testSuite.TestClasses) {
        object instance = testClass.Type.GetConstructors()
            .Any(ctor => ctor.GetParameters().Any())
            ? testClass.Construct(context)
            : testClass.Construct();

So now let’s add TestName and TestClass to the enricher, and wrap the test execution in a SerilogTimings operation so that we can keep track of our test durations over time, and ensure that failed tests appear as errors in the logs.

foreach(var test in testClass.Tests) {
    mapEnricher.Set("TestName", test.Name);
    mapEnricher.Set("TestClass", testClass.Type.Name);
    using var op =
        Operation.At(LogEventLevel.Information, LogEventLevel.Error)
            .Begin("Executing test {TestName}", test.Name);

    var result = await test.Run(instance);
    if (result is Failed failure) op.Abandon(failure.Reason);
    else op.Complete();
}

Cleanup! Bookend the test run, dispose the context, flush the logger.

}
finally {
    Log.Information("Cleaning up test context");
    context.Dispose();
    Log.CloseAndFlush();
}

What do you get?

The ability to slice and dice the entire log output of your app by TestName/TestRunId/TestClass, and no SynchronizationContext shenanigans!:

Testing Background Services and Loops with AutoResetEvent

Mon, 14 Jan 2019 00:00:00 +0000

Updates:

April 2022: I no longer recommend this as a good way to test background services due to its complexity and invasiveness. These days I recommend waiting for side effects to occur from the outside. I’m leaving this post up as a record of what I did do at one point. Thanks for reading!
February 2019 - Fixed a bug with ManualResetAwaiter in which it wouldn’t accurately wait for loops to wrap around. I’ve reworked it slightly to reduce the number of cases it needs to handle.

A common use case for for .NET Core 2.1’s BackgroundService (or its IHostedService interface) is to run a loop that waits for some work to do, and then sleeps for a little while. It might look something like this, using a contrived example of incrementing numbers:

public class MyBackgroundService : BackgroundService {
    public MyBackgroundService(List<int> workToDo) {
        _workToDo = workToDo;
    }

    protected override async Task ExecuteAsync(CancellationToken stoppingToken) {
        while(!stoppingToken.IsCancellationRequested) {
            _workToDo.Add(_workToDo.LastOrDefault() + 1);

            await Task.Delay(_checkWaitTime);
        }
    }
}

While complex work should usually be kept somewhere else outside of the loop, it’s possible that things might get hairy enough that some tests might help us out.

This presents a problem, as there’s no way to control when the loop continues. It might be possible to control the wait duration, but that could lead to unpredictable tests. What we really want is a way to control when the loop continues from the outside.

Enter AutoResetEvent

The .NET Framework has a synchronisation primitive called AutoResetEvent that can act as a handle on a loop. Instead of calling await Task.Delay(TimeSpan), we can call _autoResetEvent.WaitOne(), which then allows us to decide when to continue the loop by calling _autoResetEvent.Set():

_autoResetEvent = new AutoResetEvent(initialState: false);
protected override async Task ExecuteAsync(CancellationToken stoppingToken) {
    while(!stoppingToken.IsCancellationRequested) {
        _workToDo.Add(_workToDo.LastOrDefault() + 1);

        _autoResetEvent.WaitOne();
    }
}
...

// some other thread

_autoResetEvent.Set();

This could be useful for our tests, but first lets put it behind an interface so we can switch the behaviour and make it a bit nicer.

public interface IResetAwaiter
{
    Task Wait(TimeSpan duration, CancellationToken token = default(CancellationToken));
}

Next lets implement a timed version that will run for reals with Task.Delay:

public class TimedResetAwaiter : IResetAwaiter
{
    public Task Wait(TimeSpan duration, CancellationToken token)
    {
        return Task.Delay(duration, token);
    }
}

And then a version that we’ll use for testing that allows us to await until the service continues execution and reaches the Wait call again. The Progress(TimeSpan) method will return false if the work doesn’t complete before the timeout, which we can assert on.

public class ManualResetAwaiter : IResetAwaiter {
	AutoResetEvent _resetEvent;
	SemaphoreSlim _waitEvent;
	TaskCompletionSource<bool> _firstHit;
	
	public ManualResetAwaiter()
	{
		_resetEvent = new AutoResetEvent(false);
		_waitEvent = new SemaphoreSlim(1);
		_firstHit = new TaskCompletionSource<bool>();
	}

	public async Task<bool> Progress(TimeSpan timeout)
	{
		_resetEvent.Set();
		return await _waitEvent.WaitAsync(timeout);
	}

	public Task Wait(TimeSpan duration, CancellationToken token)
	{
		if(_firstHit.Task.IsCompleted) _waitEvent.Release();
		if(!_firstHit.Task.IsCompleted) _firstHit.SetResult(true);
		_resetEvent.WaitOne();
		return Task.CompletedTask;
	}

	public async Task WaitFirstTime()
	{
		await _waitEvent.WaitAsync();
		await _firstHit.Task;
	}
}

We can now update our worker implementation to use this new abstraction:

public MyBackgroundService(List<int> workToDo, IResetAwaiter resetAwaiter) { 
    // set locals 

}

protected async Task ExecuteAsync(CancellationToken stoppingToken) 
{
    while(!stoppingToken.IsCancellationRequested) {
        _workToDo.Add(_workToDo.LastOrDefault() + 1);

        await _resetAwaiter.Wait(TimeSpan.FromMinutes(1), stoppingToken);
    }
}

At runtime, a TimedResetAwaiter can be passed in as a constructor argument via DI or new, and for testing we can pass in a ManualResetAwaiter that gives us control over the loop operation.

The Test

And now we can write a test that looks like this:

[Fact]
public async Task MyBackgroundService_WhenLooping_ShouldBeUnderMyControl()
{
    var awaiter = new ManualResetAwaiter();
    var work = new List<int>();
    var backgroundService = new MyBackgroundService(work, awaiter);

    await backgroundService.StartAsync();

    // Wait until the service starts up and hits the Wait call the first time

    await awaiter.WaitFirstTime();

    // Continue execution and wait until Wait is hit again 

    Assert.True(await awaiter.Progress());
    Assert.Equal(new List<int>() { 1 }, work);
    
    // And again...

    Assert.True(await awaiter.Progress());
    Assert.Equal(new List<int>() { 1, 2 }, work);

    // And again...

    Assert.True(await awaiter.Progress());
    Assert.Equal(new List<int>() { 1, 2, 3}, work);
}

Conclusion

This technique can be useful for controlling any loop that depends on waiting between periods of work.

Pulling out the wait to its own abstraction means that you could also substitute more advanced wait strategies without affecting the accompanying tests.

Typed State Management with Vuex

Thu, 19 Apr 2018 00:00:00 +0000

Update April 2022: Now that Vue 3 has been released, I’d recommend checking out Pinia which is a type-safe, modular store for Vue 2/3 with all the advantages of Vuex and fewer drawbacks.

In this post I’d like to demonstrate a pattern I’ve encountered for doing type-safe, low-overhead state management with Vuex. Vuex is the default, first class state management library for Vue.js. It allows developers to define a state tree that provides getters into the tree, synchronous mutations of the tree, and asynchronous actions that allow multiple mutations. Before looking at those however,

Let’s start with a few goals for state management in general:

Everything should be typed using TypeScript, with noImplicitAny: true
There should be minimal amounts of boilerplate and ‘hooking things up’
The store API should be as simple as possible.
The store should be modular and namespaced, so that properties on different modules don’t conflict.

Vuex alone isn’t very type-friendly, as it wasn’t built from the ground up with types in mind. However, there has been a number of community efforts to add them. Several libraries now exist to achieve this goal, and most of them take the form of API’s that build up a typed store that emit a regular Vuex store for inclusion in the root Vue instance. One such library that we’ll take a closer look at is called vuex-typex.

The full source code for this post can be found here: https://github.com/gkinsman/vue-types-demo.

For these code samples, I’m porting the Vuex shopping cart example over to TypeScript and vuex-typex. The original JS implementation can be found here. I have made only minor model changes for ergonomics.

Getting Started

Let’s install the library:

yarn add vuex-typex

While it isn’t the focus of this article, I think it’s also worth mentioning av-ts as I’ve used it in my sample code - I’ve found it’s the best library for writing components with TypeScript and Vue. Here’s the author’s take on why he wrote it.

The next step is to define an interface that represents the root of the state tree. It combines all of the child states together. There will be a 1:1 mapping of child state to module, as each module will operate on a single child state.

/store/index.ts

import Vue from 'vue';
import Vuex from 'vuex';
import { getStoreBuilder } from 'vuex-typex';
// -- more imports here --

Vue.use(Vuex);

export interface RootState {
    cart: CartState;
    products: ProductsState;
}

const store = getStoreBuilder<RootState>().vuexStore();
export default store;

Once we have the root state defined, we can define each module. Firstly there’s the state interface and it’s initial state:

/store/ProductsModule.ts

export interface ProductsState {
    all: Product[];
}

const initialState: ProductsState = {
    all: [],
};

export const products {
    // our API will be exported here
}

We can now add the special sauce that is vuex-typex. This code uses the library helper getStoreBuilder to define a module with the namespace products whose root state is RootState, initialised with the initialState from above. We’ll use this builder to define getters, mutations and actions. The role of this builder is to capture the context of the store so that call sites can call store methods without needing to pass in the store context. We’ll see this in action soon.

const builder = getStoreBuilder<RootState>().module('products', initialState);

Getters

Using the builder, we can now define a getter that fetches a property from the state.

/store/ProductsModule.ts

// This read function uses the name of the function passed in as the name of the store getter.
const allProductsGetter = builder.read(function allProducts(state: ProductsState) { 
    return state.all; 
});
...
export const products {
    get allProducts() { return allProductsGetter(); },
}

The biggest drawback of this approach is that since builder is capturing the context, it uses the name of the function parameter as the store’s getter name: allProducts. This drawback only applies to getters as we’ll see next.

Mutations

Mutations are functions, and are captured using the builders commit function:

/store/ProductsModule.ts

function decrementProductInventory(state: ProductsState, id: number) {
    const product = state.all.find(p => p.id === id);
    product!.inventory--;
}
...
export const products {
    ...
    decrementProductInventory: builder.commit(decrementProductInventory),
}

Actions

Actions look very similar to functions, except that they may be asynchronous, and use the dispatch function of the builder instead of commit. When deciding between action or mutation, you should choose a mutation unless you want to commit other mutations or do async work.

/store/ProductsModule.ts

/* BareActionContext here gives us access to other modules' state if we need it - also only possible with actions */
async function getAllProducts(context: BareActionContext<ProductsState, RootState>) {
    const shopProducts = await getProducts();
    products.setProducts(shopProducts);
}
...
export const products {
    ...
    getAllProducts: builder.dispatch(getAllProducts),
}

Usage

Using this store API is straightforward: we can import the exported products object directly and access it:

...
import { products } from '@/store/ProductsModule';
...

@Component
export default class ProductList extends Vue {
  get products() {
    return products.allProducts;
  }

  @Lifecycle
  async created() {
    await products.getAllProducts();
  }
}

It works great with the dev tools too, you can see that the cart and product modules are correctly namespaced, and time travel works for each commit to the store:

And that’s it!

Bubble Burst #2 - Heuristics

Sun, 10 Jan 2016 00:00:00 +0000

This is Part 2. Part 1 can be found here

Welcome back! In part one we introduced the bubble burst game, and uncovered some complexities in designing an implementation for beating it. In this post, we’ll be taking a look at a strategy for prioritising the best looking grids and the results of that exploration.

Here’s a small contents to jump to different parts of this post:

A new selection heuristic
Priority Search
Results
The Gauntlet

A new selection heuristic

In part one we had just one selection heuristic which selected the top three best grids. This time, we’ll be adding two new ones - one that selects all the bubbles in the grid, and an interesting new one that eliminates the least common bubbles first.

Least common bubble removal

One of the conclusions from part one was that there is likely to be large pockets of scoring hidden beyond many low scoring moves:

This selection strategy attempts to identify the pockets of scoring by getting rid of the least common colours that have burstable groups available. If we do that then hopefully all that will be left is big groups of colour. Here’s the code for that selection strategy, using the familiar Func<GameMove, IEnumerable<GameMove>> signature:

Func<GameMove, IEnumerable<GameMove>> selectLeastCommonBubblesFirst = move =>
{
    var chosen =
// Statistics is a Dictionary<Bubble, int> that maps colour to total count in grid
        move.GridState.Statistics.OrderBy(x => x.Value)
// We need to make sure we don't choose colours without burstable groups
        .Where(x => move.GridState.Groups.Any(group => group.Colour == x.Key))
// Take two colours to expand the search a little
        .Select(x => x.Key).Take(2);

    var taken = move.GridState.Groups.Where(x => chosen.Contains(x.Colour)).Take(3);
    return taken.Select(x => move.BurstBubble(x.Locations.First()));
};

We’ll take a look at how well this performed a little later.

Select all the groups!

The code for the selection strategy to select all the groups is simply:

	Func<GameMove, IEnumerable<GameMove>> allSelectionStrategy =
                x => x.GridState.Groups
                    .Select(y => x.BurstBubble(y.Locations.First()));

Executing this with the breadth first search becomes a problem, as we’re now adding up to 20 nodes to the breadth queue for each visited node, instead of just three. This leads to rapid memory expansion as the breadth first search visits nodes with the most children first. We’ll still include it in the results to see how well it compares.

Priority Search

Instead of iterating the tree with a breadth first search, let’s switch to using a priority search. The implementation is similar to the breadth first search, except we now use a priority queue instead of a regular queue. I decided to use Denis Shulepov’s implementation instead of writing my own.

To prioritise the order of the nodes in the queue, we need to be able to score each game position. To recap from part one, this is what our GameMove class looks like at the moment:

public class GameMove {
    public IList<PointAndColour> Moves { get; private set; }
    public int Score { get; }
    public Grid GridState { get; }
}

The priority queue implementation uses IComparable to order items, so we’ll need GameMove to implement IComparable<GameMove>.

Grid Scoring

Next, we need a way to score grids so that both deep grids with good scores and shallow grids with not-yet-good scores are equally prioritised. This is to make sure we keep searching random parts of the tree if a particular branch stops yielding a good depth-to-score ratio.

Let’s start with the easy cases where shallower grids with better scores return -1, and deeper grids with lower scores return 1. Equal grids return 0.

int CompareTo(GameMove other)
{
    var thisMoveCount = this.Moves.Count;
    var otherMoveCount = other.Moves.Count;

    if (thisMoveCount < otherMoveCount && Score > other.Score) return -1;
    if (thisMoveCount > otherMoveCount && Score < other.Score) return 1;
    if (thisMoveCount == otherMoveCount && Score == other.Score) return 0;
...

Depth Penalty

We then need to decide on how much to penalise grids for their depth level. To do that we can apply a negative score, calculated by subtracting the depth multiplied by the depth penalty from the game score.:

gridScore = abs(score - (depth * depthPenalty))

The absolute value of the calculation is used to prevent game grids from having negative scores which occurs with low scoring, high depth grids. Allowing them to have negative scores will make them very unlikely to ever be visited which is undesirable, as it turns out that some of those grids can lead to the deep pockets of scoring we discussed earlier.

So to finish off the CompareTo function:

  
...
    var thisScore = Math.Abs(Score - (thisMoveCount * depthPenalty));
    var otherScore = Math.Abs(other.Score - (otherMoveCount * depthPenalty));

    if (thisScore == otherScore) return 0;
    return thisScore > otherScore ? -1 : 1;
}

A higher value for depthPenalty means that deeper positions will require more points to beat a shallow one, whereas a lower value means that deeper positions with lower scores might get too much attention.

To find out what the ideal depth penalty is, I ran the solver across three different game grids with depth penalties between 7 and 18:

These results suggest that decreasing the likelihood of visiting deeper nodes (via higher depth penalty) forces the solver to choose more likely candidates from shallower depths, thereby improving the scores.

Additionally, while a higher depth penalty is better, the level of benefit is also dependant on the game grid - a better algorithm might dynamically change the penalty based on the rate of top-score discovery or some other mechanism.

It’s perhaps worth noting at this point that one of the harder parts of this project is analysing results from the solver and comparing them to past results to see if it has improved. I’ve been using a script that parses solver output and generates an xlsx which I can plug into Power Pivot, which is built into Excel. The benefit of this is that it allows me to slice/filter the data by any of the available dimensions (Score, BoardId, Depth, DepthPenalty, Elapsed, TotalMoveCount) to produce these graphs.

Results!

These results were produced by running each of the selection strategies for both the breadth first and priority first searches, on each of three game grids.

The program was run for a maximum of 50,000 visited nodes for each scenario. 50,000 is a somewhat arbitrary number, but not many high scores are found after that point and it’s enough nodes that it captures a lot of interesting data.

For the priority search, a depth penalty of 18 was used.

Score

We’ll compare the maximum scores of each approach first, and then later on we’ll take a look at their speed/efficiency.

Breadth First

Taking a look at the breadth first search results, we can see that overall the top three selection strategy appears to have the best performance by quite a lot on grids 1 and 2, but actually loses on grid 3 to the LeastCommonBubblesChosen strategy.

The optimal selection strategy also appears to be very dependent on the game board, which is likely because breadth-first doesn’t adapt its ordering to the characteristics of the grid like the priority first method does. Breadth-first iterates over nodes in a fixed order, whereas the priority search is always evaluating its next move.

Priority First

It’s safe to say that the priority first search outperforms the breadth first search in every case.

This data also appears to indicate that grid #3 has the most uniform tree structure of the bunch, as its scores don’t change much between strategies.

For the LeastCommonBubbleChosen selection strategy we had a 341% increase over the breadth-first equivalent for grid #1!

Speed

Taking a look at the elapsed time for all breadth vs priority strategies, we can get an idea of how long it takes each method to reach its top scores.

Aside from being nice and colourful, this graph shows that the priority search is able to surpass breadth first’s all-time high scores within the first four seconds.

Efficient use of searched nodes

Speed is an acceptable metric for determining the effectiveness of an algorithm, but it’s accuracy and consistency is susceptible to things like CPU speed, other system activity, etc. Thankfully we have an absolute measure in the number of nodes searched to reach a certain score. By comparing the number of nodes searched to the maximum score obtained, we can determine how efficient an algorithm is.

Again we can see that the priority search with the LeastCommonBubbles strategy is the most efficient in finding the highest score. It’s interesting to note that high scores are usually found in groups, which is because a single move of 150 points with a depth penalty of 18 will win many of its children a place in the front of the queue.

Throwing down the gauntlet

Now that I’ve had a go, I challenge you to beat my scores! I’ve no doubt that there are better strategies for finding the best heuristics, so hopefully we’ll see some interesting solutions pop up.

A couple of alternative ideas that might work:

Monte carlo tree search
- Uses a random sampling of child nodes to inform decisions on where to search next
Use machine learning to train a program to detect high value grids
Depth-first search with branch elimination
Use your imagination, most of the work’s been done for you!

The code

The code is available on github. Pull requests are welcome, or just fork it and go wild!

For Fun

For fun, here is an animation of the top scoring playthrough. It used a priority search on Grid #1 with the LeastCommonBubble selection strategy:

Here are some screenshots of the highest scoring moves:

Bubble Burst #1 - Implementation

Thu, 17 Dec 2015 00:00:00 +0000

Bubble Burst is an interesting and (somewhat) fun game that’s available in most app stores in some form or other. It looks something like this (this particular game is called bubble poke on android):

The way the game works is that you tap on a group of two or more ‘bubbles’ and you then receive a score that is calculated by taking the number of bubbles in the group you popped multiplied by that same number minus one:

score = bubblesInGroup * (bubblesInGroup - 1)

There’s a clear benefit to popping as large a group as possible:

Once the game has begun there are no new bubbles that fall down or appear. A nearly completed game might look something like this:

Once a bubble group is ‘popped’, the grid adjusts itself to account for the holes left behind. Where there is a gap above a bubble, the bubbles above the gap are pushed down. If there are no more bubbles in a column then the column is removed and the column to its left is moved one place to the right. In short, bubbles are pushed down to the bottom right hand corner of the grid.

So what?

The reason Bubble Burst is interesting is that the ‘no new bubbles’ feature of the game makes it deterministic, which means that you could write a program to beat it that doesn’t have to deal with any new information. That’s what we’re gonna do.

Implement it!

In considering possible implementations of the game, an initial approach might be to create a jagged or multidimensional array to store the bubbles. The bubble colours could be represented as an enum:

enum Bubble {
	Cyan,
	Yellow,
	Green,
	Red,
	Blue
}

var bubbleGrid = new Bubble[][];

popBubble(Bubble[][], Point point) {
	// 1. Group the bubbles
	// 2. Remove the group that point belongs to
	// 3. Reflow the grid
}

The program will need to place the bubbles into groups which can then be used to ensure that only valid moves are made, and to calculate the scores of moves. Once that’s done we can implement the ‘Pop’ function which removes a group of bubbles and increments the score counter. The algorithm for that should be straightforward: push the bubbles down and to the right.

Ok, sounds good so far. What are we trying to accomplish though? We aren’t trying to implement Bubble Burst - we’re trying to beat it. Each time a group of bubbles is popped, the grid is changed and an entirely new set of outcomes becomes possible.

We need a way to undo a previous move to choose another one, or else we’ll be stuck either playing a game the entire way through for each set of possible moves, or replaying the game to that point and then choosing an alternate group.

Maybe we don’t want to change the grid after all…

An alternative implementation could, instead of mutating the grid, take a copy of it each time and pop the bubble group in the copy. If we stored each grid as a node in a tree then we could search the tree to find the leaf with the highest total score. This sounds like progress!

How many grids are we going to be storing though? How much memory is that going to use?

Some quick back of the envelope math…

In .NET arrays are 12 bytes each. We’re storing a grid of 10x10 so we’ll need ten arrays, and enums are just int’s which are 4 bytes. That comes to 120 bytes for the arrays, and 400 bytes for the ints, for a total of 520 bytes per grid.

Calculating the number of nodes at each depth is harder - we’ll need to estimate. There’s about 20 groups in our top level grid, which will reduce by one with each level. We’ll also need to assume that the tree is uniform accross its leaves, which is quite unlikely but we can’t do much better without iterating over it.

estimatedMovesRemaining = movesAtTop - currentDepth
nodesAtLevel = estimatedMovesRemaining * nodesAtPreviousLevel

nodesAtLevel(2) = (20 - 2) * 20
		= 380

Depth	# Nodes
1	20
2	380
3	6840
4	116,280
5	1,860,480
6	27,907,200
7	390,700,800
8	5,079,110,400
9	60,949,324,800
10	670,442,572,800

At only 10 levels deep we have an estimated 670 billion possible grid states. Even if our estimate is out by an order of magnitude, we know that games can reach a depth of up to 20.

670 billion * 520 bytes = ~317 TB

So now that we know we can’t iterate over the entire tree, we’ll need to use heuristics to trim our search space and prioritise grid positions that are more likely to yield high scores.

A simple heuristic

As a first step, let’s take a look at a simple heuristic which trims the tree by taking the top three groups by score. In my current implementation this kind of heuristic is defined as a function that accepts a GameMove as a parameter and produces a list of new moves. It projects the parent node into a set of child nodes.

public class GameMove { 
    public IList<PointAndColour> Moves { get; private set; }
    public int Score { get; }
    public Grid GridState { get; }
}

Func<GameMove, IEnumerable<GameMove>> topThreeSelectionStrategy =
        parent => parent.GridState.Groups
            .OrderByDescending(group => group.Score).Take(3)
            .Select(group => parent.BurstBubble(group.Points.First()));

This function can then be plugged into a Tree to generate nodes. Given a root node and a child node creation strategy, we can lazily iterate over the nodes to find the best one:

var topScore = 0;
foreach(var node in new LazyGeneratedTree<GameMove>(root, topThreeSelectionStrategy)) {
    if(node.Score > topScore) topScore = node.Score;
}

Running a breadth first search over the tree using this strategy produces the following result:

Further high scores were found after some time, though no higher scores were found before I stopped it at 30 minutes:

Score	Depth	Time	Nodes Searched
202	10	1m21s	40396
204	10	2m3s	46264
206	10	2m8s	46936
208	11	11m47s	117751
212	11	11m52s	118357
218	11	11m57s	119110
224	11	14m44s	138793

From this we can draw a couple of conclusions:

Even though we’ve trimmed the tree, there’s still too many nodes. Cutting out more nodes isn’t an option, as even by taking just the top three we’ve cut out a large part of our search space. It’s quite possible that there are pockets of high scoring nodes hidden beyond low scoring paths.
There was only a couple of higher scores found per depth level. This might suggest that there could be significant gains to be made if we could prioritise more promising branches over less promising ones.
A breadth first search is probably not the best way to search this tree, as we’re spending a lot of time evaluating nodes that are below the high score. Prioritising more likely candidates could be a good alternative.

That’s it for now! In part two of this blog series we’ll take a look at some more interesting heuristics involving a priority tree search, as well as an optimisation to reduce the memory usage of the tree nodes.

Note: I’ll be releasing all the code for this after part 2, but if you have ideas for heuristics using the above model, send me a Gist and I’ll plug it in.

Part Two can be found here