My own opinion on the test suite is as follows: The positions are too easy in the sense that good programs find the correct moves in almost all positions in a few seconds using their midgame search and evaluation. On the other hand, a standardized set of test positions makes it possible to size up the competition. I have used the test suite to evaluate modifications of Zebra's endgame code.
If you are a new programmer and have only recently started working on an Othello program: These positions are hard, it takes 2 hours and 6 minutes for Zebra to solve all positions for exact score on a Thunderbird/1333. To the best of my knowledge, this is the best performance reported for the test.
All the positions: ffotest.zip
| | | |
| #40, black to move | #41, black to move | #42, black to move | #43, white to move |
| Turner vs Monnom | Eclipse vs Logistello | Penloup vs Shaman | Brightwell vs Suekuni |
| Bruxelles 1997 | Internet (7-12) 1996 | Cambridge 1998 | WC 1997 (final) |
| | | |
| #44, white to move | #45, black to move | #46, black to move | #47, white to move |
| Shaman vs Tastet | Tamenori vs Shaman | Caspard vs Juhem | Brightwell vs Tastet |
| WC 1995 | WC 1995 (final) | FC 1994 | WC 1997 |
| | | |
| #48, white to move | #49, black to move | #50, black to move | #51, white to move |
| Brightwell vs Tastet | Lazard vs Cali | Nicolet vs Feinstein | Tastet vs Caspard |
| Paris Open 1997 | FC 1991 | WC 1996 | WC 1997 |
| | | |
| #52, white to move | #53, black to move | #54, black to move | #55, white to move |
| Edmead vs Lazard | Nicolet vs Murakami | Logistello vs Brutus | Logistello vs Hannibal |
| Cambridge 1996 | WC 1996 (final) | IOS Tournaments 1997 | Internet (1-6) 1997 |
| | | |
| #56, white to move | #57, black to move | #58, black to move | #59, black to move |
| Logistello vs Hannibal | Taniguchi vs Ralle | Logistello vs Murakami | Tastet vs Parsons |
| Internet (1-6) 1996 | WC 1984 (final) | Match 1997 | MSO 1997 |
Abbreviations:
Note: For some positions, there are several optimal moves:
Zebra configuration (thanks to Dan Lidström, he performed the test):
Complete logs are available here, a summary is given below:
| # | Date | Score | Time lock | Time exact | Nodes exact | NPS | Line |
|---|---|---|---|---|---|---|---|
| 40 | 01/02/03 | +38 | 0.8s | 2.4s | 19.2M | 8026k | a2 b1 c1 |
| 41 | 01/02/03 | +0 | 0.1s | 3.2s | 20.8M | 6600k | h4 a3 a2 |
| 42 | 01/02/03 | +6 | 0.8s | 3.6s | 28.6M | 7912k | g2 h1 c2 |
| 43 | 01/02/03 | -12 | 0.1s | 3.9s | 23.7M | 6163k | c7 h4 h5 |
| 44 | 01/02/03 | -14 | 0.1s | 3.7s | 22.3M | 5950k | d2 g5 b8 |
| 45 | 01/02/03 | +6 | 2.3s | 58.7s | 472M | 7993k | b2 c1 g5 |
| 46 | 01/02/03 | -8 | 1.2s | 9.9s | 66.6M | 6685k | b3 c1 b1 |
| 47 | 01/02/03 | +4 | 0.1s | 4.0s | 23.6M | 5920k | g2 b8 b7 |
| 48 | 01/02/03 | +28 | 0.2s | 53.3s | 361M | 6794k | f6 g5 g3 |
| 49 | 01/02/03 | +16 | 0.3s | 48.5s | 380M | 7850k | e1 h4 g6 |
| 50 | 01/02/03 | +10 | 0.6s | 294s | 2.09G | 7000k | d8 e8 g8 |
| 51 | 01/02/03 | +6 | 0.3s | 51.5s | 324M | 6267k | e2 h2 g7 |
| 52 | 01/02/03 | +0 | 29.8s | 54.9s | 338M | 6241k | a3 f2 e1 |
| 53 | 01/02/03 | -2 | 3.8s | 430s | 2.99G | 6893k | d8 b6 a5 |
| 54 | 01/02/03 | -2 | 123s | 657s | 4.62G | 7000k | c7 g6 g5 |
| 55 | 01/02/03 | +0 | 0.9s | 2411s | 14.9G | 5998k | g6 h7 c1 |
| 56 | 01/02/03 | +2 | 0.5s | 267s | 1.31G | 4856k | h5 h4 h3 |
| 57 | 01/02/03 | -10 | 167s | 406s | 2.73G | 6755k | a6 b5 b4 |
| 58 | 01/02/03 | +4 | 0.6s | 465s | 2.53G | 5510k | g1 f2 h3 |
| 59 | 01/02/03 | +64 | 0.5s | 0.7s | 1.4M | 2079k | g8 |
Abbreviations:
Last modified November 22, 2004 by Gunnar Andersson
gunnar@radagast.se