Chess Feature/Review

A leading chess expert shows you how to test and evaluate three competing programs.

Volume 1

Number 10

December 1983

Chess for three gives the White Knight a winning gambit

By ALEX BELL

ALEX BELL is the author of "Games Playing with Computers" and "The Machine Plays Chess?" Here he explains how you can test a chess program with some of the methods he has used over the last 20 years.

DID you know that there are 14,772,512 ways of placing 16 queens on a 16 by 16 chess board so that no queen attacks any other? Or that you can mate a black king on an infinite board with a white king, rook, bishop and knight?

Or that you can be a rotten chess player (like me) and still test and evaluate chess programs better than Bobby Fischer?

I was asked to comment on three BBC chess programs Acornsoft Chess (£9.95), BBC Soft White Knight (£9.95) and Computer Concepts Chess (£8.95).

If you already have a chess program that allows you to set up positions then you can compare it with these (and other) programs using the following standard tests.

TEST 1: MATE TIMER

Ask the program to find two and three move mates from the opening position. The results were (in seconds):

 

2 move

3 move

BBC Soft

<1

7

Acornsoft

9

257

Concepts

36

>600

BBC Soft was by far the fastest to report that there is no solution. Computer Concepts does not have a specific "mate in N" mode and has to be set to look at all moves at all levels down to three and five ply (a ply is a move by black or white). It chose P-K4 for a full three ply search and I got tired of waiting for the five ply response.

Acornsoft was intermediate but later testing showed that it does consider underpromotion which BBC Soft does not.

Testing the programs with genuine two move mate problems gave an average of about two seconds for BBC Soft, 50 seconds for Acornsoft and interminable for Concepts.

I was impressed by BBC Soft's speed and compared its performance using a problem that I have run on many other computers. The oldest machine was an ICL Atlas (circa 1963 and costing £3 million) with a program written in machine code which took four seconds twice as long as a £10 program on a £400 micro. But the cycle times of the two machines are comparable each could/can add about half a million numbers a second.

TEST 2: OPENING TRAPS

I always play black because there is more opportunity to lay traps and none of the programs have a book to protect them from errors. All tests gave the programs about one minute on average to play their moves. The first game was against the Acornsoft which, without any forcing, played:

1. P-K4 P-K4
2. N-KB3 N-QB3
3. B-B4

A Giuoco Piano and black has some nice traps, for example the Blackburne shilling game where you offer up a pawn with 3. N-Q5 I have never seen a program refuse this one unless protected by a book so (See Photo II):

4. NxP Q-N4

Dumb programs play NxP forking the queen and rook but QxNP threatens mate in one and white is in deep trouble. Acornsoft played:

5. BxP+ K-K2
6. BxN QxNP
7. N-B6+ KPxN
8. R-B1 QxKP +
9. Q-K2 QxQ+ +

Move seven is the classic "horizon effect" - anything to put off the evil check and checkmate. The best line at move six is to castle (and lose the knight - BQxN) followed by P-KB4.

To test BBC Soft I forced the first three moves and it played:

4. NxP Q-N4

as expected. However one of the nice features of BBC Soft is that it displays the "Best line" - what it thinks is its best move and what the opponent will probably reply down to four or five ply. It clearly saw that QxNP was going to be trouble but it played the same line as Acornsoft:

5. BxP+ K-K2
6. BxN QxNP
7. P-Q3 KxR +

An unusual move to sacrifice the rook but it does give the black king a bolt hole.

8. K-Q2 QxQ
9. KxQ RxB

and it's still playing, whereas Computer Concepts was just dumb:

5. NxBP QxNP
6. R-B1 QxKP +
7. B-K2 N-B3 + +

Another trap which can be tried from white opening Giuoco Piano is the Wilkes-Barre line, thus:

1. P-K4 P-K4
2. N-KB3 N-QB3
3. B-B4 N-B3

which invites white to attack the weak KBP with N-N5. In fact, none of the programs would play this move (although it is all right if you know what you are doing) so I forced it on all of them. Concepts played:

4. N-N5 B B4
5. NxBP BxP +
6. KxB NxP +
7. K-B3? Q-B3 +
8. KxN P-Q4 +
9. BxP Q-B5 +
10. K-Q3 Q-Q5 +
11. K-K2 B-N5 +
12. K-K1 BxQ

BBC Soft also played 5. NxBP inevitable from move six when the king moved out towards the middle of the board.

White Knight also played 5. NxBP but the B x P+ made it think for over five minutes. It can do this because it only has to average one minute for each move (whereas Concepts sticks rigidly to its time for each move).

The "Best line" showed that it considered K-B1 for over half of the five minutes because the knight check is difficult to analyse. However greed overcame caution:

6. KxB NxP +
7. K-K3 Q B3

This is better than Concepts but still greedy - if white retires the king it cannot have the black rook because of Q-B3, which either checks or threatens mate in one. But the best line is K-N1 because of the skewer or the following:

8. NxR Q N4 +
9. KxN PQ4 +
10. KxP N-N5 +
11. K-B5 Q-K2 +
12. K-N5 B-Q2 +
13. K-R5 Q-B4 + +

Acornsoft was the only program to avoid the trap with the right move:

6. K-N1 Q-K2
7. NxR

and black has a poor game.

TEST 3: END GAME PROBLEMS

Comparison of end game abilities is difficult because the time settings on the programs differ. Concepts always sticks rigidly to its one minute for each move whereas Acornsoft has playing levels which have to be set higher for an end game to play at the test speed.

BBC Soft only averages its moves to one a minute giving it a definite advantage in difficult positions.

All the programs appeared capable of performing the king, queen versus king ending (Acornsoft needs level six or seven for this and announces its move with a chime) but the king, rook versus king ending was too difficult for all of them.

They all tend to move the king to the centre of the board but then have no idea what to do with it.

An interesting question is how do these micro-programs compare with the tournament programs running on large mainframe computers?

Consider the position in Photo I, a famous problem which has baffled many people since the turn of the century and a good test of a program's appreciation of repetition of position.

Mainframe programs circa 1975 were just reaching the point where they could solve such problems because they could be set to a "play and win" mode, appreciating that repetition is a draw and underpromoting the pawns.

This is the way it worked. The pawn is given a value equal to a queen a fiddle but nevertheless equivalent to what a human expert does when asked to solve the problem.

The only move that can keep this valuable pawn is P-B7 so the program should play it without any further investigation of the consequences.

To misquote Sherlock Holmes: "When all possibilities except one are eliminated then the exception must be the answer".

Black must now check the white king, and white, in order to avoid draw by repetition and avoid losing the pawn with a skewer, must move steadily down the knight file until it can play K-QB2 which prevents the skewer threat.

Black might then play R-Q5 which threatens stalemate (if white queens the pawn then R-QB5+) but tournament programs in 1975 could search 11-ply ahead and see that the underpromotion to a rook is a win for white.

None of the micro-programs (playing black or white) had any idea of how to play this ending although BBC Soft does have a crude appreciation of repetition of position and Acornsoft can underpromote.

I give this example in order to justify my opinion that these programs' playing abilities are roughly equivalent to the state of the art about 10 years ago on the most powerful and expensive computers.

They are incredibly cheap in comparison and the modern micro is about 10 per cent of the speed of the giants of 10 years ago.

But they are obviously still limited, mainly by their store size (hence the lack of book openings and knowledge of how to play end games).

The store size will increase on the BBC Micro when the Tube arrives. This means that the display handling (which occupies about 10k in BBC Soft-about one third of the machine) can be done by the second processor and the programs should then improve their openings and end games dramatically.

However, it is invidious to compare the modern micro with its giant ancestors on purely chess playing ability.

The micro, with its interactive features and display monitor, can and does support vastly superior display and manipulation features over the old programs.

Ease of setting up positions and selecting options are important factors in designing micro chess programs and the display of information is also very important.

For example, BBC Soft display of the "Best line" is an inspiration by its author, 23-year-old Martin Bryant, and a feature that will become a must for future chess programs.

To actually see what the program is thinking about while you wait for its move is quite fascinating, particularly when testing it with traps.

Another nice feature of BBC Soft is the simple setting of its playing speed to an average time for each move. When in trouble the program reacts in an intelligent fashion by taking longer to make its reply.

CONCLUSIONS

Concepts compared poorly with the other programs in almost every test and option. In terms of playing strength Acornsoft and BBC Soft are about equal, with Acornsoft rather easier to manipulate and with rather more options. It (and Concepts) allows you to save and reload a position with a filing system, an option BBC Soft's does not provide.

Nevertheless BBC Soft's "Best line" together with its very fast problem solving mode must give it the edge in terms of entertainment and value for money.

Photo I: Acornsoft Chess

Photo II: BBC Soft White Knight

Photo III: Computer Concepts Chess