Improving performance for a TM simulator - c++

Question

I am trying to simulate a lot of 2 state, 3 symbol (One direction tape) Turing machines. Each simulation will have different input, and will run for a fixed number of steps. The current bottleneck in the program seems to be the simulator, taking a ton of memory on Turing machines which do not halt.
The task is to simulate about 650000 TMs, each with about 200 non-blank inputs. The largest number of steps I am trying is 1 billion (10**9).
Below is the code I am running. vector<vector<int> > TM is a transition table.
vector<int> fast_simulate(vector<vector<int> > TM, string TM_input, int steps) {
/* Return the state reached after supplied steps */
vector<int> tape = itotape(TM_input);
int head = 0;
int current_state = 0;
int halt_state = 2;
for(int i = 0; i < steps; i++){
// Read from tape
if(head >= tape.size()) {
tape.push_back(2);
}
int cell = tape[head];
int data = TM[current_state][cell]; // get transition for this state/input
int move = data % 2;
int write = (data % 10) % 3;
current_state = data / 10;
if(current_state == halt_state) {
// This highlights the last place that is written to in the tape
tape[head] = 4;
vector<int> res = shorten_tape(tape);
res.push_back(i+1);
return res;
}
// Write to tape
tape[head] = write;
// move head
if(move == 0) {
if(head != 0) {
head--;
}
} else {
head++;
}
}
vector<int> res {-1};
return res;
}
vector<int> itotape(string TM_input) {
vector<int> tape;
for(char &c : TM_input) {
tape.push_back(c - '0');
}
return tape;
}
vector<int> shorten_tape(vector<int> tape) {
/* Shorten the tape by removing unnecessary 2's (blanks) from the end of it.
*/
int i = tape.size()-1;
for(; i >= 0; --i) {
if(tape[i] != 2) {
tape.resize(i+1);
return tape;
}
}
return tape;
}
Is there anywhere I can make improvements in terms of performance or memory usage? Even a 2% decrease would make a noticeable difference.

Answer 1

Make sure no allocations happen during the whole TM simulation.
Preallocate a single global array at program startup, which is big enough for any state of the tape (e.g. 10^8 elements). Put the machine at the beginning of this tape array initially. Maintain the segment [0; R] of the all cells which were visited by the current machine simulation: this allows you to avoid clearing the whole tape array when you start the new simulation.
Use the smallest integer type for tape elements which is enough (e.g. use unsigned char if the alphabet surely has less than 256 characters). Perhaps you can even switch to bitsets if alphabet is very small. This reduces memory footprint and improves cache/RAM performance.
Avoid using generic integer divisions in the innermost loop (they are slow), use only divisions by powers-of-two (they turn into bit shifts). As the final optimization, you may try to remove all branches from the innermost loop (there are various clever techniques for this).

Answer 2

Here is another answer with more algorithmic approaches.
Simulation by blocks
Since you have tiny alphabet and tiny number of states, you can accelerate the simulation by processing chunks of the tape at once. This is related to the well-known speedup theorem, although I suggest a slightly different method.
Divide the tape into blocks of 8 characters each. Each such block can be represented with 16-bit number (2 bits per character). Now imagine that the machine is located either at the first or at the last character of a block. Then its subsequent behavior depends only on its initial state and the initial value on the block, until the TM moves out of the block (either to the left or to the right). We can precompute the outcome for all (block value + state + end) combinations, or maybe lazily compute them during simulation.
This method can simulate about 8 steps at once, although if you are unlucky it can do only one step per iteration (moving back and forth around block boundary). Here is the code sample:
//R = table[s][e][V] --- outcome for TM which:
// starts in state s
// runs on a tape block with initial contents V
// starts on the (e = 0: leftmost, e = 1: rightmost) char of the block
//The value R is a bitmask encoding:
// 0..15 bits: the new value of the block
// 16..17 bits: the new state
// 18 bit: TM moved to the (0: left, 1: right) of the block
// ??encode number of steps taken??
uint32_t table[2][2][1<<16];
//contents of the tape (grouped in 8-character blocks)
uint16_t tape[...];
int pos = 0; //index of current block
int end = 0; //TM is currently located at (0: start, 1: end) of the block
int state = 0; //current state
while (state != 2) {
//take the outcome of simulation on the current block
uint32_t res = table[state][end][tape[pos]];
//decode it into parts
uint16_t newValue = res & 0xFFFFU;
int newState = (res >> 16) & 3U;
int move = (res >> 18);
//write new contents to the tape
tape[pos] = newValue;
//switch to the new state
state = newState;
//move to the neighboring block
pos += (2*move-1);
end = !move;
//avoid getting out of tape on the left
if (pos < 0)
pos = 0, move = 0;
}
Halting problem
The comment says that TM simulation is expected either to finish very early, or to run all the steps up to the predefined huge limit. Since you are going to simulate many Turing machines, it might be worth investing some time in solving the halting problem.
The first type of hanging which can be detected is: when machine stays at the same place without moving far away from it. Let's maintain surrounding of TM during simulation, which is the values of segment of characters at distance < 16 from TM's current location. If you have 3 characters, you can encode surrounding in a 62-bit number.
Maintain a hash table for each position of TM (as we'll see later, only 31 tables are necessary). After each step, store tuple (state, surrounding) in the hash table of current position. Now the important part: after each move, clear all hash tables at distance >= 16 from TM (actually, only one such hash table has to be cleared). Before each step, check if (state, surrounding) is already present in the hash table. If it is, then the machine is in infinite loop.
You can also detect another type of hanging: when machine moves to the right infinitely, but never returns back. In order to achieve that, you can use the same hashtables. If TM is located at the currently last character of the tape with index p, check current tuple (state, surrounding) not only in the p-th hashtable, but also in the (p-1)-th, (p-2)-th, ..., (p-15)-th hash tables. If you find a match, then TM is in infinite loop moving to the right.

Answer 3

Change
int move = data % 2;
To
int move = data & 1;
One is a divide, the other is a bitmask, both should give 0 or 1 base on the low bit. You can do this anytime you have % by a power of two.
You're also setting
cell = tape[head];
data = TM[current_state][cell];
int move = data % 2;
int write = (data % 10) % 3;
current_state = data / 10;
Every single step, regardless of whether tape[head] has changed and even on branches where you're not accessing those values at all. Take a careful look at which branches use which data, and only update things just as they're needed. See straight after that you write:
if(current_state == halt_state) {
// This highlights the last place that is written to in the tape
tape[head] = 4;
vector<int> res = shorten_tape(tape);
res.push_back(i+1);
return res;
}
^ This code doesn't reference "move" or "write", so you can put the calculation for "move"/"write" after it and only calculate them if current_state != halt_state
Also the true-branch of an if statement is the optimized branch. By checking for not the halt state, and putting the halt condition in the else branch you can improve the CPU branch prediction a little.