Memory Alignment Issues with GCC Vector Extension - c++

Question

I'm trying to use GCC vector extension (https://gcc.gnu.org/onlinedocs/gcc/Vector-Extensions.html) to speed up matrix multiplication. The idea is to use SIMD instructions to multiply and add four float numbers at once. A minimal working example is listed below. The example works fine when multiplying a (M=10,K=12) matrix to a (K=12,N=12) matrix. When I change the parameters (say N=9), however, I get a segmentation fault.
I suspect this is due to memory alignment issues. In my understanding, when using a SIMD for a vector wich 16bytes (in this case float4), the target memory address should be a multiple of 16. There are already discussions on memory alignment issues with SIMD instructions. (e.g. Relationship between SSE vectorization and Memory alignment). In the example below, when &b(0,0) is 0x810e10, &b(1,0) is 0x810e34, which is not a multiple of 16.
My questions are,
Is it true that I'm getting the segfault for the memory alignment issues?
Can anyone tell me how to fix the problem easily? I've thought of using a two-dimensional array instead of one array, but I don't want to do this so as not to change the rest of the codes.
Minimal Working Example
#include <iostream>
#include <cstdlib>
#include <stdio.h>
#include <cstring>
#include <assert.h>
#include <algorithm>
using namespace std;
typedef float float4 __attribute__((vector_size (16)));
static inline void * alloc64(size_t sz) {
void * a = 0;
if (posix_memalign(&a, 64, sz) != 0) {
perror("posix_memalign");
exit(1);
}
return a;
}
struct Mat {
size_t m,n;
float * a;
Mat(size_t m_, size_t n_, float f) {
m = m_;
n = n_;
a = (float*) malloc(sizeof(float) * m * n);
fill(a,a + m * n,f);
}
/* a(i,j) */
float& operator()(long i, long j) {
return a[i * n + j];
}
};
Mat operator* (Mat a, Mat b) {
Mat c(a.m, b.n,0);
assert(a.n == b.m);
for (long i = 0; i < a.m; i++) {
for(long k = 0; k < a.n; k++){
float aa = a(i,k);
float4 a4 = {aa,aa,aa,aa};
long j;
for (j = 0; j <= b.n-4; j+=4) {
*((float4 *)&c(i,j)) = *((float4 *)&c(i,j)) + a4 * (*(float4 *)&b(k,j));
}
while(j < b.n){
c(i,j) += aa * b(k,j);
j++;
}
}
}
return c;
}
const int M = 10;
const int K = 12;
const int N = 12;
int main(){
Mat a(M,K,1);
Mat b(K,N,1);
Mat c = a * b;
for(int i = 0; i < M; i++){
for(int j = 0; j < N; j++)
cout << c(i,j) << " ";
cout << endl;
}
cout << endl;
}

Answer 1

In my understanding, when using a SIMD for a vector wich 16bytes (in
this case float4), the target memory address should be a multiple of
16.
That is incorrect on x64 processors. There are instructions that require alignment, but you can perfectly well write and read SIMD registers from unaligned memory locations without penalty and with absolute safety using the right instructions.
Is it true that I'm getting the segfault for the memory alignment
issues?
Yes.
But it is not related to SIMD instructions. In C/C++, it is undefined behavior to write *((float4 *)&c) = ... the way you do, and can certainly crash, but you can reproduce the problem without vectorization... Given the right circumstances, the following basic code will crash...
char * c = ...
*(int *) c = 1;
Can anyone tell me how to fix the problem easily? I've thought of
using a two-dimensional array instead of one array, but I don't want
to do this so as not to change the rest of the codes.
The typical workaround is to use memcpy. Let us look at a code example...
#include <string.h>
typedef float float4 __attribute__((vector_size (16)));
void writeover(float * x, float4 y) {
*(float4 * ) x = y;
}
void writeover2(float * x, float4 y) {
memcpy(x,&y,sizeof(y));
}
With, say, clang++, these two functions get compiled to vmovaps and vmovups. These are equivalent instructions, but the first one will crash if your pointer is not aligned on sizeof(float4). They are very fast functions on recent hardware.
The point is that you can often rely on memcpy to generate code that is nearly optimally fast. Of course, the amount of overhead you get (if any) will depend on the compiler you are using.
If you do get performance problems, then you can use Intel intrinsics or assembly instead... but chances are good that memcpy will serve you well.
A different fix is to only work in terms of float4 * pointers. This forces all your matrices to have dimensions that are divisible by four, but if you pad the leftover with zeroes you will probably get simple and really fast code.