Hello, why the transformation in ipynb for the matrix multiplication can make cache more friendly?

[mazdarx7fc3s]

before transformation:

for i0, i1, i2 in tir.grid(1024, 1024, 1024):
    with tir.block("C"):
        m, n, k = tir.axis.remap("SSR", [i0, i1, i2])

after transformation:

for i0_0, i1_0, i2, i0_1, i1_1 in tir.grid(32, 32, 1024, 32, 32):
        with tir.block("C"):
            m = tir.axis.spatial(1024, i0_0 * 32 + i0_1)
            n = tir.axis.spatial(1024, i1_0 * 32 + i1_1)
            k = tir.axis.reduce(1024, i2)

In my view, before the transformation, we need make a 1024*1024*1024-for-loop, after the transformation, we still need make the 1024*1024*1024-for-loop. Why the time costs decreases so much?

[Hzfengsy]

It's because we enhance the cache hit rate. Please see https://tvm.apache.org/docs/how_to/optimize_operators/opt_gemm.html#blocking

[mazdarx7fc3s]

Sorry, I'm not very clear about memory access. So after transformation, every time, there is a data chunk of 32*32 transmitted to cache? Then the inner loops (i0_1, i1_1) access the data chunk in cache?

[junrushao]

usually reuse of a small chunk of data helps cache-friendliness. Considering the buffer access under the inner loops, i.e. i0_1, i1_1, would you like to calculate the region that is repetitively accessed?