Indexing with byte arrays.

[mdouze]

• OS: Linux
• How you installed TC (docker, conda, source): conda
• Python version: 3.6.5
• CUDA/cuDNN version: N/A
• Conda version (if using conda): 4.5.0

In addition, including the following information will also be very helpful for us to diagnose the problem:

• A script to reproduce the issue (highly recommended if its a build issue)

import tensor_comprehensions as tc
import numpy as np
import torch

N = 10 ** 7
M = 32

codes = np.random.randint(1<<32, size=(N, M // 4)).astype('uint32')
codes = codes.view('uint8')

codes_t = torch.from_numpy(codes)

luts = np.random.randn(M, 256).astype('float32')
luts_t = torch.from_numpy(luts)

lang = """
def mindis(float(M, 256) L, uint8(N, M) codes) -> (v, s) {
    s(i) +=! L(j, codes(i, j))
    v min=! s(i)
}
"""

mindis = tc.define(lang, name="mindis")

vmin = mindis(luts_t, codes_t)

• Error messages and/or stack traces of the issue (create a gist)

[WARNING]: No mapping options passed, 'naive' type mapping options will be used and will likely have bad performance. See help(your_layer.__call__) for setting mapping options.
[ERROR]: Caught Exception: Internal error at /opt/conda/conda-bld/halide_1520457560724/work/src/IR.cpp:560
Condition failed: args[i].type() == Int(32)
Args to load from image must be type Int(32)

• Context around what you are trying to do

This code is to find the min value of a distance estimation with a large number of Product Quantization codes.
The problems in the code above are:

• the error above
• when adding a cast int32(codes(i, j)), the code crashes with https://gist.github.com/mdouze/472248f7d4205dc5a96464d0851a1a7e
• we are not interested in s, it is a temp variable
• we are interested in the argmin, but this seems to be on the roadmap.

[ftynse]

Seems to come from Halide pass, related to #98

[abadams]

Needs the changes in #262 at the least. In trunk the declarations for types smaller than int32 don't even exist at the cuda level.

[ftynse]

Why does it need #262? There is no indirection on the left hand-side in this code and I think we cherry-picked typedefs

[nicolasvasilache]

yeah #262 is unrelated, should be an easy one, I'll take it

[abadams]

I checked master and didn't see them there, even though I knew I had written them at some point. That's why I referred back to #262. I agree that this probably doesn't need that full set of changes, but there is indirection and I remembered fixing stuff related to indirection in that PR.

[mdouze]

For completeness, the full use case involves an additional dimension:

import tensor_comprehensions as tc
import numpy as np
import torch

N = 10 ** 7
M = 32
K = 16 
codes = np.random.randint(1<<32, size=(N, M // 4)).astype('uint32')
codes = codes.view('uint8')
luts = np.random.randn(K, M, 256).astype('float32')

codes_t = torch.from_numpy(codes)
luts_t = torch.from_numpy(luts)


lang = """
def mindis(float(K, M, 256) L, uint8(N, M) codes) -> (v, s) {
    s(k, i) +=! L(k, j, int32(codes(i, j)))
    v(k) min=! s(k, i)
}
"""

mindis = tc.define(lang, name="mindis")
vmin, s = mindis(luts_t, codes_t)

The same remark applies: we are not interested in s.

[ftynse]

The same remark applies: we are not interested in s.

This request is similar to #174. It would require TC to properly handle temporaries (#18) and to support some flavor of expression inlining. It has been discussed several times and is somewhere in mid-term goals.

[nicolasvasilache]

@mdouze I updated #512 with your latest example, untuned perf with naive options of the 5 kernel variant on your problem size is about 90ms, see this.