Convolution Backward Pass with kernel_size > 1

[pclucas14]

Hi,

I'm currently trying to implement a custom convolution operation (foward and backward passes). I am starting from the code provided here

I am getting an error with the backward pass, as d_o and I don't have the same Height H and Width W (when the kernel size is > 1).

Is there currently a way to define the classic convolution operator with backward and a filter size > 1 pass using Tensor Comprehension ?

(Note that I'm not using TC.make_autograd, as this functionality does not seem to be available with the currently released conda package)

Thanks,
Lucas

[kevjshih]

Yes, you can definitely do it. I have a working solution using a different set of variables for the W and H of d_O, eg. float(N,M,OH, OW) d_O. I also had to wrap the gradient formulation with the ternary operator to handle some cases when the kernel is operating on the edges of its input.

[pclucas14]

Great news!

Would you mind explaining how to implement handling of border cases ? If possible, could you share code ?

Best,
Lucas

[kevjshih]

My code ended up looking something like follows. You absolutely want to check it for errors before using it.

d_I(n, c, h, w) +=! ((h - r_kh >= 0) && (w - r_kw >= 0) && (w - r_kw + KW <= W) && (h - r_kh +KH <= H)) ? d_O( n, r_m, h - r_kh, w - r_kw) * W1(r_m, c, r_kh, r_kw) : 0 where r_kh in 0:KH, r_kw in 0:KW, h in 0:H, w in 0:W, r_m in 0:M, n in 0:N, c in 0:C

d_W1(m, c, kh, kw) +=! ((r_h >= kh) && (r_w >=kw) && (r_w - kw + KW <= W) && (r_h - kh +KH <= H) ) ? d_O( r_n, m, r_h-kh, r_w-kw) * I(r_n, c, r_h, r_w) : 0 where kh in 0:KH, kw in 0:KW, r_h in 0:H, r_w in 0:W, m in 0:M, c in 0:C

[pclucas14]

ah smart! Thanks a lot for your solution :)

[nicolasvasilache]

Hello @pclucas14 @kevjshih

My apologies for the long delay, I am sorry to say that the core TC team has moved on to other things.
As far as I know the project is not maintained anymore but someone from Facebook should voice in, I am not in a position to speak for them.

Glad to see you have found a solution though!
There are indeed a lot of stuff that you can do by just injecting some predicated control flow in your TCs, in particular triangular and sparser data structures.

Best,

N