SME1 support for int4 GEMM and GEMMV kernels

[qti-vaiskv]

Adds support for SME1 int4 GEMM and GEMMV kernels.

[dsharlet]

It looks like this is adding a new dependency, a fork of KleidiAI? https://github.com/qualcomm/kleidiai

We can't have two versions of the same library in our repo. I'm surprised it works at all.

Is it even necessary? https://github.com/qualcomm/kleidiai doesn't appear to have any commits that aren't in the upstream repo.

[qti-vaiskv]

There are few missing SME1 kernels especially the int4 datatypes in the parent Kleidiai. We have added support for these kernels in Qualcomm's fork of Kleidiai.

We are maintaining this repo to account for these missing kernels and there will not be any duplicate definitions of the functions that are already part of Kleidiai.

https://github.com/qualcomm/kleidiai/commits/kleidiai-qmx-xnnpack/

Note: In this PR we have added only the cmake support. We will add the bazel support as well in this same PR.

[fbarchard]

I agree we should support SME 1, for all devices, not just Oryon V3.
On ARM at least (lumix) the register groups are not a performance advantage
The SME 2.1 load is a win apparently, but we're on SME2.0
There are also 'xnnpack' 'sme2' wrappers that call sme1 kleidi kernels, which would work.

I assume on SME1 this is alot faster than kleidi, which falls back on Neon I8MM?

I looked at your ssve dot kernel, because its interesting youre doing ssve, not just sme fmopa
Its using a slow approach to weights
I'd suggest looking at the xnnpack neondot/i8mm kernels
shr and sub to sign extend - instead of this, use the upper 4 bits and treat as int8
and/shr/zip - youre using packed weights. suggest planar so you eliminate the zip

the 'sme' dot code is really sve, but in streaming mode. have you tried the sve version of this? seems like it would perform better with threads

we, google/xnnpack, really only use bazel, so build changes with both are preferred.