DArray: Optimize ldiv and LU

[jpsamaroo]

Written by Claude

[github-actions]

Dagger benchmarks: dirty vs master

	master	dirty	master / dirty
array/dagger/N=1024 (block 512)/add (X + X)	4.65 ± 0.14 ms	5.18 ± 0.8 ms	0.898 ± 0.14
array/dagger/N=1024 (block 512)/alloc (rand)	4.07 ± 0.34 ms	3.87 ± 0.15 ms	1.05 ± 0.096
array/dagger/N=1024 (block 512)/broadcast (X .+ 1)	3.03 ± 0.27 ms	3.05 ± 0.084 ms	0.993 ± 0.092
array/dagger/N=1024 (block 512)/map (sin.(X))	7.31 ± 0.13 ms	7.24 ± 0.41 ms	1.01 ± 0.06
array/dagger/N=1024 (block 512)/norm	1.28 ± 0.074 ms	1.28 ± 0.041 ms	0.999 ± 0.066
array/dagger/N=1024 (block 512)/reduce (sum)	2.67 ± 0.68 ms	2.71 ± 0.044 ms	0.988 ± 0.25
array/dagger/N=1024 (block 512)/transpose (permutedims)	7.35 ± 0.4 ms	9.39 ± 2.8 ms	0.782 ± 0.24
array/dagger/N=256 (block 256)/add (X + X)	1.36 ± 0.054 ms	1.55 ± 0.29 ms	0.881 ± 0.17
array/dagger/N=256 (block 256)/alloc (rand)	1.19 ± 0.11 ms	1.27 ± 0.019 ms	0.938 ± 0.086
array/dagger/N=256 (block 256)/broadcast (X .+ 1)	0.998 ± 0.36 ms	0.723 ± 0.019 ms	1.38 ± 0.5
array/dagger/N=256 (block 256)/map (sin.(X))	1.21 ± 0.38 ms	1.29 ± 0.083 ms	0.932 ± 0.3
array/dagger/N=256 (block 256)/norm	0.478 ± 0.028 ms	0.534 ± 0.084 ms	0.896 ± 0.15
array/dagger/N=256 (block 256)/reduce (sum)	0.992 ± 0.66 ms	0.96 ± 0.3 ms	1.03 ± 0.76
array/dagger/N=256 (block 256)/transpose (permutedims)	0.988 ± 0.23 ms	0.99 ± 0.079 ms	0.998 ± 0.25
linalg/dagger/N=1024 (block 512)/cholesky	20.7 ± 1.7 ms	20.1 ± 1.4 ms	1.03 ± 0.11
linalg/dagger/N=1024 (block 512)/lu	1.81 ± 0.0057 s	0.045 ± 0.0032 s	40.1 ± 2.9
linalg/dagger/N=1024 (block 512)/matmul (A*A)	0.0456 ± 0.006 s	0.0457 ± 0.0015 s	0.998 ± 0.14
linalg/dagger/N=1024 (block 512)/matvec (A*x)	3.04 ± 0.25 ms	3.06 ± 0.64 ms	0.992 ± 0.22
linalg/dagger/N=1024 (block 512)/qr	0.374 ± 0.014 s	0.366 ± 0.014 s	1.02 ± 0.055
linalg/dagger/N=1024 (block 512)/solve (A\b via lu)	2.85 s	0.0562 ± 0.0024 s	50.7
linalg/dagger/N=1024 (block 512)/syrk (A'*A)	0.0364 ± 0.003 s	0.0366 ± 0.0023 s	0.994 ± 0.1
linalg/dagger/N=256 (block 256)/cholesky	3.35 ± 2 ms	2.97 ± 0.19 ms	1.13 ± 0.68
linalg/dagger/N=256 (block 256)/lu	0.296 ± 0.013 s	4.72 ± 0.7 ms	62.6 ± 9.7
linalg/dagger/N=256 (block 256)/matmul (A*A)	2.85 ± 0.45 ms	2.85 ± 0.66 ms	1 ± 0.28
linalg/dagger/N=256 (block 256)/matvec (A*x)	1.73 ± 0.39 ms	1.51 ± 0.065 ms	1.15 ± 0.26
linalg/dagger/N=256 (block 256)/qr	7.42 ± 0.0085 ms	7.38 ± 0.31 ms	1.01 ± 0.042
linalg/dagger/N=256 (block 256)/solve (A\b via lu)	0.578 ± 0.0062 s	9.03 ± 3.2 ms	64 ± 23
linalg/dagger/N=256 (block 256)/syrk (A'*A)	3.51 ± 0.2 ms	4.48 ± 0.35 ms	0.785 ± 0.076
stencil/dagger/N=1024 (block 512)/alloc (neighbors Wrap)	0.0353 ± 0.0045 s	0.0331 ± 0.00072 s	1.07 ± 0.14
stencil/dagger/N=1024 (block 512)/assign (const)	1.45 ± 0.14 ms	1.39 ± 0.082 ms	1.04 ± 0.12
stencil/dagger/N=1024 (block 512)/multi-expr	3.74 ± 0.28 ms	3.63 ± 0.25 ms	1.03 ± 0.11
stencil/dagger/N=1024 (block 512)/neighbors (Clamp)	30 ± 1.2 ms	24 ± 8.1 ms	1.25 ± 0.43
stencil/dagger/N=1024 (block 512)/neighbors (Pad)	27.5 ± 9.7 ms	29.2 ± 4.6 ms	0.943 ± 0.36
stencil/dagger/N=1024 (block 512)/neighbors (Reflect)	28 ± 7.2 ms	29.3 ± 4.8 ms	0.958 ± 0.29
stencil/dagger/N=1024 (block 512)/neighbors (Wrap)	30.9 ± 2.1 ms	29.7 ± 7.7 ms	1.04 ± 0.28
stencil/dagger/N=1024 (block 512)/update (+)	2.14 ± 0.69 ms	2.03 ± 0.15 ms	1.05 ± 0.35
stencil/dagger/N=256 (block 256)/alloc (neighbors Wrap)	5.17 ± 0.21 ms	5.43 ± 0.66 ms	0.953 ± 0.12
stencil/dagger/N=256 (block 256)/assign (const)	0.714 ± 0.14 ms	0.657 ± 0.1 ms	1.09 ± 0.27
stencil/dagger/N=256 (block 256)/multi-expr	1.72 ± 0.72 ms	1.3 ± 0.022 ms	1.32 ± 0.55
stencil/dagger/N=256 (block 256)/neighbors (Clamp)	4.48 ± 0.034 ms	4.42 ± 0.18 ms	1.01 ± 0.041
stencil/dagger/N=256 (block 256)/neighbors (Pad)	4.91 ± 0.24 ms	4.52 ± 0.15 ms	1.09 ± 0.063
stencil/dagger/N=256 (block 256)/neighbors (Reflect)	4.56 ± 0.34 ms	4.4 ± 0.054 ms	1.04 ± 0.078
stencil/dagger/N=256 (block 256)/neighbors (Wrap)	4.86 ± 0.98 ms	4.56 ± 0.11 ms	1.07 ± 0.22
stencil/dagger/N=256 (block 256)/update (+)	0.759 ± 0.04 ms	0.744 ± 0.026 ms	1.02 ± 0.065
time_to_load	1.11 ± 0.014 s	1.02 ± 0.011 s	1.09 ± 0.018

Plots

⚠️ Regressions (> 25.0%)

• array/dagger/N=1024 (block 512)/transpose (permutedims): +27.8%
• linalg/dagger/N=256 (block 256)/syrk (A'*A): +27.4%

Improvements (> 25.0% faster)

• linalg/dagger/N=256 (block 256)/solve (A\b via lu): -98.4%
• linalg/dagger/N=256 (block 256)/lu: -98.4%
• linalg/dagger/N=1024 (block 512)/solve (A\b via lu): -98.0%
• linalg/dagger/N=1024 (block 512)/lu: -97.5%
• array/dagger/N=256 (block 256)/broadcast (X .+ 1): -27.6%

Full results and plots (download the benchmark-results artifact).