Skip to content

Benchmarks re-measured after the Granger and MI rewrites #54

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
nullsatz merged 1 commit into from
Oct 20, 2026
Merged

Benchmarks re-measured after the Granger and MI rewrites #54

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions vignettes/benchmarks-machine.csv
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,5 +4,5 @@
"CPU threads","8"
"BLAS","libopenblasp-r0.3.34.so"
"R","R version 4.5.3 (2026-03-11)"
"gputools","2.0.0.9000 (commit 9e37078)"
"date","2026-10-15"
"gputools","2.1.1"
"date","2026-10-20"
6 changes: 4 additions & 2 deletions vignettes/benchmarks.Rmd
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -97,8 +97,10 @@ that subtracts two nearly equal sums of squares, use double. `gpuGranger`
is the clear case: its F statistic is a difference of two residual sums
of squares, and in single precision it was off by more than 10 percent on
long random walks, so it now always computes in double, whatever
`precision` says. (The Granger timings in this vignette were measured
before that change; `bench/run-benchmarks.R` repeats them.)
`precision` says; its two columns in the tables are therefore the same
computation, and it still runs over a hundred times faster than the
`lm.fit` loop at 50 series, since each effect's own past is projected out
once and every cause's test against it comes from a few matrix products.

## When the GPU pays off

Expand Down
26 changes: 13 additions & 13 deletions vignettes/benchmarks.csv
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,14 +1,14 @@
"task","size","cpu","single","double"
"gpuMatMult vs %*%","2000 x 2000",0.032,NA,0.0529999999999999
"gpuMatMult vs %*%","4000 x 4000",0.22,NA,0.337
"gpuCor pearson vs cor","2000 obs x 1000 vars",0.909,0.0180000000000007,0.0299999999999994
"gpuCor kendall vs cor","1000 obs x 100 vars",61.663,0.0229999999999677,0.256000000000029
"gpuDist vs dist","5000 points x 50 dims",2.43299999999999,0.358000000000004,0.40500000000003
"gpuHclust vs hclust","5000 points",0.74799999999999,0.735000000000014,1.01499999999999
"gpuDistClust vs hclust(dist())","5000 points x 50 dims",3.15800000000002,0.546999999999969,0.814000000000021
"gpuGranger vs lm.fit loop","500 times x 50 series, lag 2",0.41500000000002,0.0989999999999895,0.111999999999966
"gpuMi (no base-R equivalent)","2000 obs x 200 vars, 10 bins",NA,0.0600000000000023,0.103000000000009
"gpuLm vs lm","100000 obs x 50 predictors",0.139999999999986,0.125,0.141999999999996
"gpuQr vs qr","4000 x 1000",1.25600000000003,0.879999999999995,1.02999999999997
"gpuSolve vs solve","2000 x 2000",0.0959999999999468,0.0400000000000205,0.105999999999995
"kernel compile, first use of a file","gpuDist kernels",NA,0.361,0.206
"gpuMatMult vs %*%","2000 x 2000",0.04,NA,0.056
"gpuMatMult vs %*%","4000 x 4000",0.286,NA,0.345
"gpuCor pearson vs cor","2000 obs x 1000 vars",0.960999999999999,0.0180000000000007,0.0299999999999994
"gpuCor kendall vs cor","1000 obs x 100 vars",61.882,0.0240000000000009,0.261000000000024
"gpuDist vs dist","5000 points x 50 dims",2.45500000000004,0.375,0.423999999999978
"gpuHclust vs hclust","5000 points",0.764999999999986,0.745999999999981,1.02499999999998
"gpuDistClust vs hclust(dist())","5000 points x 50 dims",3.19599999999997,0.550999999999988,0.819999999999993
"gpuGranger vs lm.fit loop","500 times x 50 series, lag 2",0.423000000000002,0.0029999999999859,0.00200000000000955
"gpuMi (no base-R equivalent)","2000 obs x 200 vars, 10 bins",NA,0.0550000000000068,0.0830000000000268
"gpuLm vs lm","100000 obs x 50 predictors",0.149999999999977,0.128999999999962,0.145999999999958
"gpuQr vs qr","4000 x 1000",1.51900000000001,0.893000000000029,1.04300000000001
"gpuSolve vs solve","2000 x 2000",0.107000000000028,0.0400000000000205,0.108000000000004
"kernel compile, first use of a file","gpuDist kernels",NA,0.383,0.233
8 changes: 4 additions & 4 deletions vignettes/call-overhead.csv
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
"what","seconds"
"one gpuCor call, 50 x 2",4.97512437810493e-05
"one gpuCor call, 50 x 2",4.47761194030575e-05
"one cor call, 50 x 2",1.00000000000477e-05
"1,000 pairs: a loop of gpuCor calls",0.0647500000000036
"1,000 pairs: one gpuCor call for all 1,000 x 1,000 (the 1,000 wanted are its diagonal)",0.0145624999999967
"1,000 pairs: a loop of cor calls",0.0152000000000006
"1,000 pairs: a loop of gpuCor calls",0.0602499999999964
"1,000 pairs: one gpuCor call for all 1,000 x 1,000 (the 1,000 wanted are its diagonal)",0.0141764705882378
"1,000 pairs: a loop of cor calls",0.0144374999999997
124 changes: 62 additions & 62 deletions vignettes/crossover.csv
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,63 +1,63 @@
"fn","what","size","cpu","single","double"
"gpuCor","variables (200 rows)",10,2.48756218905467e-05,6.49999999999995e-05,0.0001285
"gpuCor","variables (200 rows)",30,0.000114427860696518,9.54999999999999e-05,0.000155000000000003
"gpuCor","variables (200 rows)",100,0.00099004975124378,0.000282,0.0003815
"gpuCor","variables (200 rows)",300,0.00853846153846152,0.00156000000000001,0.00177611940298507
"gpuCor","variables (200 rows)",1000,0.0943333333333332,0.0205833333333333,0.0217272727272728
"gpuCor","variables (200 rows)",3000,0.870000000000001,0.173499999999999,0.185
"gpuCor spearman","variables (200 rows)",10,0.000277227722772299,0.000114999999999998,0.000193
"gpuCor spearman","variables (200 rows)",30,0.000910447761194029,0.000156000000000001,0.000238
"gpuCor spearman","variables (200 rows)",100,0.00358823529411764,0.000410000000000004,0.000562189054726366
"gpuCor spearman","variables (200 rows)",300,0.0162142857142857,0.00181188118811881,0.00226865671641787
"gpuCor spearman","variables (200 rows)",1000,0.1195,0.018166666666667,0.0209000000000003
"gpuCor spearman","variables (200 rows)",3000,0.951999999999998,0.167999999999999,0.184999999999999
"gpuCor kendall","variables (200 rows)",5,0.00668000000000006,0.000139303482587058,0.000217500000000001
"gpuCor kendall","variables (200 rows)",10,0.0253750000000004,0.0001615,0.000334
"gpuCor kendall","variables (200 rows)",30,0.220000000000006,0.000313432835820907,0.00130693069306929
"gpuCor kendall","variables (200 rows)",100,2.447,0.00174257425742576,0.0120588235294117
"gpuCor kendall","variables (200 rows)",300,21.837,0.0132666666666675,0.105000000000004
"gpuDist","points (20 dimensions)",50,0.000104999999999933,0.000100000000000051,0.000100000000000009
"gpuDist","points (20 dimensions)",100,0.000379999999999967,0.000171999999999997,0.000192999999999998
"gpuDist","points (20 dimensions)",300,0.00339215686274513,0.00121499999999997,0.0014100000000002
"gpuDist","points (20 dimensions)",1000,0.0383333333333364,0.0160833333333343,0.0184545454545431
"gpuDist","points (20 dimensions)",3000,0.342999999999989,0.168499999999995,0.175500000000014
"gpuHclust","points",50,5.49999999999784e-05,0.000524752475247498,0.000494999999999948
"gpuHclust","points",100,0.000211500000000001,0.000919999999999987,0.000969999999999942
"gpuHclust","points",300,0.00194000000000003,0.0032352941176469,0.00358000000000004
"gpuHclust","points",1000,0.0204000000000008,0.0203000000000003,0.0240000000000009
"gpuHclust","points",3000,0.275000000000006,0.233000000000004,0.311000000000007
"gpuDistClust","points (20 dimensions)",50,0.000168316831683128,0.00046999999999997,0.000485000000000042
"gpuDistClust","points (20 dimensions)",100,0.000580000000000069,0.000881188118811868,0.000935323383084518
"gpuDistClust","points (20 dimensions)",300,0.00509756097560983,0.00271641791044758,0.0030298507462687
"gpuDistClust","points (20 dimensions)",1000,0.0570000000000022,0.0126250000000017,0.0158461538461552
"gpuDistClust","points (20 dimensions)",3000,0.632000000000005,0.155500000000004,0.224999999999994
"gpuGranger","series (300 times, lag 2)",5,0.00311764705882393,0.00194117647058803,0.00343137254901983
"gpuGranger","series (300 times, lag 2)",10,0.0138125000000002,0.00535294117647107,0.00716000000000008
"gpuGranger","series (300 times, lag 2)",20,0.0587500000000034,0.0181818181818171,0.0207999999999998
"gpuGranger","series (300 times, lag 2)",50,0.377999999999986,0.0956666666666687,0.103999999999999
"gpuGranger","series (300 times, lag 2)",100,1.53199999999998,0.377999999999986,0.396000000000015
"gpuLm","rows (20 predictors)",1000,0.00192537313432822,0.00510344827586194,0.0072413793103451
"gpuLm","rows (20 predictors)",10000,0.00673529411764745,0.0122352941176486,0.0145333333333326
"gpuLm","rows (20 predictors)",1e+05,0.0660000000000025,0.11766666666667,0.126500000000007
"gpuLm","rows (20 predictors)",1e+06,0.576999999999998,1.083,1.04599999999999
"gpuGlm","rows (20 predictors)",1000,0.00421951219512061,0.0132142857142859,0.0251250000000027
"gpuGlm","rows (20 predictors)",10000,0.0242222222222242,0.0395000000000039,0.0464000000000055
"gpuGlm","rows (20 predictors)",1e+05,0.257999999999981,0.267999999999972,0.367999999999995
"gpuQr","columns (2 x as many rows)",50,8.95522388058321e-05,0.00361999999999966,0.00634482758620779
"gpuQr","columns (2 x as many rows)",100,0.000500499999999988,0.00755999999999858,0.00999999999999903
"gpuQr","columns (2 x as many rows)",300,0.0105263157894761,0.0273749999999993,0.0412000000000035
"gpuQr","columns (2 x as many rows)",1000,0.463999999999999,0.314999999999998,0.534999999999968
"gpuQr","columns (2 x as many rows)",2000,5.98599999999999,1.971,2.46600000000001
"gpuSolve","n (n x n system)",50,3.99999999999068e-05,0.000165000000000077,0.000201000000000022
"gpuSolve","n (n x n system)",100,0.000218999999999994,0.000264999999999986,0.000356999999999999
"gpuSolve","n (n x n system)",300,0.00125999999999976,0.00106965174129369,0.00198000000000036
"gpuSolve","n (n x n system)",1000,0.0123125000000002,0.00700000000000108,0.0126470588235313
"gpuSolve","n (n x n system)",3000,0.120500000000021,0.0442000000000007,0.0985000000000014
"gpuPrcomp","variables (1,000 rows)",20,0.000849999999999795,0.0013880597014928,0.00205970149253697
"gpuPrcomp","variables (1,000 rows)",100,0.00900000000000143,0.00796551724137912,0.0108947368421049
"gpuPrcomp","variables (1,000 rows)",300,0.0335714285714305,0.0255555555555513,0.041166666666669
"gpuPrcomp","variables (1,000 rows)",1000,0.247000000000014,0.103500000000025,0.212999999999965
"gpuMatMult","n (n x n by n x n)",100,4.97512437810493e-05,0.000124378109452623,0.000137
"gpuMatMult","n (n x n by n x n)",300,0.000296999999999997,0.00106467661691542,0.00106499999999983
"gpuMatMult","n (n x n by n x n)",1000,0.00629411764705878,0.0142631578947376,0.0118095238095234
"gpuMatMult","n (n x n by n x n)",3000,0.103499999999997,0.144500000000022,0.144999999999982
"gpuCor","variables (200 rows)",10,2.5e-05,6.50000000000001e-05,0.0001345
"gpuCor","variables (200 rows)",30,0.000115,9.20000000000001e-05,0.000154228855721394
"gpuCor","variables (200 rows)",100,0.000985074626865671,0.0002645,0.000398009950248757
"gpuCor","variables (200 rows)",300,0.00852173913043481,0.00145544554455445,0.00170297029702971
"gpuCor","variables (200 rows)",1000,0.0933333333333337,0.0170666666666667,0.0173571428571429
"gpuCor","variables (200 rows)",3000,0.860000000000001,0.151,0.16
"gpuCor spearman","variables (200 rows)",10,0.000288557213930347,0.000124378109452747,0.000192500000000001
"gpuCor spearman","variables (200 rows)",30,0.000861386138613859,0.000169154228855734,0.00024
"gpuCor spearman","variables (200 rows)",100,0.00347999999999999,0.000449999999999999,0.000534999999999997
"gpuCor spearman","variables (200 rows)",300,0.0159285714285714,0.00185148514851486,0.00208910891089107
"gpuCor spearman","variables (200 rows)",1000,0.1175,0.0156666666666666,0.0161818181818179
"gpuCor spearman","variables (200 rows)",3000,0.937999999999999,0.158000000000001,0.166999999999998
"gpuCor kendall","variables (200 rows)",5,0.00669230769230767,0.000139303482587058,0.000215499999999999
"gpuCor kendall","variables (200 rows)",10,0.0252500000000007,0.000158999999999999,0.000345000000000013
"gpuCor kendall","variables (200 rows)",30,0.219999999999999,0.000315000000000012,0.00131
"gpuCor kendall","variables (200 rows)",100,2.446,0.00177,0.0120588235294117
"gpuCor kendall","variables (200 rows)",300,21.72,0.0129374999999996,0.105499999999992
"gpuDist","points (20 dimensions)",50,0.000104477611940232,9.50000000000273e-05,9.64999999999918e-05
"gpuDist","points (20 dimensions)",100,0.000380000000000109,0.000185000000000031,0.000204999999999984
"gpuDist","points (20 dimensions)",300,0.00340298507462657,0.00112437810945273,0.00131
"gpuDist","points (20 dimensions)",1000,0.0381666666666642,0.0159285714285724,0.0157499999999994
"gpuDist","points (20 dimensions)",3000,0.341000000000008,0.144499999999994,0.152500000000003
"gpuHclust","points",50,5.49999999999784e-05,0.000519999999999925,0.000515000000000043
"gpuHclust","points",100,0.000194999999999936,0.000914999999999964,0.000974999999999966
"gpuHclust","points",300,0.00198019801980187,0.00340298507462699,0.00372000000000014
"gpuHclust","points",1000,0.0191818181818193,0.0198181818181808,0.0230999999999995
"gpuHclust","points",3000,0.241000000000014,0.224999999999994,0.299999999999983
"gpuDistClust","points (20 dimensions)",50,0.000158415841584071,0.000465346534653436,0.000485000000000042
"gpuDistClust","points (20 dimensions)",100,0.000579999999999927,0.000874999999999915,0.000930000000000035
"gpuDistClust","points (20 dimensions)",300,0.00507500000000007,0.00271641791044758,0.0030298507462687
"gpuDistClust","points (20 dimensions)",1000,0.0560000000000045,0.0125625000000014,0.0156923076923083
"gpuDistClust","points (20 dimensions)",3000,0.593999999999994,0.154499999999999,0.225999999999999
"gpuGranger","series (300 times, lag 2)",5,0.00294029850746273,0.000739999999999839,0.000739999999999981
"gpuGranger","series (300 times, lag 2)",10,0.0131875000000008,0.00070999999999998,0.00070999999999998
"gpuGranger","series (300 times, lag 2)",20,0.0559999999999974,0.000730000000000075,0.000735000000000099
"gpuGranger","series (300 times, lag 2)",50,0.361000000000018,0.00129999999999995,0.0012970297029703
"gpuGranger","series (300 times, lag 2)",100,1.465,0.0036274509803922,0.00361999999999966
"gpuLm","rows (20 predictors)",1000,0.00183582089552225,0.00510344827586194,0.00720000000000027
"gpuLm","rows (20 predictors)",10000,0.00623529411764674,0.0118749999999999,0.013857142857142
"gpuLm","rows (20 predictors)",1e+05,0.0597500000000011,0.104666666666664,0.115
"gpuLm","rows (20 predictors)",1e+06,0.513000000000005,0.984000000000009,0.994
"gpuGlm","rows (20 predictors)",1000,0.00387804878048761,0.0130714285714281,0.0238888888888893
"gpuGlm","rows (20 predictors)",10000,0.0227000000000004,0.0376666666666665,0.0447999999999979
"gpuGlm","rows (20 predictors)",1e+05,0.225999999999999,0.25,0.34099999999998
"gpuQr","columns (2 x as many rows)",50,9.00000000000034e-05,0.00360975609756088,0.00637931034482766
"gpuQr","columns (2 x as many rows)",100,0.000489999999999924,0.00744000000000028,0.0099523809523811
"gpuQr","columns (2 x as many rows)",300,0.0105789473684207,0.0271250000000016,0.0408000000000015
"gpuQr","columns (2 x as many rows)",1000,0.443999999999988,0.314999999999998,0.533000000000015
"gpuQr","columns (2 x as many rows)",2000,5.60899999999998,1.97000000000003,2.45100000000002
"gpuSolve","n (n x n system)",50,3.99999999999068e-05,0.000169154228855681,0.000199999999999818
"gpuSolve","n (n x n system)",100,0.000219999999999914,0.000260999999999996,0.000348258706467628
"gpuSolve","n (n x n system)",300,0.00125999999999976,0.00103999999999985,0.00192999999999984
"gpuSolve","n (n x n system)",1000,0.0115789473684195,0.00648275862068925,0.0117647058823523
"gpuSolve","n (n x n system)",3000,0.110499999999973,0.0396666666666666,0.0929999999999988
"gpuPrcomp","variables (1,000 rows)",20,0.000810000000000173,0.00134328358209003,0.00198507462686539
"gpuPrcomp","variables (1,000 rows)",100,0.0085769230769214,0.00751724137931096,0.0104285714285711
"gpuPrcomp","variables (1,000 rows)",300,0.031714285714291,0.0246666666666644,0.0403999999999996
"gpuPrcomp","variables (1,000 rows)",1000,0.23399999999998,0.101500000000016,0.211000000000013
"gpuMatMult","n (n x n by n x n)",100,4.49999999997885e-05,0.000120000000000005,0.000128999999999991
"gpuMatMult","n (n x n by n x n)",300,0.000291499999999985,0.000965174129353318,0.000964999999999918
"gpuMatMult","n (n x n by n x n)",1000,0.00578048780487861,0.0127894736842103,0.0126315789473689
"gpuMatMult","n (n x n by n x n)",3000,0.101666666666669,0.145499999999998,0.145499999999998
Loading