Repository navigation
Expand file tree
/
Copy pathindex.html
More file actions
executable file
·1492 lines (1398 loc) · 114 KB
/
Copy pathindex.html
File metadata and controls
executable file
·1492 lines (1398 loc) · 114 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Haggai Maron</title>
<meta name="description" content="Haggai Maron – Assistant Professor at the Technion and Senior Research Scientist at NVIDIA Research. Research in machine learning and deep learning for structured data.">
<link rel="preconnect" href="https://fonts.googleapis.com">
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
<link href="https://fonts.googleapis.com/css2?family=Inter:wght@300;400;500;600;700&display=swap" rel="stylesheet">
<link rel="stylesheet" href="css/style.css">
</head>
<body>
<!-- ═══════════════ HEADER ═══════════════ -->
<header>
<div class="nav-inner">
<a href="#" class="nav-name">Haggai Maron</a>
<button class="nav-toggle" aria-label="Toggle navigation" onclick="this.closest('header').querySelector('nav').classList.toggle('open')">☰</button>
<nav>
<ul>
<li><a href="#about">About</a></li>
<li><a href="#publications">Publications</a></li>
<li><a href="https://scholar.google.co.il/citations?user=4v8uJrIAAAAJ" target="_blank" rel="noopener">Google Scholar</a></li>
</ul>
</nav>
</div>
</header>
<!-- ═══════════════ MAIN ═══════════════ -->
<main>
<!-- ─────────── ABOUT ─────────── -->
<section id="about">
<div class="container">
<!-- Photo + Bio -->
<div class="about-card">
<div class="about-top">
<div class="about-photo">
<img src="images/haggai017.jpg" alt="Haggai Maron">
</div>
<div class="about-bio">
<h1>Haggai Maron</h1>
<p class="about-position">
<strong>Assistant Professor</strong> · <a href="https://ece.technion.ac.il/">Technion, Faculty of Electrical and Computer Engineering</a><br>
Senior Research Scientist · <a href="https://research.nvidia.com/labs/par/">NVIDIA Research, Tel Aviv</a>
</p>
<p>
My primary research interest is in machine learning, with a focus on deep learning for structured data. Specifically, I study how to apply deep learning techniques to sets, graphs, point clouds, surfaces, weight spaces and other mathematical objects that have an inherent symmetry structure. My goal is twofold: first, to understand and design deep learning architectures from a theoretical perspective, for example, by analyzing their expressive power; and second, to demonstrate their practical effectiveness on real-world problems involving structured data.
I completed my Ph.D. in 2019 at the Weizmann Institute of Science under the supervision of <a href="http://www.wisdom.weizmann.ac.il/~ylipman/">Prof. Yaron Lipman</a>.
Recipient of the ICML 2020 Outstanding Paper Award, the Alon Fellowship (2024), the <a href="https://sites.google.com/view/israel-cs-association/%D7%A7%D7%94%D7%99%D7%9C%D7%94-%D7%90%D7%A7%D7%93%D7%9E%D7%99%D7%AA/%D7%A4%D7%A8%D7%A1%D7%99%D7%9D-%D7%95%D7%A7%D7%95%D7%9C%D7%95%D7%AA-%D7%A7%D7%95%D7%A8%D7%90%D7%99%D7%9D?authuser=0">ICSA Early Career Award</a> (2026), and an <a href="https://www.technion.ac.il/blog/article/%d7%94%d7%91%d7%a2%d7%aa-%d7%90%d7%9e%d7%95%d7%9f-%d7%91%d7%9e%d7%a6%d7%95%d7%99%d7%a0%d7%95%d7%aa-%d7%9e%d7%93%d7%a2%d7%99%d7%aa/">ERC Starting Grant</a> on Geometric Deep Learning for Neural Artifacts (GDLNeuralArt) (2026).
</p>
<p>
<strong>I am looking for a postdoc</strong> to work on my ERC Starting Grant project (GDLNeuralArt). Ideal candidates have 5+ publications in top-tier ML conferences (or equivalent), strong mathematical foundations, a genuine enthusiasm for math, and solid coding skills. Experience in geometric deep learning or learning with symmetries is a plus. If interested, please reach out by email.
</p>
<p class="contact-line"><strong>Email:</strong> haggaimaron (at) technion.ac.il · <a href="https://scholar.google.co.il/citations?user=4v8uJrIAAAAJ">Google Scholar</a></p>
</div>
</div>
</div><!-- /.about-card -->
<!-- Group & News -->
<div class="about-subsections">
<!-- Group -->
<div class="sub-card">
<h3>Group</h3>
<div class="member-list">
<span class="member-category">Current Members</span>
<a href="https://noired.github.io/">Fabrizio Frasca</a> (Postdoc, 2024–today) ·
<a href="https://scholar.google.com/citations?user=INF2QpcAAAAJ&hl=iw">Yam Eitan</a> (PhD, 2024–today) ·
Adir Dayan (PhD, 2025–today) ·
Arkadi Piven (PhD, 2026–today, joint w/ <a href="https://ron.cswp.cs.technion.ac.il/">Ron Kimmel</a>) ·
Alina Sudakov (MSc, 2026–today) ·
Yonathan Wolanowsky (MSc, 2026–today)
<span class="member-category">Past Members</span>
<a href="https://barsguy.github.io/">Guy Bar-Shalom</a> (PhD, 2023–2026, joint w/ <a href="https://csaws.cs.technion.ac.il/~rani/">Ran El-Yaniv</a>, now postdoc at Cornell Tech) ·
<a href="https://yoavgelberg.github.io/">Yoav Gelberg</a> (MSc, 2024–2026, joint w/ <a href="https://www.cs.ox.ac.uk/people/michael.bronstein/">Michael Bronstein</a> at Oxford, now PhD at Stanford) ·
Ran Elbaz (MSc, 2023–2025) ·
Yaniv Galron (MSc, 2023–2025, joint w/ <a href="https://www.cs.bgu.ac.il/~erant/">Eran Treister</a>) ·
Yuval Aidan (MSc, 2023–2025, joint w/ <a href="https://webee.technion.ac.il/Sites/People/ayellet/">Ayellet Tal</a>) ·
Edan Kinderman (MSc, 2023–2025, joint w/ <a href="https://soudry.github.io/">Daniel Soudry</a>) ·
Ofir Haim (MSc, 2023–2025, joint w/ <a href="https://webee.technion.ac.il/Sites/People/shie/">Shie Mannor</a>)
<span class="member-category">Close Collaborators</span>
<a href="https://sites.google.com/berkeley.edu/moeputterman/home">Theo (Moe) Putterman</a> (UC Berkeley, 2023–2025) ·
<a href="https://beabevi.github.io/">Beatrice Bevilacqua</a> (Purdue, 2021–2025) ·
<a href="https://cptq.github.io/about/">Derek Lim</a> (MIT CSAIL, 2021–2025) ·
<a href="https://scholar.google.com/citations?user=44LKqBsAAAAJ&hl=iw">Moshe Eliasof</a> (Cambridge/BGU, 2022–today) ·
<a href="https://avivnavon.github.io/">Aviv Navon</a> (BIU/Aiola, 2021–today) ·
<a href="https://avivsham.github.io/">Aviv Shamsian</a> (BIU/Aiola, 2022–today)
</div>
</div>
<!-- News -->
<div class="sub-card">
<h3>News</h3>
<ul class="news-list">
<li>Recipient of the <a href="https://sites.google.com/view/israel-cs-association/%D7%A7%D7%94%D7%99%D7%9C%D7%94-%D7%90%D7%A7%D7%93%D7%9E%D7%99%D7%AA/%D7%A4%D7%A8%D7%A1%D7%99%D7%9D-%D7%95%D7%A7%D7%95%D7%9C%D7%95%D7%AA-%D7%A7%D7%95%D7%A8%D7%90%D7%99%D7%9D?authuser=0">ICSA Early Career Award</a> (2026).</li>
<li>Recipient of an <a href="https://www.technion.ac.il/blog/article/%d7%94%d7%91%d7%a2%d7%aa-%d7%90%d7%9e%d7%95%d7%9f-%d7%91%d7%9e%d7%a6%d7%95%d7%99%d7%a0%d7%95%d7%aa-%d7%9e%d7%93%d7%a2%d7%99%d7%aa/">ERC Starting Grant</a> on Geometric Deep Learning for Neural Artifacts (GDLNeuralArt).</li>
<li>Our <a href="https://artifactsasdata.org/">Neural Artifacts: A New Data Modality</a> workshop was accepted to NeurIPS 2026.</li>
<li>Technion article on our work on <a href="https://www.technion.ac.il/en/blog/article/technion-researchers-develop-an-innovative-approach-for-identifying-limitations-and-hallucinations-in-artificial-intelligence-models/">detecting hallucinations in LLMs</a>.</li>
<li><a href="https://ffabffrasca.substack.com/p/detecting-llm-misbehaviors-from-the">Blog post</a> on Detecting LLM Misbehaviors from the Inside Out with Deep Learning on Structured Data.</li>
<li>A <a href="https://docs.google.com/document/d/1MA68BzrITKP8hDNAyrAw9khklVEn0H2OgDswn67vec8/edit?usp=sharing">summary of our contributions</a> to geometric deep learning on neural artifacts, for anyone interested in joining the field.</li>
<li>Co-PI on the Vatat <a href="https://aiisrael.org.il/proposals/ai-for-science-moonshot/">Moonshot</a> grant (4 years, 9.88M NIS, 13 co-PIs) on <em>AI for All: Unlocking Scalable and Democratized Foundation Models</em>.</li>
<li>Keynotes at <a href="https://logconference.org/">LoG 2025</a> and <a href="https://www.tagds.com/events/tag-ds-2025">TAG-DS 2025</a> on <a href="https://www.youtube.com/live/sFXtZTI6O8k?si=gu5BufZuBoljyypB&t=3683">Geometric Deep Learning for Neural Artifacts</a>.</li>
<li>Short tutorial on <a href="https://youtu.be/ENW8T59-_eE?t=2728">the expressive power of GNNs</a> at the <a href="https://simons.berkeley.edu/workshops/graph-learning-meets-theoretical-computer-science/videos#simons-tabs">Simons Institute workshop</a>.</li>
<li>Recipient of the <a href="https://che.org.il/en/scholarships-integration-outstanding-faculty/">Alon scholarship for the Integration of Outstanding Faculty</a>.</li>
<li>Talk on <a href="https://www.youtube.com/watch?v=B9xFem6_ckQ&t=8182s">Equivariant architectures for learning in deep weight spaces</a>.</li>
<li><a href="https://developer.nvidia.com/blog/designing-deep-networks-to-process-other-deep-networks/">Blog post</a> on Equivariant architectures for learning in deep weight spaces.</li>
<li>Tutorial on <a href="https://www.youtube.com/watch?v=ASQYjbUBYzs">expressive graph neural networks</a> at <a href="https://logconference.org/">LoG Conference</a> (with Fabrizio Frasca and Beatrice Bevilacqua).</li>
<li><a href="https://towardsdatascience.com/using-subgraphs-for-more-expressive-gnns-8d06418d5ab">Blog post on Subgraph GNNs</a> on Towards Data Science.</li>
<li>ICML 2020 paper <em>On Learning Sets of Symmetric Elements</em> received the <strong>Outstanding Paper Award</strong>. See this <a href="https://aihub.org/2020/08/20/interview-with-haggai-maron-icml2020-award-winner/">interview</a>.</li>
</ul>
</div>
</div>
<!-- Talks & Teaching -->
<div class="about-subsections">
<!-- Talks & Media -->
<div class="sub-card">
<h3>Talks & Media</h3>
<ul class="talks-list">
<li class="talk-yt"><a href="https://www.youtube.com/live/sFXtZTI6O8k?si=gu5BufZuBoljyypB&t=3683">Geometric Deep Learning for Neural Artifacts</a> – Symmetry-Aware Learning across Trained Model Weights, Internal Representations, and Gradients</li>
<li class="talk-yt"><a href="https://youtu.be/ENW8T59-_eE?t=2728">The expressive power of GNNs – short tutorial</a> at the Simons Institute</li>
<li class="talk-yt"><a href="https://www.youtube.com/watch?v=B9xFem6_ckQ&t=8182s">Equivariant architectures for learning in deep weight spaces</a></li>
<li class="talk-yt"><a href="https://library.cirm-math.fr/Record.htm?idlist=1&record=19280739124910089119">Subgraph-based networks for expressive, efficient, and domain-independent graph learning</a> (CIRM)</li>
<li class="talk-yt"><a href="https://www.youtube.com/watch?v=k3E3XyMtQcA&list=PL7VfmMOOwNlJ5PndlhiMwjvqDTA6m1Z9i&index=2&t=120s">Leveraging Permutation Group Symmetries for Equivariant Neural Networks</a></li>
<li class="talk-pod"><a href="https://explainable.podbean.com/e/%d7%a8%d7%a9%d7%aa%d7%95%d7%aa%d7%a0%d7%95%d7%99%d7%a8%d7%95%d7%a0%d7%99%d7%9d%d7%a2%d7%9c-%d7%92%d7%a8%d7%a4%d7%99%d7%9d-%d7%a2/?fbclid=IwAR2oz8wjYc0qSwIt2clNWrFeMzNJMvV5Ycop166V6p5ZSB0JYeHF8sEkghg">Podcast: Graph neural networks</a> (Hebrew)</li>
</ul>
</div>
<!-- Teaching -->
<div class="sub-card">
<h3>Teaching</h3>
<ul class="teaching-list">
<li>2025/Spring (Technion): Introduction to Machine Learning</li>
<li>2025/Spring (Technion): Deep Learning and Groups</li>
<li>2024/Winter (Technion): Topics in Learning on Graphs</li>
<li>2024/Spring (Technion): Deep Learning and Groups</li>
<li>2023/Winter (Technion): Topics in Learning on Graphs</li>
<li>2019/Spring (WIS): Geometric and Algebraic Methods in Deep Learning</li>
<li>2018/Winter (WIS): Geometry and Deep Learning</li>
</ul>
</div>
</div>
</div>
</section>
<!-- ─────────── PUBLICATIONS ─────────── -->
<section id="publications">
<div class="container">
<h2>Publications</h2>
<!-- KV-CAT -->
<div class="pub-card">
<div class="pub-img"><img src="projects/KVCAT/rep.png" alt="KV-CAT"></div>
<div class="pub-info">
<div class="pub-title">Training Transformers for KV Cache Compressibility</div>
<div class="pub-authors">Yoav Gelberg, Yam Eitan, Michael Bronstein, Yarin Gal, Haggai Maron</div>
<div class="pub-venue">40th Annual Conference on Neural Information Processing Systems (NeurIPS 2026)</div>
<div class="pub-links">
<details class="abstract-toggle"><summary>Summary</summary>
<div class="abstract-text">Long-context language models are constrained by the KV cache, whose memory and access costs grow linearly with prefix length. Existing compression methods are applied post-training, so they are limited by how compressible the model's internal representations already are. We formalize KV compressibility as a property of the learned representations and show that almost any sequence-to-vector function can be implemented by both highly compressible and non-compressible transformers. We propose KV-Compression Aware Training (KV-CAT), a continued pretraining method that uses a train-time sparsification policy to mask KV slots, pushing the model to learn representations that compress well. KV-CAT improves the quality-budget tradeoff of downstream compression methods on retrieval, long-context QA, and perplexity evaluation.</div>
</details>
<a href="https://arxiv.org/abs/2605.05971" class="btn">Paper</a>
</div>
</div>
</div>
<!-- PE-Equiv -->
<div class="pub-card">
<div class="pub-img"><img src="projects/PE_Equiv/rep.png" alt="PE-Equiv"></div>
<div class="pub-info">
<div class="pub-title">Positional Encoding Is All You Need For Scalable Equivariance Constraint Relaxation</div>
<div class="pub-authors">Hagay Michaeli, Haggai Maron, Daniel Soudry</div>
<div class="pub-venue">40th Annual Conference on Neural Information Processing Systems (NeurIPS 2026)</div>
<div class="pub-links">
</div>
</div>
</div>
<!-- WSL-Bench -->
<div class="pub-card">
<div class="pub-img"><img src="projects/WSL_Bench/rep.png" alt="WSL-Bench"></div>
<div class="pub-info">
<div class="pub-title">Weight Space Learning needs to unify benchmarking! A taxonomy of evaluation practices</div>
<div class="pub-authors">Tobias Ettling, Damian Falk, Aron Asefaw, Léo Meynent, Konstantinos Tzevelekakis, Zehong Wang, Kai Wang, Boris Knyazev, Haggai Maron, Damian Borth</div>
<div class="pub-venue">40th Annual Conference on Neural Information Processing Systems (NeurIPS 2026)</div>
<div class="pub-award">Position Paper</div>
<div class="pub-links">
</div>
</div>
</div>
<!-- LoRWeB -->
<div class="pub-card">
<div class="pub-img"><img src="projects/LoRWeB/rep.png" alt="LoRWeB"></div>
<div class="pub-info">
<div class="pub-title">Spanning the Visual Analogy Space with a Weight Basis of LoRAs</div>
<div class="pub-authors">Hila Manor, Rinon Gal, Haggai Maron, Tomer Michaeli, Gal Chechik</div>
<div class="pub-venue">European Conference on Computer Vision (ECCV) 2026</div>
<div class="pub-links">
<details class="abstract-toggle"><summary>Summary</summary>
<div class="abstract-text">Visual analogy learning lets users edit images by example: given a pair showing a transformation, the model applies the same change to a new image. Existing methods use a single LoRA module, which struggles to cover the wide variety of possible transformations. We propose LoRWeB, which uses a learnable basis of LoRAs, each capturing a different transformation, and a lightweight encoder that weights them based on the input example pair in a single inference pass. LoRWeB achieves state-of-the-art results and better generalization to unseen transformations.</div>
</details>
<a href="https://arxiv.org/abs/2602.15727" class="btn">Paper</a>
<a href="https://research.nvidia.com/labs/par/lorweb/" class="btn">Project Page</a>
</div>
</div>
</div>
<!-- SHINE -->
<div class="pub-card">
<div class="pub-img"><img src="projects/SHINE/rep.png" alt="SHINE"></div>
<div class="pub-info">
<div class="pub-title">SHINE: A Scalable In-Context Hypernetwork for Mapping Context to LoRA in a Single Pass</div>
<div class="pub-authors">Yewei Liu, Xiyuan Wang, Yansheng Mao, Yoav Gelberg, Haggai Maron, Muhan Zhang</div>
<div class="pub-venue">International Conference on Machine Learning (ICML) 2026</div>
<div class="pub-links">
<details class="abstract-toggle"><summary>Summary</summary>
<div class="abstract-text">We propose SHINE (Scalable Hyper In-context NEtwork), a scalable hypernetwork that can map diverse meaningful contexts into high-quality LoRA adapters for large language models (LLM). By reusing the frozen LLM's own parameters in an in-context hypernetwork design and introducing architectural innovations, SHINE overcomes key limitations of prior hypernetworks and achieves strong expressive power with a relatively small number of parameters. We introduce a pretraining and instruction fine-tuning pipeline, and train our hypernetwork to generate high quality LoRA adapters from diverse meaningful contexts in a single forward pass. It updates LLM parameters without any fine-tuning, and immediately enables complex question answering tasks related to the context without directly accessing the context, effectively transforming in-context knowledge to in-parameter knowledge in one pass.</div>
</details>
<a href="https://arxiv.org/abs/2602.06358" class="btn">Paper</a>
<a href="https://github.com/Yewei-Liu/SHINE" class="btn">Code</a>
</div>
</div>
</div>
<!-- PEWSN -->
<div class="pub-card">
<div class="pub-img"><img src="projects/PEWSN/rep.png" alt="PEWSN"></div>
<div class="pub-info">
<div class="pub-title">On the Expressive Power of Permutation-Equivariant Weight-Space Networks</div>
<div class="pub-authors">Adir Dayan, Yam Eitan, Haggai Maron</div>
<div class="pub-venue">International Conference on Machine Learning (ICML) 2026</div>
<div class="pub-award">Spotlight</div>
<div class="pub-links">
<details class="abstract-toggle"><summary>Summary</summary>
<div class="abstract-text">Weight-space learning studies neural architectures that operate directly on the parameters of other neural networks. We develop a systematic theory for expressivity of weight-space networks. We first prove that all prominent permutation-equivariant networks are equivalent in expressive power. We then establish universality in both weight- and function-space settings under mild, natural assumptions on the input weights, and characterize the edge-case regimes where universality no longer holds. Together, these results provide a strong and unified foundation for the expressivity of weight-space networks.</div>
</details>
<a href="https://arxiv.org/abs/2602.01083" class="btn">Paper</a>
</div>
</div>
</div>
<!-- Rewiring -->
<div class="pub-card">
<div class="pub-img"><img src="projects/Rewiring/rep.png" alt="Rewiring"></div>
<div class="pub-info">
<div class="pub-title">From Linking Homophily and Label Informativeness to Rewiring in GNNs</div>
<div class="pub-authors">Harel Mendelman, Haggai Maron, Ronen Talmon</div>
<div class="pub-venue">TMLR 2026</div>
<div class="pub-links">
<details class="abstract-toggle"><summary>Summary</summary>
<div class="abstract-text">Graph Neural Networks perform poorly on heterophilic graphs, where connected nodes often have different classes. We study graph rewiring for this problem, linking edge homophily, GNN embedding smoothness, and classification performance theoretically. We propose a rewiring method that increases homophily using a reference graph, with guarantees on the rewired graph's homophily. For cases without a reference graph, we build a homophilic one from node features and training labels using a label-driven diffusion approach. On 11 real-world heterophilic datasets, our method outperforms existing rewiring techniques and specialized heterophily GNNs while remaining efficient and scalable.</div>
</details>
<a href="https://arxiv.org/abs/2505.12411" class="btn">Paper</a>
</div>
</div>
</div>
<!-- WSL Survey -->
<div class="pub-card">
<div class="pub-img"><img src="projects/WSL_Survey/rep.png" alt="WSL Survey"></div>
<div class="pub-info">
<div class="pub-title">A Survey of Weight Space Learning: Understanding, Representation, and Generation</div>
<div class="pub-authors">Xiaolong Han, Zehong Wang, Bo Zhao, Binchi Zhang, Jundong Li, Damian Borth, Rose Yu, Haggai Maron, Yanfang Ye, Lu Yin, Ferrante Neri</div>
<div class="pub-venue">Preprint</div>
<div class="pub-links">
<a href="https://arxiv.org/abs/2603.10090" class="btn">Paper</a>
</div>
</div>
</div>
<!-- CHARM -->
<div class="pub-card">
<div class="pub-img"><img src="projects/CHARM/rep.png" alt="CHARM"></div>
<div class="pub-info">
<div class="pub-title">Neural Message-Passing on Attention Graphs for Hallucination Detection</div>
<div class="pub-authors">Fabrizio Frasca, Guy Bar-Shalom, Yftah Ziser, Haggai Maron</div>
<div class="pub-venue">The Fourteenth International Conference on Learning Representations (ICLR 2026)</div>
<div class="pub-links">
<details class="abstract-toggle"><summary>Summary</summary>
<div class="abstract-text">Large Language Models (LLMs) often generate incorrect or unsupported content, known as hallucinations. Existing detection methods rely on heuristics or simple models over isolated computational traces such as activations, or attention maps. We unify these signals by representing them as attributed graphs, where tokens are nodes, edges follow attentional flows, and both carry features from attention scores and activations. Our approach, CHARM, casts hallucination detection as a graph learning task and tackles it by applying GNNs over the above attributed graphs. We show that CHARM provably subsumes prior attention-based heuristics and, experimentally, it consistently outperforms other leading approaches across diverse benchmarks.</div>
</details>
<a href="https://arxiv.org/abs/2509.24770" class="btn">Paper</a>
<a href="https://openreview.net/forum?id=4twbqwV4br" class="btn">OpenReview</a>
<a href="https://ffabffrasca.substack.com/p/detecting-llm-misbehaviors-from-the" class="btn">Blog Post</a>
</div>
</div>
</div>
<!-- FS-KAN -->
<div class="pub-card">
<div class="pub-img"><img src="projects/FSKAN/rep.png" alt="FS-KAN"></div>
<div class="pub-info">
<div class="pub-title">FS-KAN: Permutation Equivariant Kolmogorov-Arnold Networks via Function Sharing</div>
<div class="pub-authors">Ran Elbaz, Guy Bar-Shalom, Yam Eitan, Fabrizio Frasca, Haggai Maron</div>
<div class="pub-venue">The Fourteenth International Conference on Learning Representations (ICLR 2026)</div>
<div class="pub-links">
<details class="abstract-toggle"><summary>Summary</summary>
<div class="abstract-text">This paper introduces Function Sharing KAN (FS-KAN), a principled approach to constructing equivariant and invariant KA layers for arbitrary permutation symmetry groups, unifying and significantly extending previous work in this domain. We derive the basic construction of these FS-KAN layers by generalizing parameter-sharing schemes to the Kolmogorov-Arnold setup and provide a theoretical analysis demonstrating that FS-KANs have the same expressive power as networks that use standard parameter-sharing layers. Empirical evaluations show that FS-KANs exhibit superior data efficiency compared to standard parameter-sharing layers, making them an excellent architecture choice in low-data regimes.</div>
</details>
<a href="https://arxiv.org/abs/2509.24472" class="btn">Paper</a>
<a href="https://openreview.net/forum?id=l4m4HK6gJN" class="btn">OpenReview</a>
</div>
</div>
</div>
<!-- HOD-GNN -->
<div class="pub-card">
<div class="pub-img"><img src="projects/HODGNN/rep.png" alt="HOD-GNN"></div>
<div class="pub-info">
<div class="pub-title">On The Expressive Power of GNN Derivatives</div>
<div class="pub-authors">Yam Eitan, Moshe Eliasof, Yoav Gelberg, Fabrizio Frasca, Guy Bar-Shalom, Haggai Maron</div>
<div class="pub-venue">The Fourteenth International Conference on Learning Representations (ICLR 2026)</div>
<div class="pub-links">
<details class="abstract-toggle"><summary>Summary</summary>
<div class="abstract-text">We introduce High-Order Derivative GNN (HOD-GNN), a novel method that enhances the expressivity of Message Passing Neural Networks (MPNNs) by leveraging high-order node derivatives of the base model. These derivatives generate expressive structure-aware node embeddings processed by a second GNN in an end-to-end trainable architecture. Theoretically, we show that the resulting architecture family's expressive power aligns with the WL hierarchy. We also draw deep connections between HOD-GNN, Subgraph GNNs, and popular structural encoding schemes.</div>
</details>
<a href="https://arxiv.org/abs/2510.02565" class="btn">Paper</a>
</div>
</div>
</div>
<!-- WS-KAN -->
<div class="pub-card">
<div class="pub-img"><img src="projects/WSKAN/rep.png" alt="WS-KAN"></div>
<div class="pub-info">
<div class="pub-title">A Graph Meta-Network for Learning on Kolmogorov–Arnold Networks</div>
<div class="pub-authors">Guy Bar-Shalom, Ami Tavory, Itay Evron, Maya Bechler-Speicher, Ido Guy, Haggai Maron</div>
<div class="pub-venue">The Fourteenth International Conference on Learning Representations (ICLR 2026)</div>
<div class="pub-links">
<details class="abstract-toggle"><summary>Summary</summary>
<div class="abstract-text">We show that KANs share the same permutation symmetries as MLPs, and propose the KAN-graph, a graph representation of their computation. Building on this, we develop WS-KAN, the first weight-space architecture that learns on KANs, which naturally accounts for their symmetry. We analyze WS-KAN's expressive power and construct a comprehensive 'zoo' of trained KANs as benchmarks. Across all tasks, WS-KAN consistently outperforms structure-agnostic baselines, often by a substantial margin.</div>
</details>
<a href="https://openreview.net/forum?id=ONpyYavBqR" class="btn">OpenReview</a>
</div>
</div>
</div>
<!-- HistoGraph -->
<div class="pub-card">
<div class="pub-img"><img src="projects/HistoGraph/rep.png" alt="HistoGraph"></div>
<div class="pub-info">
<div class="pub-title">Learning from Historical Activations in Graph Neural Networks</div>
<div class="pub-authors">Yaniv Galron, Hadar Sinai, Haggai Maron, Moshe Eliasof</div>
<div class="pub-venue">The Fourteenth International Conference on Learning Representations (ICLR 2026)</div>
<div class="pub-links">
<details class="abstract-toggle"><summary>Summary</summary>
<div class="abstract-text">We introduce HistoGraph, a novel two-stage attention-based final aggregation layer that first applies a unified layer-wise attention over intermediate activations, followed by node-wise attention. By modeling the evolution of node representations across layers, our HistoGraph leverages both the activation history of nodes and the graph structure to refine features used for final prediction. Empirical results demonstrate that HistoGraph offers strong performance that consistently improves traditional techniques, with particularly strong robustness in deep GNNs.</div>
</details>
<a href="https://arxiv.org/abs/2601.01123" class="btn">Paper</a>
</div>
</div>
</div>
<!-- LOS / Beyond Next Token -->
<div class="pub-card">
<div class="pub-img"><img src="projects/LOS/rep.png" alt="LOS"></div>
<div class="pub-info">
<div class="pub-title">Beyond Next Token Probabilities: Learnable, Fast Detection of Hallucinations and Data Contamination on LLM Output Distributions</div>
<div class="pub-authors">Guy Bar-Shalom*, Fabrizio Frasca*, Derek Lim, Yoav Gelberg, Yftah Ziser, Ran El-Yaniv, Gal Chechik, Haggai Maron</div>
<div class="pub-venue">The 40th Annual AAAI Conference on Artificial Intelligence (AAAI 2026)</div>
<div class="pub-venue">Spotlight at ICLR 2025 workshop 'Quantify Uncertainty and Hallucination in Foundation Models'</div>
<div class="pub-links">
<details class="abstract-toggle"><summary>Summary</summary>
<div class="abstract-text">We propose that gray-box analysis should leverage the complete observable output of LLMs, consisting of both token probabilities and the complete token distribution sequences – a unified data type we term LOS (LLM Output Signature). We develop a transformer-based approach to process LOS that theoretically guarantees approximation of existing techniques while enabling more nuanced analysis. Our approach achieves superior performance on hallucination and data contamination detection in gray-box settings, significantly outperforming existing baselines.</div>
</details>
<a href="https://arxiv.org/abs/2503.14043" class="btn">Paper</a>
<a href="https://github.com/BarSGuy/LLM-Output-Signatures-Network" class="btn">Code</a>
<a href="https://ffabffrasca.substack.com/p/detecting-llm-misbehaviors-from-the" class="btn">Blog Post</a>
</div>
</div>
</div>
<!-- LoRA / GL Equivariant Metanetworks -->
<div class="pub-card">
<div class="pub-img"><img src="projects/LoRA/rep.png" alt="LoL"></div>
<div class="pub-info">
<div class="pub-title">GL Equivariant Metanetworks for Learning on Low Rank Weight Spaces</div>
<div class="pub-authors">Theo Putterman, Derek Lim, Yoav Gelberg, Michael M. Bronstein, Stefanie Jegelka, Haggai Maron</div>
<div class="pub-venue">The Fourth Learning on Graphs Conference (LOG 2025)</div>
<div class="pub-award">Oral Presentation</div>
<div class="pub-links">
<details class="abstract-toggle"><summary>Summary</summary>
<div class="abstract-text">We investigate the potential of Learning on LoRAs (LoL), a setup where machine learning models learn and make predictions on datasets of LoRA weights. We first identify the inherent parameter symmetries of low-rank decompositions of weights and develop several symmetry-aware invariant or equivariant LoL models. In diverse experiments, we show that our LoL architectures can process LoRA weights to predict CLIP scores, finetuning data attributes, and accuracy on downstream tasks over 50,000 times faster than standard evaluation.</div>
</details>
<a href="https://arxiv.org/abs/2410.04207" class="btn">Paper</a>
</div>
</div>
</div>
<!-- ACT-ViT -->
<div class="pub-card">
<div class="pub-img"><img src="projects/ACTVIT/rep.png" alt="ACT-ViT"></div>
<div class="pub-info">
<div class="pub-title">Beyond Token Probes: Hallucination Detection via Activation Tensors with ACT-ViT</div>
<div class="pub-authors">Guy Bar-Shalom, Fabrizio Frasca, Yaniv Galron, Yftah Ziser, Haggai Maron</div>
<div class="pub-venue">39th Annual Conference on Neural Information Processing Systems (NeurIPS 2025)</div>
<div class="pub-links">
<details class="abstract-toggle"><summary>Summary</summary>
<div class="abstract-text">We introduce ACT-ViT, a Vision Transformer-inspired model that can be effectively and efficiently applied to activation tensors and supports training on data from multiple LLMs simultaneously. Through comprehensive experiments encompassing diverse LLMs and datasets, we demonstrate that ACT-ViT consistently outperforms traditional probing techniques while remaining extremely efficient for deployment. Our architecture benefits substantially from multi-LLM training, achieves strong zero-shot performance on unseen datasets, and can be transferred effectively to new LLMs through fine-tuning.</div>
</details>
<a href="https://arxiv.org/abs/2510.00296" class="btn">Paper</a>
<a href="https://ffabffrasca.substack.com/p/detecting-llm-misbehaviors-from-the" class="btn">Blog Post</a>
</div>
</div>
</div>
<!-- GradMetaNet -->
<div class="pub-card">
<div class="pub-img"><img src="projects/gradmetanet/rep.png" alt="GradMetaNet"></div>
<div class="pub-info">
<div class="pub-title">GradMetaNet: An Equivariant Architecture for Learning on Gradients</div>
<div class="pub-authors">Yoav Gelberg, Yam Eitan, Aviv Navon, Aviv Shamsian, Theo Putterman, Michael M. Bronstein, Haggai Maron</div>
<div class="pub-venue">39th Annual Conference on Neural Information Processing Systems (NeurIPS 2025)</div>
<div class="pub-links">
<details class="abstract-toggle"><summary>Summary</summary>
<div class="abstract-text">We present a principled approach for designing architectures that process gradients guided by three principles: (1) equivariant design that preserves neuron permutation symmetries, (2) processing sets of gradients across multiple data points to capture curvature information, and (3) efficient gradient representation through rank-1 decomposition. We introduce GradMetaNet, a novel architecture for learning on gradients, and prove universality results. We demonstrate GradMetaNet's effectiveness on diverse gradient-based tasks for MLPs and transformers, such as learned optimization, INR editing, and loss landscape curvature estimation.</div>
</details>
<a href="https://arxiv.org/abs/2507.01649" class="btn">Paper</a>
</div>
</div>
</div>
<!-- Temporal GNN Laplacian -->
<div class="pub-card">
<div class="pub-img"><img src="projects/temp_grad/rep.png" alt="Temporal Laplacian PE"></div>
<div class="pub-info">
<div class="pub-title">Understanding and Improving Laplacian Positional Encodings For Temporal GNNs</div>
<div class="pub-authors">Yaniv Galron, Fabrizio Frasca, Haggai Maron, Eran Treister, Moshe Eliasof</div>
<div class="pub-venue">ECML-PKDD 2025</div>
<div class="pub-links">
<details class="abstract-toggle"><summary>Summary</summary>
<div class="abstract-text">We address key challenges in supra-Laplacian positional encodings for temporal graphs: high eigendecomposition costs, limited theoretical understanding, and ambiguity about when to apply these encodings. We offer a theoretical framework connecting supra-Laplacian encodings to per-time-slice encodings, introduce novel methods to reduce computational overhead (achieving up to 56x faster runtimes), and conduct an extensive experimental study to identify which models, tasks, and datasets benefit most from these encodings.</div>
</details>
<a href="https://arxiv.org/abs/2506.01596" class="btn">Paper</a>
</div>
</div>
</div>
<!-- HyMN -->
<div class="pub-card">
<div class="pub-img"><img src="projects/HyMN/rep.png" alt="HyMN"></div>
<div class="pub-info">
<div class="pub-title">Balancing Efficiency and Expressiveness: Subgraph GNNs with Walk-Based Centrality</div>
<div class="pub-authors">Joshua Southern*, Yam Eitan, Guy Bar-Shalom, Michael Bronstein, Haggai Maron, Fabrizio Frasca*</div>
<div class="pub-venue">International Conference on Machine Learning (ICML) 2025</div>
<div class="pub-links">
<details class="abstract-toggle"><summary>Summary</summary>
<div class="abstract-text">We propose HyMN, an expressive and efficient approach that combines Subgraph GNNs and Structural Encodings (SEs) via walk-based centrality measures. By drawing a connection to perturbation analysis, we highlight the effectiveness of centrality-based sampling, which significantly reduces the computational burden of Subgraph GNNs. HyMN effectively addresses the expressiveness limitations of MPNNs while mitigating the computational costs of Subgraph GNNs, outperforming other subgraph sampling approaches while being competitive with full-bag Subgraph GNNs.</div>
</details>
<a href="https://arxiv.org/abs/2501.03113" class="btn">Paper</a>
<a href="https://github.com/jks17/HyMN/" class="btn">Code</a>
</div>
</div>
</div>
<!-- TDL / Topological Blind Spots -->
<div class="pub-card">
<div class="pub-img"><img src="projects/TDL/rep.png" alt="TDL"></div>
<div class="pub-info">
<div class="pub-title">Topological Blind Spots: Understanding and Extending Topological Deep Learning Through the Lens of Expressivity</div>
<div class="pub-authors">Yam Eitan, Yoav Gelberg, Guy Bar-Shalom, Fabrizio Frasca, Michael Bronstein, Haggai Maron</div>
<div class="pub-venue">International Conference on Learning Representations (ICLR) 2025</div>
<div class="pub-award">Oral Presentation</div>
<div class="pub-links">
<details class="abstract-toggle"><summary>Summary</summary>
<div class="abstract-text">We aim to explore both the strengths and weaknesses of higher-order message-passing (HOMP)'s expressive power and subsequently design novel architectures to address its limitations. We demonstrate HOMP's inability to distinguish between topological objects based on fundamental topological and metric properties such as diameter, orientability, planarity, and homology. We then develop two new classes of TDL models: multi-cellular networks (MCN) and scalable multi-cellular networks (SMCN), which mitigate many of HOMP's expressivity limitations.</div>
</details>
<a href="https://openreview.net/forum?id=EzjsoomYEb" class="btn">Paper</a>
</div>
</div>
</div>
<!-- Homomorphism Expressivity Spectral GNNs -->
<div class="pub-card">
<div class="pub-img"><img src="projects/spectral_homom/rep.png" alt="Spectral Homomorphism"></div>
<div class="pub-info">
<div class="pub-title">Homomorphism Expressivity of Spectral Invariant Graph Neural Networks</div>
<div class="pub-authors">Jingchu Gai, Yiheng Du, Bohang Zhang, Haggai Maron, Liwei Wang</div>
<div class="pub-venue">International Conference on Learning Representations (ICLR) 2025</div>
<div class="pub-award">Oral Presentation</div>
<div class="pub-links">
<details class="abstract-toggle"><summary>Summary</summary>
<div class="abstract-text">We address the fundamental question of the power of spectral invariants in GNNs through the lens of homomorphism expressivity, providing a comprehensive and quantitative analysis. We prove that spectral invariant GNNs can homomorphism-count exactly a class of specific tree-like graphs referred to as parallel trees, and highlight the significance of this result in establishing quantitative expressiveness hierarchies across different architectural variants.</div>
</details>
<a href="https://openreview.net/forum?id=rdv6yeMFpn" class="btn">Paper</a>
</div>
</div>
</div>
<!-- Lightning-Fast Image Inversion -->
<div class="pub-card">
<div class="pub-img"><img src="projects/lightning-fast/rep.png" alt="Lightning-Fast Inversion"></div>
<div class="pub-info">
<div class="pub-title">Lightning-Fast Image Inversion and Editing for Text-to-Image Diffusion Models</div>
<div class="pub-authors">Dvir Samuel, Barak Meiri, Haggai Maron, Yoad Tewel, Nir Darshan, Shai Avidan, Gal Chechik, Rami Ben-Ari</div>
<div class="pub-venue">International Conference on Learning Representations (ICLR) 2025</div>
<div class="pub-links">
<details class="abstract-toggle"><summary>Summary</summary>
<div class="abstract-text">We formulate diffusion inversion as finding the roots of an implicit equation and design a method based on Newton-Raphson (NR). We derive an efficient guided formulation that fastly converges and provides high-quality reconstructions and editing. Our solution, Guided Newton-Raphson Inversion, inverts an image within 0.4 sec (on an A100 GPU) for few-step models (SDXL-Turbo and Flux.1), opening the door for interactive image editing.</div>
</details>
<a href="https://openreview.net/forum?id=t9l63huPRt" class="btn">Paper</a>
</div>
</div>
</div>
<!-- Directed Graph Generation Heat Kernels -->
<div class="pub-card">
<div class="pub-img"><img src="projects/heat/rep.png" alt="Heat Kernels"></div>
<div class="pub-info">
<div class="pub-title">Directed Graph Generation with Heat Kernels</div>
<div class="pub-authors">Marc T Law, Karsten Kreis, Haggai Maron</div>
<div class="pub-venue">TMLR 2025</div>
<div class="pub-links">
<details class="abstract-toggle"><summary>Summary</summary>
<div class="abstract-text">We propose a denoising autoencoder-based generative model that exploits the global structure of directed graphs via their Laplacian dynamics and enables one-shot generation. Our approach generalizes a special class of exponential kernels over discrete structures, called diffusion kernels or heat kernels, to the non-symmetric case via Reproducing Kernel Banach Spaces. We demonstrate that our model is able to generate directed graphs that follow the distribution of the training dataset even if it is multimodal.</div>
</details>
<a href="https://openreview.net/forum?id=60Gi1w6hte" class="btn">Paper</a>
</div>
</div>
</div>
<!-- Foldable Supernets -->
<div class="pub-card">
<div class="pub-img"><img src="projects/foldable/rep.png" alt="Foldable Supernets"></div>
<div class="pub-info">
<div class="pub-title">Foldable Supernets: Scalable Merging of Transformers with Different Initializations and Tasks</div>
<div class="pub-authors">Edan Kinderman, Itay Hubara, Haggai Maron, Daniel Soudry</div>
<div class="pub-venue">TMLR 2025</div>
<div class="pub-links">
<details class="abstract-toggle"><summary>Summary</summary>
<div class="abstract-text">We propose Foldable SuperNet Merge (FS-Merge), a method that optimizes a SuperNet to fuse original models using a feature reconstruction loss. FS-Merge is simple, data-efficient, and capable of merging models of varying widths. We test FS-Merge against existing methods on MLPs and transformers across various settings, sizes, tasks, and modalities. FS-Merge consistently outperforms them, achieving SOTA results, particularly in limited data scenarios.</div>
</details>
<a href="https://arxiv.org/abs/2410.01483" class="btn">Paper</a>
</div>
</div>
</div>
<!-- Flexible Equivariant Framework for Subgraph GNNs -->
<div class="pub-card">
<div class="pub-img"><img src="projects/coarsening/rep.png" alt="Graph Coarsening Subgraph GNN"></div>
<div class="pub-info">
<div class="pub-title">A Flexible, Equivariant Framework for Subgraph GNNs via Graph Products and Graph Coarsening</div>
<div class="pub-authors">Guy Bar-Shalom, Yam Eitan, Fabrizio Frasca, Haggai Maron</div>
<div class="pub-venue">38th Annual Conference on Neural Information Processing Systems (NeurIPS 2024)</div>
<div class="pub-award">Best Paper Award at Symmetry and Geometry in Neural Representations Workshop @ NeurIPS 2024</div>
<div class="pub-links">
<details class="abstract-toggle"><summary>Summary</summary>
<div class="abstract-text">We introduce a new Subgraph GNNs framework employing a graph coarsening function to cluster nodes into super-nodes. The product between the coarsened and original graph reveals an implicit structure whereby subgraphs are associated with specific sets of nodes. By running generalized message-passing on such graph product, our method effectively implements an efficient yet powerful Subgraph GNN. We discover that the resulting node feature tensor exhibits new, unexplored permutation symmetries and leverage this structure to characterize the associated linear equivariant layers.</div>
</details>
<a href="https://arxiv.org/abs/2406.09291" class="btn">Paper</a>
</div>
</div>
</div>
<!-- TracksTo4D -->
<div class="pub-card">
<div class="pub-img"><img src="projects/4d/rep.png" alt="TracksTo4D"></div>
<div class="pub-info">
<div class="pub-title">Fast Encoder-Based 3D from Casual Videos via Point Track Processing</div>
<div class="pub-authors">Yoni Kasten, Wuyue Lu, Haggai Maron</div>
<div class="pub-venue">38th Annual Conference on Neural Information Processing Systems (NeurIPS 2024)</div>
<div class="pub-links">
<details class="abstract-toggle"><summary>Summary</summary>
<div class="abstract-text">We present TracksTo4D, a learning-based approach that enables inferring 3D structure and camera positions from dynamic content originating from casual videos using a single efficient feed-forward pass. We propose operating directly over 2D point tracks as input and designing an architecture tailored for processing 2D point tracks. TracksTo4D is trained in an unsupervised way and can reconstruct a temporal point cloud and camera positions with accuracy comparable to state-of-the-art methods, while drastically reducing runtime by up to 95%.</div>
</details>
<a href="https://arxiv.org/abs/2404.07097" class="btn">Paper</a>
</div>
</div>
</div>
<!-- GRANOLA -->
<div class="pub-card">
<div class="pub-img"><img src="projects/granola/rep.png" alt="GRANOLA"></div>
<div class="pub-info">
<div class="pub-title">GRANOLA: Adaptive Normalization for Graph Neural Networks</div>
<div class="pub-authors">Moshe Eliasof, Beatrice Bevilacqua, Carola-Bibiane Schönlieb, Haggai Maron</div>
<div class="pub-venue">38th Annual Conference on Neural Information Processing Systems (NeurIPS 2024)</div>
<div class="pub-links">
<details class="abstract-toggle"><summary>Summary</summary>
<div class="abstract-text">We propose GRANOLA, a novel graph-adaptive normalization layer. Unlike existing normalization layers, GRANOLA normalizes node features by adapting to the specific characteristics of the graph, particularly by generating expressive representations of its neighborhood structure via Random Node Features (RNF). Our extensive empirical evaluation underscores the superior performance of GRANOLA over existing normalization techniques, emerging as the top-performing method within the same time complexity of MPNNs.</div>
</details>
<a href="https://arxiv.org/abs/2404.13344" class="btn">Paper</a>
</div>
</div>
</div>
<!-- Empirical Impact of Neural Parameter Symmetries -->
<div class="pub-card">
<div class="pub-img"><img src="projects/asymetric_networks/rep.png" alt="Parameter Symmetries"></div>
<div class="pub-info">
<div class="pub-title">The Empirical Impact of Neural Parameter Symmetries, or Lack Thereof</div>
<div class="pub-authors">Derek Lim*, Theo (Moe) Putterman*, Robin Walters, Haggai Maron, Stefanie Jegelka</div>
<div class="pub-venue">38th Annual Conference on Neural Information Processing Systems (NeurIPS 2024)</div>
<div class="pub-award">Best Paper Award at HiLD 2024: 2nd Workshop on High-dimensional Learning Dynamics @ ICML 2024</div>
<div class="pub-links">
<details class="abstract-toggle"><summary>Summary</summary>
<div class="abstract-text">We empirically investigate the impact of neural parameter symmetries by introducing new neural network architectures that have reduced parameter space symmetries. We develop two methods of modifying standard neural networks to reduce parameter space symmetries and conduct a comprehensive experimental study assessing the effect of removing these symmetries. Our experiments reveal several interesting observations; for instance, we observe linear mode connectivity and monotonic linear interpolation in our networks, without any alignment of weight spaces.</div>
</details>
<a href="https://arxiv.org/abs/2405.20231" class="btn">Paper</a>
<a href="https://openreview.net/pdf?id=gkrynRGiOi" class="btn">Workshop Paper</a>
</div>
</div>
</div>
<!-- Foundation Models on Graphs -->
<div class="pub-card">
<div class="pub-img"><img src="projects/graph_found/rep.png" alt="Graph Foundation Models"></div>
<div class="pub-info">
<div class="pub-title">Towards Foundation Models on Graphs: An Analysis on Cross-Dataset Transfer of Pretrained GNNs</div>
<div class="pub-authors">Fabrizio Frasca, Fabian Jogl, Moshe Eliasof, Matan Ostrovsky, Carola-Bibiane Schönlieb, Thomas Gärtner, Haggai Maron</div>
<div class="pub-venue">Symmetry and Geometry in Neural Representations (NeurReps) @ NeurIPS 2024</div>
<div class="pub-links">
<details class="abstract-toggle"><summary>Summary</summary>
<div class="abstract-text">We study the extent to which pretrained Graph Neural Networks can be applied across datasets, requiring feature-agnostic approaches. We build upon a purely structural pretraining approach and propose an extension to capture feature information while still being feature-agnostic. Our preliminary results indicate that embeddings from pretrained models improve generalization only with enough downstream data points and in a degree which depends on the quantity and properties of pretraining data.</div>
</details>
<a href="https://arxiv.org/abs/2412.17609" class="btn">Paper</a>
</div>
</div>
</div>
<!-- Future Directions Graph ML -->
<div class="pub-card">
<div class="pub-img"><img src="projects/future_graph_learning/rep.png" alt="Future Graph ML"></div>
<div class="pub-info">
<div class="pub-title">Future Directions in Foundations of Graph Machine Learning</div>
<div class="pub-authors">Christopher Morris, Nadav Dym, Haggai Maron, İsmail İlkan Ceylan, Fabrizio Frasca, Ron Levie, Derek Lim, Michael Bronstein, Martin Grohe, Stefanie Jegelka</div>
<div class="pub-venue">International Conference on Machine Learning (ICML) 2024</div>
<div class="pub-links">
<details class="abstract-toggle"><summary>Summary</summary>
<div class="abstract-text">We argue that the graph machine learning community needs to shift its attention to developing a more balanced theory of graph machine learning, focusing on a more thorough understanding of the interplay of expressive power, generalization, and optimization.</div>
</details>
<a href="https://arxiv.org/abs/2402.02287" class="btn">Paper</a>
</div>
</div>
</div>
<!-- Spectral Expressive Power -->
<div class="pub-card">
<div class="pub-img"><img src="projects/spectral_expressive_power/rep.png" alt="Spectral Expressive Power"></div>
<div class="pub-info">
<div class="pub-title">On the Expressive Power of Spectral Invariant Graph Neural Networks</div>
<div class="pub-authors">Bohang Zhang, Lingxiao Zhao, Haggai Maron</div>
<div class="pub-venue">International Conference on Machine Learning (ICML) 2024</div>
<div class="pub-links">
<details class="abstract-toggle"><summary>Summary</summary>
<div class="abstract-text">We introduce a novel message-passing framework for designing spectral invariant GNNs, called Eigenspace Projection GNN (EPNN). Our comprehensive analysis shows that EPNN essentially unifies all prior spectral invariant architectures, and they are either strictly less expressive or equivalent to EPNN. We present a surprising result that EPNN itself is bounded by a recently proposed class of Subgraph GNNs, implying that all spectral invariant architectures are strictly less expressive than 3-WL.</div>
</details>
<a href="https://arxiv.org/abs/2406.04336" class="btn">Paper</a>
</div>
</div>
</div>
<!-- Deep Align -->
<div class="pub-card">
<div class="pub-img"><img src="projects/network_align/rep.png" alt="Deep-Align"></div>
<div class="pub-info">
<div class="pub-title">Equivariant Deep Weight Space Alignment</div>
<div class="pub-authors">Aviv Navon, Aviv Shamsian, Ethan Fetaya, Gal Chechik, Nadav Dym, Haggai Maron</div>
<div class="pub-venue">International Conference on Machine Learning (ICML) 2024</div>
<div class="pub-links">
<details class="abstract-toggle"><summary>Summary</summary>
<div class="abstract-text">We propose Deep-Align, a novel framework aimed at learning to solve the weight alignment problem. We demonstrate that weight alignment adheres to two fundamental symmetries and propose a deep architecture that respects these symmetries. Our framework does not require any labeled data. Our experimental results indicate that a feed-forward pass with Deep-Align produces better or equivalent alignments compared to those produced by current optimization algorithms.</div>
</details>
<a href="https://arxiv.org/abs/2310.13397" class="btn">Paper</a>
<a href="https://github.com/AvivNavon/deep-align" class="btn">GitHub</a>
</div>
</div>
</div>
<!-- Subgraphormer -->
<div class="pub-card">
<div class="pub-img"><img src="projects/subgraphormer/rep.png" alt="Subgraphormer"></div>
<div class="pub-info">
<div class="pub-title">Subgraphormer: Unifying Subgraph GNNs and Graph Transformers via Graph Products</div>
<div class="pub-authors">Guy Bar-Shalom, Beatrice Bevilacqua, Haggai Maron</div>
<div class="pub-venue">International Conference on Machine Learning (ICML) 2024</div>
<div class="pub-links">
<details class="abstract-toggle"><summary>Summary</summary>
<div class="abstract-text">We propose Subgraphormer, an architecture that integrates Subgraph GNNs and Graph Transformers, combining enhanced expressive power and message-passing mechanisms from Subgraph GNNs with attention and positional encodings from Graph Transformers. Our method is based on a new connection between Subgraph GNNs and product graphs, suggesting that Subgraph GNNs can be formulated as MPNNs operating on a product of the graph with itself.</div>
</details>
<a href="https://arxiv.org/abs/2402.08450" class="btn">Paper</a>
</div>
</div>
</div>
<!-- Weight Space Augmentations -->
<div class="pub-card">
<div class="pub-img"><img src="projects/weight_aug/rep.png" alt="Weight Space Augmentation"></div>
<div class="pub-info">
<div class="pub-title">Improved Generalization of Weight Space Networks via Augmentations</div>
<div class="pub-authors">Aviv Shamsian, Aviv Navon, David W. Zhang, Yan Zhang, Ethan Fetaya, Gal Chechik, Haggai Maron</div>
<div class="pub-venue">International Conference on Machine Learning (ICML) 2024</div>
<div class="pub-award">Oral Presentation at NeurReps Workshop @ NeurIPS 2023</div>
<div class="pub-links">
<details class="abstract-toggle"><summary>Summary</summary>
<div class="abstract-text">We empirically analyze the reasons for overfitting in deep weight space models and find that a key reason is the lack of diversity in DWS datasets. We explore strategies for data augmentation in weight spaces and propose a MixUp method adapted for weight spaces. These methods improve classification performance similarly to having up to 10 times more data, and yield substantial 5–10% gains in downstream classification under contrastive learning.</div>
</details>
<a href="https://arxiv.org/abs/2402.04081" class="btn">Paper</a>
<a href="https://github.com/AvivSham/deep-weight-space-augmentations" class="btn">GitHub</a>
</div>
</div>
</div>
<!-- Policy-Learn / Efficient Subgraph GNNs -->
<div class="pub-card">
<div class="pub-img"><img src="projects/choose_subgraphs/rep.png" alt="Policy-Learn"></div>
<div class="pub-info">
<div class="pub-title">Efficient Subgraph GNNs by Learning Effective Selection Policies</div>
<div class="pub-authors">Beatrice Bevilacqua, Moshe Eliasof, Eli Meirom, Bruno Ribeiro, Haggai Maron</div>
<div class="pub-venue">International Conference on Learning Representations (ICLR) 2024</div>
<div class="pub-links">
<details class="abstract-toggle"><summary>Summary</summary>
<div class="abstract-text">We consider the problem of learning to select a small subset of the large set of possible subgraphs in a data-driven fashion. We prove that there are families of WL-indistinguishable graphs for which there exist efficient subgraph selection policies. We then propose Policy-Learn, that learns how to select subgraphs in an iterative manner. We prove that, unlike popular random policies and prior work, our architecture is able to learn these efficient policies.</div>
</details>
<a href="https://arxiv.org/abs/2310.20082" class="btn">Paper</a>
</div>
</div>
</div>
<!-- Graph Metanetworks -->
<div class="pub-card">
<div class="pub-img"><img src="projects/metanetworks/rep.png" alt="Graph Metanetworks"></div>
<div class="pub-info">
<div class="pub-title">Graph Metanetworks for Processing Diverse Neural Architectures</div>
<div class="pub-authors">Derek Lim, Haggai Maron, Marc T. Law, Jonathan Lorraine, James Lucas</div>
<div class="pub-venue">International Conference on Learning Representations (ICLR) 2024</div>
<div class="pub-award">Spotlight Presentation</div>
<div class="pub-links">
<details class="abstract-toggle"><summary>Summary</summary>
<div class="abstract-text">We build new metanetworks – neural networks that take weights from other neural networks as input – by carefully building graphs representing the input neural networks and processing them using graph neural networks. Our approach, Graph Metanetworks (GMNs), generalizes to neural architectures where competing methods struggle, such as multi-head attention layers, normalization layers, convolutional layers, ResNet blocks, and group-equivariant linear layers. We prove that GMNs are expressive and equivariant to parameter permutation symmetries.</div>
</details>
<a href="https://arxiv.org/abs/2312.04501" class="btn">Paper</a>
</div>
</div>
</div>
<!-- Sign Equivariant Networks -->
<div class="pub-card">
<div class="pub-img"><img src="projects/signequi/rep.png" alt="Sign Equivariant Networks"></div>
<div class="pub-info">
<div class="pub-title">Expressive Sign Equivariant Networks for Spectral Geometric Learning</div>
<div class="pub-authors">Derek Lim, Joshua Robinson, Stefanie Jegelka, Haggai Maron</div>
<div class="pub-venue">37th Annual Conference on Neural Information Processing Systems (NeurIPS 2023)</div>
<div class="pub-award">Spotlight Presentation</div>
<div class="pub-links">
<details class="abstract-toggle"><summary>Summary</summary>
<div class="abstract-text">We demonstrate that sign equivariance is useful for applications such as building orthogonally equivariant models and link prediction. We develop novel sign equivariant neural network architectures based on an analytic characterization of the sign equivariant polynomials and thus inherit provable expressiveness properties.</div>
</details>
</div>
</div>
</div>
<!-- Norm-guided latent space -->
<div class="pub-card">
<div class="pub-img"><img src="projects/norm-guided/rep.png" alt="Norm-guided latent space"></div>
<div class="pub-info">
<div class="pub-title">Norm-Guided Latent Space Exploration for Text-to-Image Generation</div>
<div class="pub-authors">Dvir Samuel, Rami Ben-Ari, Nir Darshan, Haggai Maron, Gal Chechik</div>
<div class="pub-venue">37th Annual Conference on Neural Information Processing Systems (NeurIPS 2023)</div>
<div class="pub-links">
<details class="abstract-toggle"><summary>Summary</summary>
<div class="abstract-text">We make the observation that current training procedures make diffusion models biased toward inputs with a narrow range of norm values. We propose a novel method for interpolating between two seeds that defines a new non-Euclidean metric taking into account a norm-based prior on seeds. We show that our new interpolation and centroid evaluation techniques significantly enhance the generation of rare concept images, leading to state-of-the-art performance on few-shot and long-tail benchmarks.</div>
</details>
<a href="https://arxiv.org/abs/2306.08687" class="btn">Paper</a>
<a href="https://github.com/dvirsamuel/SeedSelect" class="btn">Code</a>
</div>
</div>
</div>
<!-- DWS Nets -->
<div class="pub-card">
<div class="pub-img"><img src="projects/DWS/rep.png" alt="DWS Nets"></div>
<div class="pub-info">
<div class="pub-title">Equivariant Architectures for Learning in Deep Weight Spaces</div>
<div class="pub-authors">Aviv Navon, Aviv Shamsian, Idan Achituve, Ethan Fetaya, Gal Chechik, Haggai Maron</div>
<div class="pub-venue">International Conference on Machine Learning (ICML) 2023</div>
<div class="pub-award">Oral Presentation</div>
<div class="pub-links">
<details class="abstract-toggle"><summary>Summary</summary>
<div class="abstract-text">We present a novel network architecture for learning in deep weight spaces. It takes as input a concatenation of weights and biases of a pre-trained MLP and processes it using a composition of layers that are equivariant to the natural permutation symmetry of the MLP's weights. We provide a full characterization of all affine equivariant and invariant layers for these symmetries and demonstrate the effectiveness of our architecture on a variety of learning tasks.</div>
</details>
<a href="https://arxiv.org/abs/2301.12780" class="btn">Paper</a>
<a href="https://www.youtube.com/watch?v=B9xFem6_ckQ&t=8182s" class="btn">Talk</a>
<a href="https://avivnavon.github.io/DWSNets/" class="btn">Code</a>
<a href="https://developer.nvidia.com/blog/designing-deep-networks-to-process-other-deep-networks/" class="btn">Blog</a>
</div>
</div>
</div>
<!-- RFP -->
<div class="pub-card">
<div class="pub-img"><img src="projects/RFP/rep.png" alt="RFP"></div>
<div class="pub-info">
<div class="pub-title">Graph Positional Encoding via Random Feature Propagation</div>
<div class="pub-authors">Moshe Eliasof, Fabrizio Frasca, Beatrice Bevilacqua, Eran Treister, Gal Chechik, Haggai Maron</div>
<div class="pub-venue">International Conference on Machine Learning (ICML) 2023</div>
<div class="pub-links">
<details class="abstract-toggle"><summary>Summary</summary>
<div class="abstract-text">We propose Random Feature Propagation (RFP), a novel family of positional encoding schemes that draws a link between random features and spectral positional encoding. RFP is inspired by the power iteration method and concatenates several intermediate steps of an iterative algorithm for computing dominant eigenvectors of a propagation matrix, starting from random node features. RFP significantly outperforms both spectral PE and random features in multiple node classification and graph classification benchmarks.</div>
</details>
<a href="https://arxiv.org/abs/2303.02918" class="btn">Paper</a>
</div>
</div>
</div>
<!-- Equivariant Polynomials -->
<div class="pub-card">
<div class="pub-img"><img src="projects/equipoly/rep.png" alt="Equivariant Polynomials"></div>
<div class="pub-info">
<div class="pub-title">Equivariant Polynomials for Graph Neural Networks</div>
<div class="pub-authors">Omri Puny, Derek Lim, Bobak T. Kiani, Haggai Maron, Yaron Lipman</div>
<div class="pub-venue">International Conference on Machine Learning (ICML) 2023</div>
<div class="pub-award">Oral Presentation</div>
<div class="pub-links">
<details class="abstract-toggle"><summary>Summary</summary>
<div class="abstract-text">We introduce an alternative expressive power hierarchy for GNNs based on their ability to calculate equivariant polynomials of a certain degree. We provide a full characterization of all equivariant graph polynomials by introducing a concrete basis, significantly generalizing previous results. We propose algorithmic tools for evaluating the expressiveness of GNNs using tensor contraction sequences, and enhance the expressivity of common GNN architectures by adding polynomial features or additional operations inspired by our theory.</div>
</details>
<a href="https://arxiv.org/abs/2302.11556" class="btn">Paper</a>
<a href="https://github.com/omri1348/Equivariant-Polynomaials" class="btn">Code</a>
</div>
</div>
</div>
<!-- SignNet / BasisNet -->
<div class="pub-card">
<div class="pub-img"><img src="projects/signnet/rep.png" alt="SignNet"></div>
<div class="pub-info">
<div class="pub-title">Sign and Basis Invariant Networks for Spectral Graph Representation Learning</div>
<div class="pub-authors">Derek Lim*, Joshua Robinson*, Lingxiao Zhao, Tess Smidt, Suvrit Sra, Haggai Maron, Stefanie Jegelka</div>
<div class="pub-venue">International Conference on Learning Representations (ICLR 2023)</div>
<div class="pub-award">Notable-top-25% (Spotlight)</div>
<div class="pub-links">
<details class="abstract-toggle"><summary>Summary</summary>
<div class="abstract-text">We introduce SignNet and BasisNet – new neural architectures that are invariant to all requisite symmetries and hence process collections of eigenspaces in a principled manner. Our networks are universal and theoretically strong for graph representation learning – they can approximate any spectral graph convolution, compute spectral invariants beyond MPNNs, and provably simulate previously proposed graph positional encodings.</div>
</details>
<a href="https://arxiv.org/abs/2202.13013" class="btn">Paper</a>
<a href="https://github.com/cptq/SignNet-BasisNet" class="btn">Code</a>
</div>
</div>
</div>
<!-- WL Survey -->
<div class="pub-card">
<div class="pub-img"><img src="projects/wl_survey/rep.png" alt="WL Survey"></div>
<div class="pub-info">
<div class="pub-title">Weisfeiler and Leman go Machine Learning: The Story so far</div>
<div class="pub-authors">Christopher Morris, Yaron Lipman, Haggai Maron, Bastian Rieck, Nils M. Kriege, Martin Grohe, Matthias Fey, Karsten Borgwardt</div>
<div class="pub-venue">JMLR 2023</div>
<div class="pub-links">
<details class="abstract-toggle"><summary>Summary</summary>
<div class="abstract-text">We give a comprehensive overview of the Weisfeiler-Leman algorithm's use in a machine learning setting, focusing on the supervised regime. We discuss the theoretical background, show how to use it for supervised graph- and node representation learning, discuss recent extensions, and outline the algorithm's connection to permutation-equivariant neural architectures.</div>
</details>
<a href="https://arxiv.org/abs/2112.09992" class="btn">Paper</a>
</div>
</div>
</div>
<!-- Rethinking Subgraph GNNs / SUN -->
<div class="pub-card">
<div class="pub-img"><img src="projects/rethinking_subgraphs/rep.png" alt="SUN / Rethinking Subgraph GNNs"></div>
<div class="pub-info">
<div class="pub-title">Understanding and Extending Subgraph GNNs by Rethinking Their Symmetries</div>
<div class="pub-authors">Fabrizio Frasca*, Beatrice Bevilacqua*, Michael M. Bronstein, Haggai Maron</div>
<div class="pub-venue">36th Annual Conference on Neural Information Processing Systems (NeurIPS 2022)</div>
<div class="pub-award">Oral Presentation (1.7% acceptance rate)</div>
<div class="pub-links">
<details class="abstract-toggle"><summary>Summary</summary>
<div class="abstract-text">We study the most prominent form of subgraph methods employing node-based subgraph selection policies. A novel symmetry analysis shows that modeling the symmetries of node-based subgraph collections requires a significantly smaller symmetry group than the one adopted in previous works. We bound the expressive power of subgraph methods by 3-WL and propose SUN, a general family of message-passing layers for subgraph methods that generalizes all previous node-based Subgraph GNNs.</div>
</details>
<a href="https://arxiv.org/abs/2206.11140" class="btn">Paper</a>
<a href="https://github.com/beabevi/SUN" class="btn">Code</a>
<a href="https://library.cirm-math.fr/Record.htm?idlist=1&record=19280739124910089119" class="btn">Talk</a>
</div>
</div>
</div>
<!-- Generalized Laplacian PE -->
<div class="pub-card">
<div class="pub-img"><img src="projects/gen_pos_enc/rep.png" alt="Generalized Laplacian PE"></div>
<div class="pub-info">
<div class="pub-title">Generalized Laplacian Positional Encoding for Graph Representation Learning</div>
<div class="pub-authors">Sohir Maskey*, Ali Parviz*, Maximilian Thiessen, Hannes Stärk, Ylli Sadikaj, Haggai Maron</div>
<div class="pub-venue">NeurIPS 2022 Workshop on Symmetry and Geometry in Neural Representations</div>
<div class="pub-links">
<details class="abstract-toggle"><summary>Summary</summary>
<div class="abstract-text">We define a novel family of positional encoding schemes for graphs by generalizing the optimization problem that defines the Laplace embedding to more general dissimilarity functions rather than the 2-norm used in the original formulation. This family of positional encodings is instantiated by considering p-norms and can improve the expressive power of MPNNs.</div>
</details>
<a href="https://arxiv.org/abs/2210.15956" class="btn">Paper</a>
</div>
</div>
</div>
<!-- Simple Rotation Equivariant -->
<div class="pub-card">
<div class="pub-img"><img src="projects/simple_rot_equi/rep.png" alt="Rotation Equivariant Network"></div>
<div class="pub-info">
<div class="pub-title">A Simple and Universal Rotation Equivariant Point-Cloud Network</div>
<div class="pub-authors">Ben Finkelshtein, Chaim Baskin, Haggai Maron, Nadav Dym</div>
<div class="pub-venue">Workshop on Topology, Algebra, and Geometry in Learning, ICML 2022</div>
<div class="pub-links">
<details class="abstract-toggle"><summary>Summary</summary>
<div class="abstract-text">We suggest a much simpler architecture than the equivariant Tensor Field Network, prove that it enjoys the same universality guarantees, and evaluate its performance on Modelnet40.</div>
</details>
<a href="https://arxiv.org/abs/2203.01216" class="btn">Paper</a>
<a href="https://github.com/simpleinvariance/UniversalNetwork" class="btn">Code</a>
</div>
</div>
</div>
<!-- StyleGAN-NADA -->
<div class="pub-card">
<div class="pub-img"><img src="projects/NADA/rep.png" alt="StyleGAN-NADA"></div>
<div class="pub-info">
<div class="pub-title">StyleGAN-NADA: CLIP-Guided Domain Adaptation of Image Generators</div>
<div class="pub-authors">Rinon Gal, Or Patashnik, Haggai Maron, Gal Chechik, Daniel Cohen-Or</div>
<div class="pub-venue">ACM SIGGRAPH 2022</div>
<div class="pub-links">
<details class="abstract-toggle"><summary>Summary</summary>
<div class="abstract-text">Leveraging the semantic power of large scale CLIP models, we present a text-driven method that allows shifting a generative model to new domains without collecting even a single image from those domains. Through natural language prompts and a few minutes of training, our method can adapt a generator across a multitude of domains characterized by diverse styles and shapes.</div>
</details>
<a href="https://arxiv.org/abs/2108.00946" class="btn">Paper</a>
<a href="https://github.com/rinongal/StyleGAN-nada" class="btn">Code</a>
<a href="https://replicate.com/rinongal/stylegan-nada" class="btn">Interactive Demo</a>
</div>
</div>
</div>
<!-- Nash-MTL -->
<div class="pub-card">
<div class="pub-img"><img src="projects/mtl/rep.png" alt="Nash-MTL"></div>
<div class="pub-info">
<div class="pub-title">Multi-Task Learning as a Bargaining Game</div>
<div class="pub-authors">Aviv Navon, Aviv Shamsian, Idan Achituve, Haggai Maron, Kenji Kawaguchi, Gal Chechik, Ethan Fetaya</div>
<div class="pub-venue">International Conference on Machine Learning (ICML) 2022</div>
<div class="pub-links">
<details class="abstract-toggle"><summary>Summary</summary>
<div class="abstract-text">We propose viewing the gradients combination step in MTL as a bargaining game, where tasks negotiate to reach an agreement on a joint direction of parameter update. Under certain assumptions, the bargaining problem has a unique solution, known as the Nash Bargaining Solution, which we propose to use as a principled approach to multi-task learning. We describe Nash-MTL and derive theoretical guarantees for its convergence, achieving state-of-the-art results on multiple benchmarks.</div>
</details>
<a href="https://arxiv.org/abs/2202.01017" class="btn">Paper</a>
</div>
</div>
</div>
<!-- Tensor Contraction RL -->
<div class="pub-card">
<div class="pub-img"><img src="projects/tensor_contraction/rep.png" alt="Tensor Contraction RL"></div>
<div class="pub-info">
<div class="pub-title">Optimizing Tensor Network Contraction Using Reinforcement Learning</div>
<div class="pub-authors">Eli A Meirom, Haggai Maron, Shie Mannor, Gal Chechik</div>
<div class="pub-venue">International Conference on Machine Learning (ICML) 2022</div>
<div class="pub-links">
<details class="abstract-toggle"><summary>Summary</summary>
<div class="abstract-text">We propose an RL approach combined with GNNs to address the contraction ordering problem for quantum circuit simulation. We show how a carefully implemented RL-agent using a GNN as the basic policy construct can obtain significant improvements over state-of-the-art techniques in three varieties of circuits.</div>
</details>
<a href="https://arxiv.org/abs/2204.09052" class="btn">Paper</a>
</div>
</div>
</div>
<!-- Federated Learning with Graph HyperNetworks -->
<div class="pub-card">
<div class="pub-img"><img src="projects/federated_graph_hn/rep.png" alt="Federated Graph HyperNetworks"></div>
<div class="pub-info">
<div class="pub-title">Federated Learning with Heterogeneous Architectures using Graph HyperNetworks</div>
<div class="pub-authors">Or Litany, Haggai Maron, David Acuna, Jan Kautz, Gal Chechik, Sanja Fidler</div>
<div class="pub-venue">Technical report, 2022</div>
<div class="pub-links">
<details class="abstract-toggle"><summary>Summary</summary>
<div class="abstract-text">We propose a new FL framework that accommodates heterogeneous client architectures by adopting a graph hypernetwork for parameter sharing. Unlike existing solutions, our framework does not limit clients to sharing the same architecture type, makes no use of external data, and does not require clients to disclose their model architecture.</div>
</details>
<a href="https://arxiv.org/abs/2201.08459" class="btn">Paper</a>
</div>
</div>
</div>
<!-- ESAN -->
<div class="pub-card">
<div class="pub-img"><img src="projects/ESAN/rep.png" alt="ESAN"></div>
<div class="pub-info">
<div class="pub-title">Equivariant Subgraph Aggregation Networks</div>
<div class="pub-authors">Beatrice Bevilacqua*, Fabrizio Frasca*, Derek Lim*, Balasubramaniam Srinivasan, Chen Cai, Gopinath Balamurugan, Michael M. Bronstein, Haggai Maron (*equal contribution)</div>
<div class="pub-venue">International Conference on Learning Representations (ICLR) 2022</div>
<div class="pub-award">Spotlight Presentation (5% acceptance rate)</div>
<div class="pub-links">
<details class="abstract-toggle"><summary>Summary</summary>
<div class="abstract-text">We propose ESAN, a novel framework where each graph is represented as a set of subgraphs derived by some predefined policy, and processed using a suitable equivariant architecture. We develop novel variants of the 1-WL test for graph isomorphism and prove lower bounds on the expressiveness of ESAN. We further prove that our approach increases the expressive power of both MPNNs and more expressive architectures.</div>
</details>
<a href="https://arxiv.org/abs/2110.02910" class="btn">Paper</a>
<a href="https://github.com/beabevi/ESAN" class="btn">GitHub</a>
<a href="https://towardsdatascience.com/using-subgraphs-for-more-expressive-gnns-8d06418d5ab" class="btn">Blog Post</a>
<a href="https://library.cirm-math.fr/Record.htm?idlist=1&record=19280739124910089119" class="btn">Talk</a>
</div>
</div>
</div>