Issues

• Thank you very much for your code. Do you have the code for how to count the memory in Table 1? Thank you very much

  OpenLMLab/LOMO#80 · Filter by author LiZhangMing opened on Nov 28, 2024
  1

• Serious conclusion: LOMO does not significantly reduce GPU memory usage！

  OpenLMLab/LOMO#72 · Filter by author misonsky opened on Jan 3, 2024
  19

• 实测LOMO++ deepspeed zero2 7b qlora llama 显存占用 感觉比正常的qlora + deepspeed zero2 显存占用大1倍

  OpenLMLab/LOMO#70 · Filter by author zlh1992 opened on Dec 26, 2023
  1

• Can you provide detailed dependency versions?

  OpenLMLab/LOMO#69 · Filter by author misonsky opened on Dec 23, 2023

• adalomo在使用chatglm2模型出现错误

  OpenLMLab/LOMO#68 · Filter by author JorunoJobana opened on Nov 7, 2023
  2

• adalomo 试图在 llama 2 70b 模型训练时出现 NCCL communicator 这类超时的错误

  OpenLMLab/LOMO#67 · Filter by author HappyLynn opened on Nov 7, 2023
  5

• eval environment for opencompass

  OpenLMLab/LOMO#65 · Filter by author KaiLv69 opened on Oct 31, 2023
  4

• Runtime error on 2nd epoch, trying lora only

  OpenLMLab/LOMO#56 · Filter by author wasifferoze opened on Oct 3, 2023

• a bug found in save_model of LOMOTrainer

  OpenLMLab/LOMO#54 · Filter by author DingQiang2018 opened on Aug 30, 2023
  10

  

• LlaMA-7B + LoRA在16GB的V100上OOM

  OpenLMLab/LOMO#53 · Filter by author zhenqincn opened on Aug 30, 2023
  2

• Customized loss value

  OpenLMLab/LOMO#52 · Filter by author ZN1010 opened on Aug 22, 2023
  4

• 请问自定义Dataset只能是classification的数据集吗

  OpenLMLab/LOMO#51 · Filter by author Tonystark64 opened on Jul 31, 2023
  1

• 小学习率问题

  OpenLMLab/LOMO#50 · Filter by author wpcool opened on Jul 31, 2023
  1

• 选取bloom-1b7作为model和wic作为数据集出现IndexError: tuple index out of range

  OpenLMLab/LOMO#49 · Filter by author Tonystark64 opened on Jul 28, 2023
  1

• CLIP梯度和梯度overflow的影响

  OpenLMLab/LOMO#48 · Filter by author tzjtatata opened on Jul 21, 2023
  2

• 为什么LOMO并没有火起来呢？

  OpenLMLab/LOMO#47 · Filter by author Flywolfs opened on Jul 21, 2023
  5

• Memory Usage continues to grow

  OpenLMLab/LOMO#45 · Filter by author Jetcodery opened on Jul 18, 2023
  5

• 关于代码理解和显存占用的问题

  OpenLMLab/LOMO#41 · Filter by author ghost opened on Jul 15, 2023
  1

• How to calculate the used GPU memory for each part as in the paper?

  OpenLMLab/LOMO#36 · Filter by author liming-ai opened on Jul 6, 2023
  2

• about torch.stack(self.grad_norms)

  OpenLMLab/LOMO#30 · Filter by author jinzitian opened on Jun 30, 2023
  3

• llama-33B/llama-65B均报OOM，8*V100跑不起来怎么回事呢？

  OpenLMLab/LOMO#28 · Filter by author alisyzhu opened on Jun 29, 2023
  7

• Some confusion about the method of the paper

  OpenLMLab/LOMO#27 · Filter by author JorunoJobana opened on Jun 29, 2023
  3

• Memory consumption first grows up then falls down.

  OpenLMLab/LOMO#26 · Filter by author zhenqin96 opened on Jun 29, 2023
  3

  

• Performance Model after Full Fine-tuning by LOMOTrainer

  OpenLMLab/LOMO#25 · Filter by author dat-browny opened on Jun 28, 2023
  9

• 更充分实验，与Adam的实验效果进行比较

  OpenLMLab/LOMO#21 · Filter by author yangjianxin1 opened on Jun 26, 2023
  6