load_model=‘/mnt/c/Users/Administrator/RWKV-PEFT/model/RWKV-x060-World-1B6-v2.1-20240328-ctx4096.pth’
proj_dir=‘/mnt/c/Users/Administrator/RWKV-PEFT/out_model/roleplay’
data_file=‘/mnt/c/Users/Administrator/RWKV-PEFT/data/roleplay’
n_layer=24
n_embd=2048
micro_bsz=1
epoch_save=1
epoch_steps=500
ctx_len=512
python train.py --load_model $load_model \
–proj_dir $proj_dir --data_file $data_file \
–data_type binidx --vocab_size 65536 \
–ctx_len $ctx_len --epoch_steps $epoch_steps --epoch_count 5 --epoch_begin 0 --epoch_save $epoch_save --micro_bsz $micro_bsz \
–n_layer $n_layer --n_embd $n_embd \
–pre_ffn 0 --head_qk 0 --lr_init 1 --lr_final 1e-2 --warmup_steps 10 --beta1 0.9 --beta2 0.99 --adam_eps 1e-8 \
–accelerator gpu --devices 1 --precision bf16 --strategy deepspeed_stage_1 --grad_cp 1 \
–my_testing “x060” \
–train_type “state” --dataload pad --wandb PEFT-State-tuning
以下为可选项
–fla
–quant int8/nf4 （是否量化训练）
–wandb PEFT-State-tuning （是否使用 wandb 监控训练过程）
demo-state-tuning.sh设置内容，以下是提示错误内容：
[rank0]: with open(path, “rb”) as stream:
[rank0]: FileNotFoundError: [Errno 2] No such file or directory: ‘/mnt/c/Users/Administrator/RWKV-PEFT/data/roleplay.idx’
Exception ignored in: <function MMapIndexedDataset.Index.del at 0x7f9d5a09bc70>
Traceback (most recent call last):
File “/mnt/c/Users/Administrator/RWKV-PEFT/src/binidx.py”, line 150, in del
self._bin_buffer_mmap._mmap.close()
AttributeError: ‘Index’ object has no attribute ‘_bin_buffer_mmap’
Exception ignored in: <function MMapIndexedDataset.del at 0x7f9d59ecc280>
Traceback (most recent call last):
File “/mnt/c/Users/Administrator/RWKV-PEFT/src/binidx.py”, line 202, in del
self._bin_buffer_mmap._mmap.close()
AttributeError: ‘MMapIndexedDataset’ object has no attribute ‘_bin_buffer_mmap’
[rank0]:[W1222 13:23:09.784221997 ProcessGroupNCCL.cpp:1250] Warning: WARNING: process group has NOT been destroyed before we destruct ProcessGroupNCCL. On normal program exit, the application should call destroy_process_group to ensure that any pending NCCL operations have finished in this process. In rare cases this process can exit before this point and block the progress of another member of the process group. This constraint has always been present, but this warning has only been added since PyTorch 2.4 (function operator())
是哪里设置错误吗？？
在各位大佬的帮助下终于明白了。
总结，没有BIn的文件名，记住只要文件名不要结尾的.bin
不好意思，没说明白。统一回复，替换以下文件名即可。
data_file=‘/mnt/c/Users/Administrator/RWKV-PEFT/data/roleplay/文件名’
2024-12-22-20:22 编辑RWKVdocument 中 微调环境 搭建 及 sate微调 教程 都遗漏了 streamlit安装。
No module named ‘streamlit’
当运行出现以上提示请使用以下命令在linux环境下安装：
conda install streamlit
或
pip install streamlit
国内建议使用清华镜像源如下：
pip install streamlit -i https://pypi.tsinghua.edu.cn/simple
更新完毕！如有错误请提醒。
