哪类小网站容易做,安徽建设厅网站证件查询,最专业网站建设哪家好,电子工程网如何删除帖子项目地址 #xff1a;https://github.com/billvsme/train_law_llm
✏️LLM微调上手项目
一步一步使用Colab训练法律LLM#xff0c;基于microsoft/phi-1_5 。通过本项目你可以0成本手动了解微调LLM。
nameColabDatasets自我认知lora-SFT微调train_self_cognition.ipynbsel…
项目地址 https://github.com/billvsme/train_law_llm
✏️LLM微调上手项目
一步一步使用Colab训练法律LLM基于microsoft/phi-1_5 。通过本项目你可以0成本手动了解微调LLM。
nameColabDatasets自我认知lora-SFT微调train_self_cognition.ipynbself_cognition.json法律问答lora-SFT微调train_law.ipynbDISC-LawLLM法律问答 全参数-SFT微调*train_law_full.ipynbDISC-LawLLM
*如果是Colab Pro会员用户可以尝试全参数-SFT微调使用高内存T41000条数据大概需要20小时
目标
使用colab免费的T4显卡完成法律问答 指令监督微调(SFT) microsoft/phi-1_5 模型
自我认知微调
自我认知数据来源self_cognition.json
80条数据使用T4 lora微调phi-1_5几分钟就可以微调完毕
微调参数具体步骤详见colab
python src/train_bash.py \--stage sft \--model_name_or_path microsoft/phi-1_5 \--do_train True\--finetuning_type lora \--template vanilla \--flash_attn False \--shift_attn False \--dataset_dir data \--dataset self_cognition \--cutoff_len 1024 \--learning_rate 2e-04 \--num_train_epochs 20.0 \--max_samples 1000 \--per_device_train_batch_size 6 \--per_device_eval_batch_size 6 \--gradient_accumulation_steps 1 \--lr_scheduler_type cosine \--max_grad_norm 1.0 \--logging_steps 5 \--save_steps 100 \--warmup_steps 0 \--neft_alpha 0 \--train_on_prompt False \--upcast_layernorm False \--lora_rank 8 \--lora_dropout 0.1 \--lora_target Wqkv \--resume_lora_training True \--output_dir saves/Phi1.5-1.3B/lora/my \--fp16 True \--plot_loss True效果 法律问答微调
法律问答数据来源DISC-LawLLM 为了减省显存使用deepspeed stage2cutoff_len可以最多到1792再多就要爆显存了
deepspeed配置
{train_batch_size: auto,train_micro_batch_size_per_gpu: auto,gradient_accumulation_steps: auto,gradient_clipping: auto,zero_allow_untested_optimizer: true,fp16: {enabled: auto,loss_scale: 0,initial_scale_power: 16,loss_scale_window: 1000,hysteresis: 2,min_loss_scale: 1},zero_optimization: {stage: 2,offload_optimizer: {device: cpu,pin_memory: true},allgather_partitions: true,allgather_bucket_size: 2e8,reduce_scatter: true,reduce_bucket_size: 2e8,overlap_comm: false,contiguous_gradients: true}
}微调参数
1000条数据T4大概需要60分钟
deepspeed --num_gpus 1 --master_port9901 src/train_bash.py \--deepspeed ds_config.json \--stage sft \--model_name_or_path microsoft/phi-1_5 \--do_train True \--finetuning_type lora \--template vanilla \--flash_attn False \--shift_attn False \--dataset_dir data \--dataset self_cognition,law_sft_triplet \--cutoff_len 1792 \--learning_rate 2e-04 \--num_train_epochs 5.0 \--max_samples 1000 \--per_device_train_batch_size 1 \--per_device_eval_batch_size 1 \--gradient_accumulation_steps 1 \--lr_scheduler_type cosine \--max_grad_norm 1.0 \--logging_steps 5 \--save_steps 1000 \--warmup_steps 0 \--neft_alpha 0 \--train_on_prompt False \--upcast_layernorm False \--lora_rank 8 \--lora_dropout 0.1 \--lora_target Wqkv \--resume_lora_training True \--output_dir saves/Phi1.5-1.3B/lora/law \--fp16 True \--plot_loss True全参微调
可以通过estimate_zero3_model_states_mem_needs_all_live查看deepspeed各个ZeRO stage 所需要的内存。
from transformers import AutoModel, AutoModelForCausalLM
from deepspeed.runtime.zero.stage3 import estimate_zero3_model_states_mem_needs_all_livemodel_name microsoft/phi-1_5
model AutoModelForCausalLM.from_pretrained(model_name, trust_remote_codeTrue)
estimate_zero3_model_states_mem_needs_all_live(model, num_gpus_per_node1, num_nodes1)如图所适 offload_optimizer - cpu 后microsoft/phi-1_5 需要32G内存colab高内存有52G可以满足需求。 deepspeed配置
{train_batch_size: auto,train_micro_batch_size_per_gpu: auto,gradient_accumulation_steps: auto,gradient_clipping: auto,zero_allow_untested_optimizer: true,fp16: {enabled: auto,loss_scale: 0,initial_scale_power: 16,loss_scale_window: 1000,hysteresis: 2,min_loss_scale: 1},zero_optimization: {stage: 2,offload_optimizer: {device: cpu,pin_memory: true},allgather_partitions: true,allgather_bucket_size: 2e8,reduce_scatter: true,reduce_bucket_size: 2e8,overlap_comm: false,contiguous_gradients: true}
}deepspeed --num_gpus 1 --master_port9901 src/train_bash.py \--deepspeed ds_config.json \--stage sft \--model_name_or_path microsoft/phi-1_5 \--do_train True \--finetuning_type full \--template vanilla \--flash_attn False \--shift_attn False \--dataset_dir data \--dataset self_cognition,law_sft_triplet \--cutoff_len 1024 \--learning_rate 2e-04 \--num_train_epochs 10.0 \--max_samples 1000 \--per_device_train_batch_size 1 \--per_device_eval_batch_size 1 \--gradient_accumulation_steps 1 \--lr_scheduler_type cosine \--max_grad_norm 1.0 \--logging_steps 5 \--save_steps 1000 \--warmup_steps 0 \--neft_alpha 0 \--train_on_prompt False \--upcast_layernorm False \--lora_rank 8 \--lora_dropout 0.1 \--lora_target Wqkv \--resume_lora_training True \--output_dir saves/Phi1.5-1.3B/lora/law_full \--fp16 True \--plot_loss True
