Qwen3-4B-LoRA-ZH-WebNovelty-v0.0

基于阿里巴巴 Qwen/Qwen3-4B，使用 LoRA 对中文网络小说写作任务进行微调，适合“场景＋类型”指令下的创作。

模型概览

• Base Model：Qwen/Qwen3-4B
• 微调框架：LoRA (Low-Rank Adaptation)
• 微调任务：根据“场景”+“类型”指令生成小说段落

超参数配置

参数	值	说明
Batch Size	16	每卡样本数
Learning Rate	1e-4	学习率
Epochs	2	训练轮次
Packing	Enabled	多条样本拼接，提升显存利用率
LoRA Rank	16	低秩分解秩
LoRA Alpha	8	LoRA 缩放因子
LoRA Dropout	0.05	LoRA 层 Dropout 概率
Warmup Ratio	0	预热步数占比
Weight Decay	0	权重衰减
Seed	114514	随机种子

数据格式

训练和验证使用 Instruction 格式的 JSONL：

{"prompt":"请根据下面的场景和类型创作一段文字：\n场景：古代茶馆相遇\n类型：dialogue","completion":"“这茶香真不错”，他轻声说道。"}

• prompt：包括场景说明与创作类型
• completion：对应要生成的小说段落

使用示例

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

# 1. 加载基础模型（原始 Instruct 或带 Adapter 的 repo）
base_model = AutoModelForCausalLM.from_pretrained(
"Qwen/Qwen3-4B",
trust_remote_code=True,
load_in_8bit=True, # 量化方式可选
device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained(
"Qwen/Qwen3-4B",
trust_remote_code=True
)

# 2. 加载 LoRA Adapter
model = PeftModel.from_pretrained(
base_model,
"TanXS/Qwen3-4B-LoRA-ZH-WebNovelty-v0.0"
)

# 3. 推理示例
prompt = "请根据场景写一段旁白：夜色下的古城塔楼矗立…"
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=200)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Qwen3-4B-LoRA-ZH-WebNovelty-v0.0

Fine-tuned on Chinese web novel writing using LoRA, based on Alibaba’s Qwen/Qwen3-4B. Suitable for “Scene + Type” instruction-driven creative generation in Chinese.

Model Overview

• Base Model: Qwen/Qwen3-4B
• Fine-tuning Framework: LoRA (Low-Rank Adaptation)
• Task: Generate novel passages given a “Scene” and “Type” instruction

Hyperparameters

Parameter	Value	Description
Batch Size	16	Samples per GPU
Learning Rate	1e-4	Learning rate
Epochs	2	Number of training epochs
Packing	Enabled	Concatenate multiple samples for efficiency
LoRA Rank	16	Low-rank decomposition rank
LoRA Alpha	8	Scaling factor for the LoRA update
LoRA Dropout	0.05	Dropout rate in LoRA layers
Warmup Ratio	0	No learning-rate warmup
Weight Decay	0	No weight decay
Seed	114514	Random seed

Data Format

Training and validation use an Instruction JSONL format:

{"prompt":"请根据下面的场景和类型创作一段文字：\n场景：古代茶馆相遇\n类型：dialogue","completion":"“这茶香真不错”，他轻声说道。"}

• prompt: Includes scene description and creative type
• completion: The target novel passage to generate

Usage Example

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

# 1. Load the base model (original Instruct or with Adapter)
base_model = AutoModelForCausalLM.from_pretrained(
"Qwen/Qwen3-4B",
trust_remote_code=True,
load_in_8bit=True, # Optional quantization
device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained(
"Qwen/Qwen3-4B",
trust_remote_code=True
)

# 2. Load the LoRA Adapter
model = PeftModel.from_pretrained(
base_model,
"TanXS/Qwen3-4B-LoRA-ZH-WebNovelty-v0.0"
)

# 3. Inference example
prompt = "请根据场景写一段旁白：夜色下的古城塔楼矗立…"
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=200)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))