获取Pytorch中间某一层权重或者特征的例子
问题:训练好的网络模型想知道中间某一层的权重或者看看中间某一层的特征,如何处理呢?
1、获取某一层权重,并保存到excel中;
以resnet18为例说明:
importtorch
importpandasaspd
importnumpyasnp
importtorchvision.modelsasmodels
resnet18=models.resnet18(pretrained=True)
parm={}
forname,parametersinresnet18.named_parameters():
print(name,':',parameters.size())
parm[name]=parameters.detach().numpy()
上述代码将每个模块参数存入parm字典中,parameters.detach().numpy()将tensor类型变量转换成numpyarray形式,方便后续存储到表格中.输出为：
conv1.weight:torch.Size([64,3,7,7]) bn1.weight:torch.Size([64]) bn1.bias:torch.Size([64]) layer1.0.conv1.weight:torch.Size([64,64,3,3]) layer1.0.bn1.weight:torch.Size([64]) layer1.0.bn1.bias:torch.Size([64]) layer1.0.conv2.weight:torch.Size([64,64,3,3]) layer1.0.bn2.weight:torch.Size([64]) layer1.0.bn2.bias:torch.Size([64]) layer1.1.conv1.weight:torch.Size([64,64,3,3]) layer1.1.bn1.weight:torch.Size([64]) layer1.1.bn1.bias:torch.Size([64]) layer1.1.conv2.weight:torch.Size([64,64,3,3]) layer1.1.bn2.weight:torch.Size([64]) layer1.1.bn2.bias:torch.Size([64]) layer2.0.conv1.weight:torch.Size([128,64,3,3]) layer2.0.bn1.weight:torch.Size([128]) layer2.0.bn1.bias:torch.Size([128]) layer2.0.conv2.weight:torch.Size([128,128,3,3]) layer2.0.bn2.weight:torch.Size([128]) layer2.0.bn2.bias:torch.Size([128]) layer2.0.downsample.0.weight:torch.Size([128,64,1,1]) layer2.0.downsample.1.weight:torch.Size([128]) layer2.0.downsample.1.bias:torch.Size([128]) layer2.1.conv1.weight:torch.Size([128,128,3,3]) layer2.1.bn1.weight:torch.Size([128]) layer2.1.bn1.bias:torch.Size([128]) layer2.1.conv2.weight:torch.Size([128,128,3,3]) layer2.1.bn2.weight:torch.Size([128]) layer2.1.bn2.bias:torch.Size([128]) layer3.0.conv1.weight:torch.Size([256,128,3,3]) layer3.0.bn1.weight:torch.Size([256]) layer3.0.bn1.bias:torch.Size([256]) layer3.0.conv2.weight:torch.Size([256,256,3,3]) layer3.0.bn2.weight:torch.Size([256]) layer3.0.bn2.bias:torch.Size([256]) layer3.0.downsample.0.weight:torch.Size([256,128,1,1]) layer3.0.downsample.1.weight:torch.Size([256]) layer3.0.downsample.1.bias:torch.Size([256]) layer3.1.conv1.weight:torch.Size([256,256,3,3]) layer3.1.bn1.weight:torch.Size([256]) layer3.1.bn1.bias:torch.Size([256]) layer3.1.conv2.weight:torch.Size([256,256,3,3]) layer3.1.bn2.weight:torch.Size([256]) layer3.1.bn2.bias:torch.Size([256]) layer4.0.conv1.weight:torch.Size([512,256,3,3]) layer4.0.bn1.weight:torch.Size([512]) layer4.0.bn1.bias:torch.Size([512]) layer4.0.conv2.weight:torch.Size([512,512,3,3]) layer4.0.bn2.weight:torch.Size([512]) layer4.0.bn2.bias:torch.Size([512]) layer4.0.downsample.0.weight:torch.Size([512,256,1,1]) layer4.0.downsample.1.weight:torch.Size([512]) layer4.0.downsample.1.bias:torch.Size([512]) layer4.1.conv1.weight:torch.Size([512,512,3,3]) layer4.1.bn1.weight:torch.Size([512]) layer4.1.bn1.bias:torch.Size([512]) layer4.1.conv2.weight:torch.Size([512,512,3,3]) layer4.1.bn2.weight:torch.Size([512]) layer4.1.bn2.bias:torch.Size([512]) fc.weight:torch.Size([1000,512]) fc.bias:torch.Size([1000])
parm['layer1.0.conv1.weight'][0,0,:,:]
输出为:
array([[0.05759342,-0.09511436,-0.02027232], [-0.07455588,-0.799308,-0.21283598], [0.06557069,-0.09653367,-0.01211061]],dtype=float32)
利用如下函数将某一层的所有参数保存到表格中,数据维持卷积核特征大小,如3*3的卷积保存后还是3x3的.
defparm_to_excel(excel_name,key_name,parm): withpd.ExcelWriter(excel_name)aswriter: [output_num,input_num,filter_size,_]=parm[key_name].size() foriinrange(output_num): forjinrange(input_num): data=pd.DataFrame(parm[key_name][i,j,:,:].detach().numpy()) #print(data) data.to_excel(writer,index=False,header=True,startrow=i*(filter_size+1),startcol=j*filter_size)
由于权重矩阵中有很多的值非常小,取出固定大小的值,并将全部权重写入excel
counter=1
withpd.ExcelWriter('test1.xlsx')aswriter:
forkeyinparm_resnet50.keys():
data=parm_resnet50[key].reshape(-1,1)
data=data[data>0.001]
data=pd.DataFrame(data,columns=[key])
data.to_excel(writer,index=False,startcol=counter)
counter+=1
2、获取中间某一层的特性
重写一个函数,将需要输出的层输出即可.
defresnet_cifar(net,input_data): x=net.conv1(input_data) x=net.bn1(x) x=F.relu(x) x=net.layer1(x) x=net.layer2(x) x=net.layer3(x) x=net.layer4[0].conv1(x)#这样就提取了layer4第一块的第一个卷积层的输出 x=x.view(x.shape[0],-1) returnx model=models.resnet18() x=resnet_cifar(model,input_data)
以上这篇获取Pytorch中间某一层权重或者特征的例子就是小编分享给大家的全部内容了，希望能给大家一个参考，也希望大家多多支持毛票票。