tensorflow模型转ncnn的操作方式
第一步把tensorflow保存的.ckpt模型转为pb模型,并记下模型的输入输出名字.
第二步去ncnn的github上把仓库clone下来,按照上面的要求装好依赖并make.
第三步是修改ncnn的CMakeList,具体修改的位置有:
ncnn/CMakeList.txt文件,在文件开头处加入add_definitions(-std=c++11),末尾处加上add_subdirectory(examples),如果ncnn没有examples文件夹,就新建一个,并加上CMakeList.txt文件.
ncnn/tools/CMakeList.txt文件,加入add_subdirectory(tensorflow)
原版的tools/tensorflow/tensorflow2ncnn.cpp里,不支持tensorflow的elu,FusedBathNormalization,Conv2dBackpropback操作,其实elu是支持的,只需要仿照relu的格式,在.cpp文件里加上就行.FusedBatchNormalization就是ncnn/layer/里实现的batchnorm.cpp,只是`tensorflow2ncnn里没有写上,可以增加下面的内容：
elseif(node.op()=="FusedBatchNorm")
{
fprintf(pp,"%-16s","BatchNorm");
}
...
elseif(node.op()=="FusedBatchNorm")
{
std::cout<<"nodenameisFusedBatchNorm"<(gamma.tensor_content().c_str());
constfloat*mean_data=reinterpret_cast(mean.tensor_content().c_str());
constfloat*var_data=reinterpret_cast(var.tensor_content().c_str());
constfloat*b_data=reinterpret_cast(Beta.tensor_content().c_str());
for(inti=0;i
同理,Conv2dBackpropback其实就是ncnn里的反卷积操作,只不过ncnn实现反卷积的操作和tensorflow内部实现反卷积的操作过程不一样,但结果是一致的,需要仿照普通卷积的写法加上去.
ncnn同样支持空洞卷积,但无法识别tensorflow的空洞卷积,具体原理可以看tensorflow空洞卷积的原理,tensorflow是改变featuremap做空洞卷积,而ncnn是改变kernel做空洞卷积,结果都一样.需要对.proto文件修改即可完成空洞卷积.
总之ncnn对tensorflow的支持很不友好,有的层还需要自己手动去实现,还是很麻烦.
补充知识：pytorch模型转mxnet
介绍
gluon把mxnet再进行封装，封装的风格非常接近pytorch
使用gluon的好处是非常容易把pytorch模型向mxnet转化
唯一的问题是gluon封装还不成熟，封装好的layer不多，很多常用的layer如concat,upsampling等layer都没有
这里关注如何把pytorch模型快速转换成mxnet基于symbol和exector设计的网络
pytorch转mxnetmodule
关键点:
mxnet设计网络时symbol名称要和pytorch初始化中各网络层名称对应
torch.load()读入pytorch模型checkpoint字典，取当中的'state_dict'元素，也是一个字典
pytorchstate_dict字典中key是网络层参数的名称，val是参数ndarray
pytorch的参数名称的组织形式和mxnet一样，但是连接符号不同，pytorch是'.'，而mxnet是'_'比如：
pytorch'0.conv1.0.weight'
mxnet'0_conv1_0_weight'
pytorch的参数array和mxnet的参数array完全一样，只要名称对上，直接赋值即可初始化mxnet模型
需要做的有以下几点：
设计和pytorch网络对应的mxnet网络
加载pytorchcheckpoint
调整pytorchcheckpointstate_dict的key名称和mxnet命名格式一致
FlowNet2SPytorchToMxnet
pytorchflownet2S的checkpoint可以在github上搜到
importmxnetasmx
fromsymbol_utilimport*
importpickle
defget_loss(data,label,loss_scale,name,get_input=False,is_sparse=False,type='stereo'):
iftype=='stereo':
data=mx.sym.Activation(data=data,act_type='relu',name=name+'relu')
#loss
ifis_sparse:
loss=mx.symbol.Custom(data=data,label=label,name=name,loss_scale=loss_scale,is_l1=True,
op_type='SparseRegressionLoss')
else:
loss=mx.sym.MAERegressionOutput(data=data,label=label,name=name,grad_scale=loss_scale)
return(loss,data)ifget_inputelseloss
defflownet_s(loss_scale,is_sparse=False,name=''):
img1=mx.symbol.Variable('img1')
img2=mx.symbol.Variable('img2')
data=mx.symbol.concat(img1,img2,dim=1)
labels={'loss{}'.format(i):mx.sym.Variable('loss{}_label'.format(i))foriinrange(0,7)}
#print('labels:',labels)
prediction={}#adictforlosscollection
loss=[]#alist
#normalize
data=(data-125)/255
#extractfeatrue
conv1=mx.sym.Convolution(data,pad=(3,3),kernel=(7,7),stride=(2,2),num_filter=64,name=name+'conv1_0')
conv1=mx.sym.LeakyReLU(data=conv1,act_type='leaky',slope=0.1)
conv2=mx.sym.Convolution(conv1,pad=(2,2),kernel=(5,5),stride=(2,2),num_filter=128,name=name+'conv2_0')
conv2=mx.sym.LeakyReLU(data=conv2,act_type='leaky',slope=0.1)
conv3a=mx.sym.Convolution(conv2,pad=(2,2),kernel=(5,5),stride=(2,2),num_filter=256,name=name+'conv3_0')
conv3a=mx.sym.LeakyReLU(data=conv3a,act_type='leaky',slope=0.1)
conv3b=mx.sym.Convolution(conv3a,pad=(1,1),kernel=(3,3),stride=(1,1),num_filter=256,name=name+'conv3_1_0')
conv3b=mx.sym.LeakyReLU(data=conv3b,act_type='leaky',slope=0.1)
conv4a=mx.sym.Convolution(conv3b,pad=(1,1),kernel=(3,3),stride=(2,2),num_filter=512,name=name+'conv4_0')
conv4a=mx.sym.LeakyReLU(data=conv4a,act_type='leaky',slope=0.1)
conv4b=mx.sym.Convolution(conv4a,pad=(1,1),kernel=(3,3),stride=(1,1),num_filter=512,name=name+'conv4_1_0')
conv4b=mx.sym.LeakyReLU(data=conv4b,act_type='leaky',slope=0.1)
conv5a=mx.sym.Convolution(conv4b,pad=(1,1),kernel=(3,3),stride=(2,2),num_filter=512,name=name+'conv5_0')
conv5a=mx.sym.LeakyReLU(data=conv5a,act_type='leaky',slope=0.1)
conv5b=mx.sym.Convolution(conv5a,pad=(1,1),kernel=(3,3),stride=(1,1),num_filter=512,name=name+'conv5_1_0')
conv5b=mx.sym.LeakyReLU(data=conv5b,act_type='leaky',slope=0.1)
conv6a=mx.sym.Convolution(conv5b,pad=(1,1),kernel=(3,3),stride=(2,2),num_filter=1024,name=name+'conv6_0')
conv6a=mx.sym.LeakyReLU(data=conv6a,act_type='leaky',slope=0.1)
conv6b=mx.sym.Convolution(conv6a,pad=(1,1),kernel=(3,3),stride=(1,1),num_filter=1024,
name=name+'conv6_1_0')
conv6b=mx.sym.LeakyReLU(data=conv6b,act_type='leaky',slope=0.1,)
#predictflow
pr6=mx.sym.Convolution(conv6b,pad=(1,1),kernel=(3,3),stride=(1,1),num_filter=2,
name=name+'predict_flow6')
prediction['loss6']=pr6
upsample_pr6to5=mx.sym.Deconvolution(pr6,pad=(1,1),kernel=(4,4),stride=(2,2),num_filter=2,
name=name+'upsampled_flow6_to_5',no_bias=True)
upconv5=mx.sym.Deconvolution(conv6b,pad=(1,1),kernel=(4,4),stride=(2,2),num_filter=512,
name=name+'deconv5_0',no_bias=False)
upconv5=mx.sym.LeakyReLU(data=upconv5,act_type='leaky',slope=0.1)
iconv5=mx.sym.Concat(conv5b,upconv5,upsample_pr6to5,dim=1)
pr5=mx.sym.Convolution(iconv5,pad=(1,1),kernel=(3,3),stride=(1,1),num_filter=2,
name=name+'predict_flow5')
prediction['loss5']=pr5
upconv4=mx.sym.Deconvolution(iconv5,pad=(1,1),kernel=(4,4),stride=(2,2),num_filter=256,
name=name+'deconv4_0',no_bias=False)
upconv4=mx.sym.LeakyReLU(data=upconv4,act_type='leaky',slope=0.1)
upsample_pr5to4=mx.sym.Deconvolution(pr5,pad=(1,1),kernel=(4,4),stride=(2,2),num_filter=2,
name=name+'upsampled_flow5_to_4',no_bias=True)
iconv4=mx.sym.Concat(conv4b,upconv4,upsample_pr5to4)
pr4=mx.sym.Convolution(iconv4,pad=(1,1),kernel=(3,3),stride=(1,1),num_filter=2,
name=name+'predict_flow4')
prediction['loss4']=pr4
upconv3=mx.sym.Deconvolution(iconv4,pad=(1,1),kernel=(4,4),stride=(2,2),num_filter=128,
name=name+'deconv3_0',no_bias=False)
upconv3=mx.sym.LeakyReLU(data=upconv3,act_type='leaky',slope=0.1)
upsample_pr4to3=mx.sym.Deconvolution(pr4,pad=(1,1),kernel=(4,4),stride=(2,2),num_filter=2,
name=name+'upsampled_flow4_to_3',no_bias=True)
iconv3=mx.sym.Concat(conv3b,upconv3,upsample_pr4to3)
pr3=mx.sym.Convolution(iconv3,pad=(1,1),kernel=(3,3),stride=(1,1),num_filter=2,
name=name+'predict_flow3')
prediction['loss3']=pr3
upconv2=mx.sym.Deconvolution(iconv3,pad=(1,1),kernel=(4,4),stride=(2,2),num_filter=64,
name=name+'deconv2_0',no_bias=False)
upconv2=mx.sym.LeakyReLU(data=upconv2,act_type='leaky',slope=0.1)
upsample_pr3to2=mx.sym.Deconvolution(pr3,pad=(1,1),kernel=(4,4),stride=(2,2),num_filter=2,
name=name+'upsampled_flow3_to_2',no_bias=True)
iconv2=mx.sym.Concat(conv2,upconv2,upsample_pr3to2)
pr2=mx.sym.Convolution(iconv2,pad=(1,1),kernel=(3,3),stride=(1,1),num_filter=2,
name=name+'predict_flow2')
prediction['loss2']=pr2
flow=mx.sym.UpSampling(arg0=pr2,scale=4,num_filter=2,num_args=1,sample_type='nearest',name='upsample_flow2_to_1')
#ignorethelossfunctionswithlossscaleofzero
keys=loss_scale.keys()
#keys.sort()
#obtainthesymbolofthelosses
forkeyinkeys:
#loss.append(get_loss(prediction[key]*20,labels[key],loss_scale[key],name=key+name,get_input=False,is_sparse=is_sparse,type='flow'))
loss.append(mx.sym.MAERegressionOutput(data=prediction[key]*20,label=labels[key],name=key+name,grad_scale=loss_scale[key]))
#print('loss:',loss)
#group暂时不知道为嘛要group
loss_group=mx.sym.Group(loss)
#print('net:',loss_group)
returnloss_group,flow
importgluonbookasgb
importtorch
fromutils.frame_utilsimport*
importnumpyasnp
if__name__=='__main__':
checkpoint=torch.load("C:/Users/junjie.huang/PycharmProjects/flownet2_mxnet/flownet2_pytorch/FlowNet2-S_checkpoint.pth.tar")
##checkpoint是一个字典
print(isinstance(checkpoint['state_dict'],dict))
##打印checkpoint字典中的key名
print('keysofcheckpoint:')
foriincheckpoint:
print(i)
print('')
##pytorch模型参数保存在一个key名为'state_dict'的元素中
state_dict=checkpoint['state_dict']
##state_dict也是一个字典
print('keysofstate_dict:')
foriinstate_dict:
print(i)
#print(state_dict[i].size())
print('')
#print(state_dict)
#字典的value是torch.tensor
print(torch.is_tensor(state_dict['conv1.0.weight']))
#查看某个value的size
print(state_dict['conv1.0.weight'].size())
#flownet-mxnetinit
loss_scale={'loss2':1.00,
'loss3':1.00,
'loss4':1.00,
'loss5':1.00,
'loss6':1.00}
loss,flow=flownet_s(loss_scale=loss_scale,is_sparse=False)
print('lossinformation:')
print('loss:',loss)
print('type:',type(loss))
print('list_arguments:',loss.list_arguments())
print('list_outputs:',loss.list_outputs())
print('list_inputs:',loss.list_inputs())
print('')
print('flowinformation:')
print('flow:',flow)
print('type:',type(flow))
print('list_arguments:',flow.list_arguments())
print('list_outputs:',flow.list_outputs())
print('list_inputs:',flow.list_inputs())
print('')
name_mxnet=symbol.list_arguments()
print(type(name_mxnet))
forkeyinname_mxnet:
print(key)
name_mxnet.sort()
forkeyinname_mxnet:
print(key)
print(name_mxnet)
shapes=(1,3,384,512)
ctx=gb.try_gpu()
#exe=symbol.simple_bind(ctx=ctx,img1=shapes,img2=shapes)
exe=flow.simple_bind(ctx=ctx,img1=shapes,img2=shapes)
print('exetype:',type(exe))
print('exe:',exe)
#module
#mod=mx.mod.Module(flow)
#print('modtype:',type(exe))
#print('mod:',exe)
pim1=read_gen("C:/Users/junjie.huang/PycharmProjects/flownet2_mxnet/data/0000007-img0.ppm")
pim2=read_gen("C:/Users/junjie.huang/PycharmProjects/flownet2_mxnet/data/0000007-img1.ppm")
print(pim1.shape)
'''使用pytorch的state_dict初始化mxnet模型参数'''
forkeyinstate_dict:
#print(type(key))
k_split=key.split('.')
key_mx='_'.join(k_split)
#print(key,key_mx)
try:
exe.arg_dict[key_mx][:]=state_dict[key].data
except:
print(key,exe.arg_dict[key_mx].shape,state_dict[key].data.shape)
exe.arg_dict['img1'][:]=pim1[np.newaxis,:,:,:].transpose(0,3,1,2).data
exe.arg_dict['img2'][:]=pim2[np.newaxis,:,:,:].transpose(0,3,1,2).data
result=exe.forward()
print('result:',type(result))
#fortmpinresult:
#print(type(tmp))
#print(tmp.shape)
#color=flow2color(exe.outputs[0].asnumpy()[0].transpose(1,2,0))
outputs=exe.outputs
print('outputtype:',type(outputs))
#fortmpinoutputs:
#print(type(tmp))
#print(tmp.shape)
#来自pytrochflownet2
fromvisualizeimportflow2color
#color=flow2color(exe.outputs[0].asnumpy()[0].transpose(1,2,0))
flow_color=flow2color(exe.outputs[0].asnumpy()[0].transpose(1,2,0))
print('colortype:',type(flow_color))
importmatplotlib.pyplotasplt
#来自pytorch
fromtorchvision.transformsimportToPILImage
TF=ToPILImage()
images=TF(flow_color)
images.show()
#plt.imshow(color)
以上这篇tensorflow模型转ncnn的操作方式就是小编分享给大家的全部内容了，希望能给大家一个参考，也希望大家多多支持毛票票。