当前位置:网站首页>【mindspore】【训练警告】执行训练代码时存在的警告
【mindspore】【训练警告】执行训练代码时存在的警告
2022-07-20 14:44:00 【小乐快乐】
1、使用Ascend训练efficientnet的时候出现以下警告,这里我怀疑是mindspore环境更新导致的(环境从mindspore1.0.1更新到了mindspore1.1.1),当然其中的一些警告是之前的见过的,也有一些是没有见过的。想了解一下这些警告出现的原因以及有什么解决办法吗?
2、使用之前环境训练得到的训练模型(ckpt文件)换了更新后的环境(mindspore1.1.1)无法加载之前的模型进行继续训练或者验证结果(精度)。
【操作步骤&问题现象】
1、针对上面描述的问题1,使用更新后的环境mindspore1.1.1运行训练代码时,会出现如下警告,其中同类警告我已经忽略:
【1】WARNING: 'ControlDepend' is deprecated from version 1.1 and will be removed in a future version, use 'Depend' instead.
【2】[WARNING] ME(8272:140026733385536,MainProcess):2021-02-17-09:45:52.440.210 [mindspore/ops/operations/array_ops.py:2302] WARN_DEPRECATED: The usage of Pack is deprecated. Please use Stack.
【3】[WARNING] DEVICE(8272,python):2021-02-17-09:47:03.284.745 [mindspore/ccsrc/runtime/device/ascend/kernel_select_ascend.cc:282] TagRaiseReduce] node:[DropoutGenMask]reduce precision from int64 to int32
【4】[WARNING] SESSION(8272,python):2021-02-17-09:47:19.915.525 [mindspore/ccsrc/backend/session/ascend_session.cc:1412] SelectKernel] There are 1 node/nodes used reduce precision to selected the kernel!
【5】[WARNING] KERNEL(8272,python):2021-02-17-09:47:47.034.963 [mindspore/ccsrc/backend/kernel_compiler/tbe/tbe_utils.cc:58] SaveJsonInfo] Create info file failed(./kernel_meta/te_fusion_conv2d_backprop_input_d_tile_d_7453389728167276878_0.info).
【6】[WARNING] KERNEL(8272,python):2021-02-17-09:47:47.081.467 [mindspore/ccsrc/backend/kernel_compiler/tbe/tbe_utils.cc:58] SaveJsonInfo] Create info file failed(./kernel_meta/te_fusion_depthwise_conv2d_bn_training_reduce_7468152902432878638_0.info).
【7】[WARNING] DEVICE(8272,python):2021-02-17-09:50:23.020.968 [mindspore/ccsrc/runtime/device/kernel_runtime.cc:339] AssignStaticMemoryInput] It is not suggested to use a lonely weight parameter as the output of graph
2、针对上面的问题2,使用就环境生成的模型文件(不要在意模型文件命名错误,只是代码命名的地方没改)和完全不变的代码加载运行,会报错,错误信息如下,尝试过旧环境加载运行没有问题,我这里怀疑是不是mindspore在更新后存在不兼容,某些算子(我这里出现问题的是卷积算子mindspore.nn.conv2d())计算方式已经修改就是从NCHW修改为了CNHW?你们觉得我的判断是正确的吗?
WARNING: 'ControlDepend' is deprecated from version 1.1 and will be removed in a future version, use 'Depend' instead.
[WARNING] ME(5604:140668626671424,MainProcess):2021-02-17-09:44:27.437.987 [mindspore/ops/operations/array_ops.py:2302] WARN_DEPRECATED: The usage of Pack is deprecated. Please use Stack.
[ERROR] ME(5604:140668626671424,MainProcess):2021-02-17-09:44:32.796.212 [mindspore/train/serialization.py:84] Failed to combine the net and the parameters for param blocks.0.layers.0.depth_wise_conv.0.weight.
Traceback (most recent call last):
File "train.py", line 138, in <module>
load_param_into_net(net, ckpt)
File "/root/miniconda3/envs/ci3.7/lib/python3.7/site-packages/mindspore/train/serialization.py", line 373, in load_param_into_net
_update_param(param, new_param)
File "/root/miniconda3/envs/ci3.7/lib/python3.7/site-packages/mindspore/train/serialization.py", line 87, in _update_param
raise RuntimeError(msg)
RuntimeError: Net parameters blocks.0.layers.0.depth_wise_conv.0.weight shape((32, 1, 3, 3)) different from parameter_dict's((1, 32, 3, 3))
【截图信息】
对于问题一:
对于问题二:
报错信息为:net里的权重shape和ckpt里的不一致.
net中conv.0.weight的shape为(32, 1, 3, 3),checkpoint里conv.0.weight的shape为(1, 32, 3, 3)
如果网络定义相同的话,是不是跨后端了,比如GPU训练好的checkpoint拿到了Ascend进行推理。
解决方案为:
1、同硬件训练和推理
2、转化conv.0.weight的shape,使ckpt和net的shape匹配
边栏推荐
- About the basic setup of win7/win10 system installation
- iperf详细使用方法
- Why does cloud speed live broadcast win the favor of enterprises?
- [harmonyos] [arkui] Hongmeng linear gradient to achieve gradient, how to dynamically set it? I tried it for your reference
- 【mindspore】【模式】PYNATIVE_MODE模式和GRAPH模式的区别
- [composition principle V system bus]
- One of the learning tutorials of SAP Hana cloud: how to create an instance of SAP Hana cloud on SAP BTP
- Tdengine数据库运行出现的bug
- DOS汇编程序提高练习
- 【Mindspore-ascend】【自定义算子】重复地对一个Tensor赋值为什么会影响另一个Tensor?
猜你喜欢
随机推荐
Automated testing methods for SAP ui5 applications
C#递归获取文件夹下所有文件 并绑定到 TreeView控件中
How to use node JS access data in SAP Hana cloud database
LayoutInflater 布局渲染工具
【argoverse】argoverse-api 安装
【阿里云服务器】
Jackson parsing JSON data ignore parsing field annotation @jsonignoreproperties
多线程进阶(下)
Record Locks(记录锁)
[Bert] model return value analysis
Improt静态加载 动态加载
Compréhension approfondie de la haute concurrence
Access SAP Hana cloud instance in SAP Business Application studio
移动端 触摸事件
线性电路特性的研究与multisim仿真(附工程文件)
常用的锂电池升压IC
js 数组reduce方法求和 求最大值 求最小值方法
c语言力扣第二题之链表两数相加。遍历法
Binary number inversion (C language)
About whether the processing of SSR will continue after SAP Spartacus CSR fallback